Methods, apparatuses, and media for video processing

Abstract

Embodiments of the present disclosure provide a solution for video processing. A method for video processing is proposed. The method comprises determining, for a conversion between a video unit of a video and a bitstream of the video, whether to apply at least one collocated picture constraint or motion vector scaling process to a candidate derivation or coding information storage associated with the video unit; and performing the conversion based on the determination.

Description

Technical Field

[0001] The embodiments of this disclosure generally relate to video processing techniques, and more specifically, to temporal buffers, CTU buffers, and temporal block locations in video encoding and decoding. Background Technology

[0002] Today, digital video capabilities are being applied to all aspects of people's lives. Various video compression technologies have been proposed for video encoding / decoding, such as MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4 Part 10 Advanced Video Codec (AVC), ITU-T H.265 High Efficiency Video Codec (HEVC) standard, and Multi-Functional Video Codec (VVC) standard. However, the encoding and decoding efficiency of video encoding and decoding technologies is generally expected to be further improved. Summary of the Invention

[0003] Embodiments of this disclosure provide a solution for video processing.

[0004] In a first aspect, a method for video processing is proposed. The method includes: for a conversion between video units and a bitstream of the video, determining whether to apply at least one isotopic image constraint or motion vector scaling process to candidate derivation or encoding / decoding information storage associated with the video unit; and performing the conversion based on the determination. The method according to the first aspect of this disclosure can advantageously improve encoding / decoding efficiency and quality.

[0005] In a second aspect, an apparatus for video processing is provided. The apparatus includes a processor and a non-transitory memory having instructions thereon. When executed by the processor, the instructions cause the processor to perform the method according to the first aspect of this disclosure.

[0006] In a third aspect, a non-transitory computer-readable storage medium is proposed. This non-transitory computer-readable storage medium stores instructions that cause a processor to execute the method according to the first aspect of this disclosure.

[0007] In a fourth aspect, another non-transitory computer-readable recording medium is proposed. This non-transitory computer-readable recording medium stores a bitstream of video generated by a method performed by a video processing apparatus. The method includes: determining whether to apply at least one isotopic picture constraint or motion vector scaling process to candidate derivation or encoding / decoding information storage associated with video units of the video; and generating a bitstream based on the determination.

[0008] In the fifth aspect, a method for storing a bitstream of video is proposed. The method includes: determining whether to apply at least one isotopic image constraint or motion vector scaling process to the storage of candidate derivation or encoding / decoding information associated with video units of the video; generating a bitstream based on the determination; and storing the bitstream in a non-transitory computer-readable recording medium.

[0009] This synopsis aims to present, in a simplified form, the selected concepts further described below in the detailed embodiments. This synopsis is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Attached Figure Description

[0010] The above and other objects, features and advantages of exemplary embodiments of the present disclosure will become clearer from the following detailed description with reference to the accompanying drawings, in which the same reference numerals generally refer to the same parts.

[0011] Figure 1 A block diagram of an example video codec system according to some embodiments of the present disclosure is shown; Figure 2 A block diagram of a first example video encoder according to some embodiments of the present disclosure is shown; Figure 3 A block diagram of an example video decoder according to some embodiments of the present disclosure is shown; Figure 4 This diagram illustrates the effect of the slope adjustment parameter "u". Left: Model created using the current CCLM. Right: Updated model as proposed. Figure 5 The neighboring blocks (L, A, BL, AR, AL) used in the derivation of the general MPM list are shown. Figure 6 The neighbor reconstructed sample points used for the DIMD chromaticity mode are shown; Figure 7 The intra-frame template matching search area used is shown; Figure 8 The use of the IntraTMP block vector for IBC blocks is shown; Figure 9A and Figure 9B The method for dividing angle patterns is shown; Figure 10 An expanded list of MRL candidates is shown; Figure 11 A schematic diagram of the template area is shown; Figure 12 The spatial portion of the convolution filter is shown; Figure 13The reference region (and its padding) used to derive the filter coefficients is shown. Figure 14 Four Sobel-based gradient modes for GLM are shown; Figure 15 The non-downsampled brightness samples are shown; Figure 16 The reference area for BVG-CCCM is shown; Figure 17 Spatial sampling points used for GL-CCCM are shown; Figure 18 Various downsampling filters used in cross-component models are shown; Figure 19 The filter on the sample points of MM-CCLM / MM-CCCM is shown; Figure 20 The spatial GPM candidates are shown; Figure 21 The GPM template is shown; Figure 22 GPM mixing is shown; Figure 23 The transformation selection process for directional planar modes is illustrated; Figure 24 A luminance block is shown for deriving the direct block vector; Figure 25 The spatial neighbor block used to derive spatial merge candidates is shown; Figure 26 This demonstrates the generation of sub-block templates for SbTMVP; Figure 27 This demonstrates template matching execution over the search area surrounding the initial MV; Figure 28 The diamond-shaped area in the search region is shown; Figure 29 An example template is shown; Figure 30 The first HPT and the second HPT are shown; Figure 31A The candidate spatial neighbors used to derive inheritance are shown; Figure 31B Candidate spatial neighbors for deriving the first type of construction are shown; Figure 32 Affine Merge / AMVP candidates are shown, ranging from non-nearest neighbors to the first type of construction; Figure 33 The frequency response of the interpolation filter and the VVC interpolation filter at half-pixel phase is shown; Figure 34The template and its reference sample points in the reference image are shown; Figure 35 The template and reference sample points of the template are shown for a block with sub-block motion information using the current block; Figure 36 Additional directions along a diagonal angle of k×π / 8 are shown (the positions of the black-filled circles are used in the anchor points); Figure 37 The neighboring 4x4 sub-blocks used for RMVF parameter derivation are shown. W and H are the width and height of the current CU; Figure 38 The ramp function for the weights used in GPM blending is shown, which is based on the displacement (d) from the predicted sample location to the GPM split boundary and the blending region size (τ). Figures 39A to 39D GPM with inter-frame and intra-frame prediction is shown; Figure 40 The edges on the template are shown; Figure 41 The filling candidates for replacing the zero vector in the IBC list are shown; Figure 42 Candidate IBC clusters based on L2 distance and TM cost are shown; Figure 43 The IBC reference area is shown, depending on the current CU location; Figure 44 The reference region for IBC is shown when CTU(m,n) is encoded and decoded; Figure 45 The prediction of BVD is shown; Figure 46 The boundary filling method for motion compensation is shown; Figure 47 An example of deriving an M×4 fill block with a left fill direction is shown; Figure 48A and Figure 48B The BV adjustments for horizontal and vertical flips are shown separately. Figure 49 The InterCCCM method on the decoder is shown; Figure 50 The luminance samples L0, ..., L5 are shown relative to the chromaticity sample C; Figure 51 An example of the derivation of CMVP candidates is shown; Figure 52 This illustrates how to reference the source and destination of an MV in a CMVP; Figure 53A flowchart of a method for video processing according to embodiments of the present disclosure is shown; and Figure 54 A block diagram of a computing device in which various embodiments of the present disclosure may be implemented is shown.

[0012] In all accompanying drawings, the same or similar reference numerals usually refer to the same or similar elements. Detailed Implementation

[0013] The principles of this disclosure will now be described with reference to some embodiments. It should be understood that these embodiments are described for illustrative purposes only and to help those skilled in the art understand and implement this disclosure, and do not imply any limitation on the scope of this disclosure. In addition to the methods described below, the disclosure described herein can be implemented in various other ways.

[0014] In the following description and claims, unless otherwise defined, all scientific and technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0015] The terms "an embodiment," "embodiment," "example embodiment," etc., used in this disclosure refer to embodiments that may include specific features, structures, or characteristics, but not every embodiment is required to include that specific feature, structure, or characteristic. Furthermore, these phrases do not necessarily refer to the same embodiment. Additionally, when a specific feature, structure, or characteristic is described in conjunction with an example embodiment, whether explicitly described or not, it is believed that such a feature, structure, or characteristic affecting its relation to other embodiments is within the knowledge of those skilled in the art.

[0016] It should be understood that although the terms “first” and “second”, etc., can be used to describe various elements, these elements should not be limited to these terms. These terms are used only to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element, without departing from the scope of the exemplary embodiments. As used herein, the term “and / or” includes any and all combinations of one or more of the listed terms.

[0017] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments. As used herein, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprising,” “including,” “having,” “containing,” and / or “comprising” as used herein indicate the presence of the said features, elements, and / or components, but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof.

[0018] Example Environment Figure 1 This is a block diagram illustrating an example video encoding / decoding system 100 from which the techniques of this disclosure may be utilized. As shown, the video encoding / decoding system 100 may include a source device 110 and a destination device 120. The source device 110 may also be referred to as a video encoding device, and the destination device 120 may also be referred to as a video decoding device. In operation, the source device 110 may be configured to generate encoded video data, and the destination device 120 may be configured to decode the encoded video data generated by the source device 110. The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0019] Video source 112 may include sources such as video capture devices. Examples of video capture devices include, but are not limited to, interfaces for receiving video data from video content providers, computer graphics systems for generating video data, and / or combinations thereof.

[0020] Video data may include one or more images. Video encoder 114 encodes the video data from video source 112 to generate a bitstream. The bitstream may include a sequence of bits forming a codec representation of the video data. The bitstream may include codec images and associated data. The codec images are codec representations of images. The associated data may include sequence parameter sets, image parameter sets, and other syntax structures. I / O interface 116 may include a modulator / demodulator and / or a transmitter. Encoded video data can be directly transmitted to destination device 120 via network 130A through I / O interface 116. Encoded video data may also be stored on storage medium / server 130B for access by destination device 120.

[0021] The destination device 120 may include an I / O interface 126, a video decoder 124, and a display device 122. The I / O interface 126 may include a receiver and / or a modem. The I / O interface 126 may acquire encoded video data from the source device 110 or the storage medium / server 130B. The video decoder 124 may decode the encoded video data. The display device 122 may display the decoded video data to a user. The display device 122 may be integrated with the destination device 120, or it may be external to the destination device 120, which is configured to interface with an external display device.

[0022] The video encoder 114 and the video decoder 124 can operate according to video compression standards, such as the High Efficiency Video Codec (HEVC) standard, the Multi-Functional Video Codec (VVC) standard, and other existing and / or further standards.

[0023] Figure 2This is a block diagram illustrating an example of a video encoder 200 according to some embodiments of the present disclosure. The video encoder 200 may be... Figure 1 An example of a video encoder 114 in system 100 is shown.

[0024] The video encoder 200 can be configured to implement any or all of the technologies disclosed herein. Figure 2 In the example, the video encoder 200 includes multiple functional components. The techniques described in this disclosure can be shared among the various components of the video encoder 200. In some examples, the processor can be configured to perform any or all of the techniques described in this disclosure.

[0025] In some embodiments, the video encoder 200 may include a segmentation unit 201, a prediction unit 202, a residual generation unit 207, a transform unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213, and an entropy coding unit 214. The prediction unit 202 may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205, and an intra-frame prediction unit 206.

[0026] In other examples, the video encoder 200 may include more, fewer, or different functional components. In one example, the prediction unit 202 may include an intra-block copy (IBC) unit. The IBC unit can perform prediction in an IBC mode, where at least one reference picture is the picture in which the current video block is located.

[0027] Furthermore, although some components (such as motion estimation unit 204 and motion compensation unit 205) can be integrated, for interpretable purposes, these components are... Figure 2 The examples are shown separately.

[0028] The segmentation unit 201 can segment an image into one or more video blocks. The video encoder 200 and the video decoder 300 can support various video block sizes.

[0029] The mode selection unit 203 can select one of several codec modes (intra-frame codec or inter-frame codec) based, for example, on the error result, and provide the resulting intra-frame or inter-frame codec block to the residual generation unit 207 to generate residual block data, and to the reconstruction unit 212 to reconstruct the coded block for use as a reference image. In some examples, the mode selection unit 203 can select an intra-frame / inter-frame joint prediction (CIIP) mode, where prediction is based on inter-frame prediction signals and intra-frame prediction signals. In the case of inter-frame prediction, the mode selection unit 203 can also select a resolution for the block based on the motion vector (e.g., sub-pixel precision or integer pixel precision).

[0030] To perform inter-frame prediction on the current video block, motion estimation unit 204 can generate motion information for the current video block by comparing one or more reference frames from buffer 213 with the current video block. Motion compensation unit 205 can determine the predicted video block for the current video block based on the motion information and decoded samples of images from buffer 213 other than the image associated with the current video block.

[0031] The motion estimation unit 204 and the motion compensation unit 205 can perform different operations on the current video block, for example, depending on whether the current video block is in an I-strip, P-strip, or B-strip. As used herein, an "I-strip" can refer to a portion of an image composed of macroblocks, all of which are based on macroblocks within the same image. Furthermore, as used herein, in some aspects, "P-strip" and "B-strip" can refer to portions of an image composed of macroblocks that do not depend on macroblocks within the same image.

[0032] In some examples, motion estimation unit 204 can perform unidirectional prediction on the current video block, and can search reference images in list 0 or list 1 to find a reference video block for the current video block. Motion estimation unit 204 can then generate a reference index indicating the reference image containing the reference video block in list 0 or list 1, and a motion vector indicating the spatial displacement between the current video block and the reference video block. Motion estimation unit 204 can output the reference index, prediction direction indicator, and motion vector as motion information for the current video block. Motion compensation unit 205 can generate a predicted video block for the current video block based on the reference video block indicated by the motion information of the current video block.

[0033] Alternatively, in other examples, motion estimation unit 204 can perform bidirectional prediction on the current video block. Motion estimation unit 204 can search for reference images in list 0 to find a reference video block for the current video block, and can also search for reference images in list 1 to find another reference video block for the current video block. Motion estimation unit 204 can then generate a reference index and a motion vector, the reference index indicating the reference images containing the reference video blocks in lists 0 and 1, and the motion vector indicating the spatial displacement between the reference video block and the current video block. Motion estimation unit 204 can output the reference index and motion vector of the current video block as motion information for the current video block. Motion compensation unit 205 can generate a predicted video block for the current video block based on the reference video blocks indicated by the motion information of the current video block.

[0034] In some examples, the motion estimation unit 204 can output a complete set of motion information for use in the decoder's decoding process. Alternatively, in some embodiments, the motion estimation unit 204 can reference the motion information of another video block to transmit the motion information of the current video block via a signal. For example, the motion estimation unit 204 can determine that the motion information of the current video block is sufficiently similar to the motion information of neighboring video blocks.

[0035] In one example, the motion estimation unit 204 may indicate a value to the video decoder 300 in the syntax structure associated with the current video block, which indicates that the current video block has the same motion information as another video block.

[0036] In another example, motion estimation unit 204 may identify another video block and motion vector difference (MVD) in the syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. Video decoder 300 can use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0037] As discussed above, the video encoder 200 can transmit motion vectors via signals in a predictive manner. Two examples of predictive signaling techniques that can be implemented by the video encoder 200 include Advanced Motion Vector Prediction (AMVP) and Merge Pattern Signaling.

[0038] Intra-prediction unit 206 can perform intra-prediction on the current video block. When intra-prediction unit 206 performs intra-prediction on the current video block, it can generate prediction data for the current video block based on decoded samples from other video blocks in the same frame. The prediction data for the current video block can include the predicted video block and various syntax elements.

[0039] The residual generation unit 207 can generate residual data for the current video block by subtracting (e.g., indicated by a minus sign) multiple predicted video blocks from the current video block. The residual data for the current video block can include residual video blocks corresponding to different sample components of the samples in the current video block.

[0040] In other examples, there may be no residual data for the current video block, such as in skip mode, and the residual generation unit 207 may not perform subtraction operations.

[0041] The transform processing unit 208 can generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to the residual video blocks associated with the current video block.

[0042] After the transform processing unit 208 generates a transform coefficient video block associated with the current video block, the quantization unit 209 can quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0043] The inverse quantization unit 210 and the inverse transform unit 211 can apply inverse quantization and inverse transform to the transform coefficient video block respectively to reconstruct the residual video block from the transform coefficient video block. The reconstruction unit 212 can add the reconstructed residual video block to the corresponding samples of one or more predicted video blocks generated by the prediction unit 202 to generate a reconstructed video block associated with the current video block for storage in the buffer 213.

[0044] After the video block is reconstructed by reconstruction unit 212, a loop filtering operation can be performed to reduce video block artifacts in the video block.

[0045] Entropy encoding unit 214 can receive data from other functional components of video encoder 200. When entropy encoding unit 214 receives data, it can perform one or more entropy encoding operations to generate entropy-encoded data and output a bitstream including the entropy-encoded data.

[0046] Figure 3 This is a block diagram illustrating an example of a video decoder 300 according to some embodiments of the present disclosure. The video decoder 300 may be... Figure 1 An example of video decoder 124 in system 100 is shown.

[0047] The video decoder 300 can be configured to perform any or all of the technologies disclosed herein. Figure 3 In the example, the video decoder 300 includes multiple functional components. The techniques described in this disclosure can be shared among the various components of the video decoder 300. In some examples, the processor can be configured to perform any or all of the techniques described in this disclosure.

[0048] exist Figure 3 In the example, the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra-frame prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306, and a buffer 307. In some examples, the video decoder 300 can perform a decoding process that is generally contrasted with the encoding process described with respect to the video encoder 200.

[0049] Entropy decoding unit 301 can retrieve the encoded bitstream. The encoded bitstream may include entropy-encoded video data (e.g., encoded video data blocks). Entropy decoding unit 301 can decode the entropy-encoded video data, and motion compensation unit 302 can determine motion information from the entropy-decoded video data, which includes motion vectors, motion vector precision, reference picture list indices, and other motion information. Motion compensation unit 302 can determine this information, for example, by performing AMVP and Merge mode. AMVP is used, which involves deriving several most likely candidates based on data from adjacent PBs and reference pictures. Motion information typically includes horizontal and vertical motion vector displacement values, one or two reference picture indices, and, in the case of a prediction region in a B-strip, an identifier of which reference picture list is associated with each index. As used herein, in some aspects, "Merge mode" may refer to deriving motion information from spatially or temporally adjacent blocks.

[0050] The motion compensation unit 302 can generate motion compensation blocks and can perform interpolation based on an interpolation filter. The identifier of the interpolation filter to be used with sub-pixel accuracy can be included in the syntax element.

[0051] The motion compensation unit 302 can use the interpolation filter used by the video encoder 200 during the encoding of the video block to calculate the interpolation for sub-integer pixels of the reference block. The motion compensation unit 302 can determine the interpolation filter used by the video encoder 200 based on the received syntax information, and the motion compensation unit 302 can use the interpolation filter to generate the prediction block.

[0052] Motion compensation unit 302 may use at least some of the syntax information to determine the block size of the frames(multiple) and / or stripes(multiple) used to encode the encoded video sequence, segmentation information describing how each macroblock of the image of the encoded video sequence is segmented, a pattern indicating how each segment is encoded, one or more reference frames (and a list of reference frames) for each inter-frame coded block, and other information for decoding the encoded video sequence. As used herein, in some aspects, a “strip” can refer to a data structure that can be decoded independently of other stripes of the same image in terms of entropy encoding / decoding, signal prediction, and residual signal reconstruction. A strip can be the entire image or a region of the image.

[0053] Intra-prediction unit 303 can use, for example, an intra-prediction mode received in the bitstream to form prediction blocks from spatially adjacent blocks. Inverse quantization unit 304 inverse quantizes (i.e., dequantizes) the quantized video block coefficients provided in the bitstream and decoded by entropy decoding unit 301. Inverse transform unit 305 applies the inverse transform.

[0054] The reconstruction unit 306 can obtain the decoded block, for example, by adding the residual block to the corresponding predicted block generated by the motion compensation unit 302 or the intra-frame prediction unit 303. If necessary, a deblocking filter can also be used to filter the decoded block to remove block artifacts. The decoded video block is then stored in a buffer 307, which provides a reference block for subsequent motion compensation / intra-frame prediction and also generates decoded video for presentation on a display device.

[0055] Some exemplary embodiments of this disclosure will be described in detail below. It should be understood that section headings are used in this document for ease of understanding and not to limit the embodiments disclosed in a section to that section only. Furthermore, although some embodiments are described with reference to multi-functional video codecs or other specific video codecs, the disclosed techniques are also applicable to other video codec techniques. Furthermore, although some embodiments describe video codec steps in detail, it will be understood that the corresponding decoding steps of the inverse codec will be implemented by the decoder. Additionally, the term "video processing" includes video codec or compression, video decoding or decompression, and video transcoding, wherein video pixels are represented from one compression format to another or at different compression bitrates.

[0056] 1 Introduction This disclosure relates to video codec technology. Specifically, it concerns buffering in image / video codecs. It can be applied to existing video codec standards such as HEVC, VVC, etc. It can also be applied to future video codec standards or video codecs.

[0057] 2 Introduction Video codec standards have primarily evolved through the development of well-known ITU-T and ISO / IEC standards. ITU-T developed the H.261 and H.263 standards, while ISO / IEC developed MPEG-1 and MPEG-4 Vision. The two organizations jointly developed the H.262 / MPEG-2 video standard, the H.264 / MPEG-4 Advanced Video Codec (AVC) standard, and the H.265 / HEVC standard. Starting with H.262, video codec standards are based on a hybrid video codec architecture, utilizing temporal prediction plus transform coding. To explore future video codec technologies beyond HEVC, the Joint Video Exploration Team (JVET) was established in 2015 by VCEG and MPEG. JVET meetings are held quarterly, and the new video codec standard was officially named Multifunctional Video Codec (VVC) at the April 2018 JVET meeting, where the first version of the VVC Test Model (VTM) was also released. The VVC working draft and the VTM test model are updated after each meeting. The VVC project achieved technical completion (FDIS) at a meeting in July 2020.

[0058] 2.1 Intra-frame prediction In intra-frame prediction, the minimum chroma intra-frame prediction unit (SCIPU) constraint in the VVC is removed. Additionally, the VPDU constraint used to reduce CCLM prediction latency is also removed.

[0059] 2.1.1 Multi-Model LM (MMLM) The CCLM included in VVC is extended by adding three multi-model LM (MMLM) modes (JVET-D0110). In each MMLM mode, using a threshold as the average of the luminance reconstructed neighboring samples, the reconstructed neighboring samples are classified into two categories. A linear model for each category is derived using the least mean square (LMS) method. For the CCLM mode, the linear model is also derived using the LMS method. A slope adjustment is applied to the cross-component linear model (CCLM) and multi-model LM predictions. This adjustment is a linear function that maps luminance values ​​to chrominance values, tilted relative to a center point determined by the average luminance values ​​of the reference samples.

[0060] 2.1.1.1 Slope Adjustment of CCLM CCLM uses a two-parameter model to map luminance values ​​to chrominance values. The slope parameter "a" and the bias parameter "b" define the mapping as follows: chromaVal = a LumaVal + b, The slope parameter "u" is adjusted via signal transmission to update the model to the following form: chromaVa = a' LumaVal + b', in a' = a + u b' = b - u y r .

[0061] By making this selection, the mapping function revolves around a value with brightness y. r The points are tilted or rotated. The average value of the reference brightness samples used in model creation is used as y. r This is to allow for meaningful modifications to the model. The image below illustrates this process.

[0062] Figure 4 The effect of the slope adjustment parameter "u" is shown. The left side shows the model created with the current CCLM, and the right side shows the updated model as proposed.

[0063] Implementation The slope adjustment parameter is provided as an integer between -4 and 4, including -4 and 4, and is transmitted via signal in the bitstream. The unit of the slope adjustment parameter is 1 / 8 of the chroma sample value per luminance sample value (for 10-bit content).

[0064] The CCLM model (“LM_CHROMA_IDX” and “MMLM_CHROMA_IDX”) is adjusted to apply to reference samples used on both the top and left sides of the block, but not to the “one-sided” mode. This choice is based on a trade-off between encoding / decoding efficiency and complexity.

[0065] When slope adjustment is applied to a multi-mode CCLM model, both models can be adjusted, so at most two slope updates are transmitted via signal transmission for a single chroma block.

[0066] Encoder method The proposed encoder method performs a SATD-based search for the optimal slope update for Cr and a similar SATD-based search for Cb. If either results in a non-zero slope adjustment parameter, the combined slope adjustment pair (SATD-based update for Cr, SATD-based update for Cb) is included in the list of RD checks for TU.

[0067] 2.1.2 Gradient PDPC In VVC, PDPC may not be applied in some cases due to the unavailability of auxiliary reference samples. In these cases, gradient-based PDPC (JVET-Q0391), extended from the horizontal / vertical mode, is applied. The PDPC weights (wT / wL) and nScale parameter, used to determine the decay of the PDPC weights relative to the distance from the left / top boundary, are set to equal the corresponding parameters in the horizontal / vertical mode, respectively. When the auxiliary reference sample is located at the fractional sample position, bilinear interpolation is applied.

[0068] 2.1.3 Primary MPM and Auxiliary MPM As described in JVET-D0114, an auxiliary MPM list is introduced. The existing main MPM (PMPM) list consists of 6 entries, and the auxiliary MPM (SMPM) list includes 16 entries. First, a general MPM list with 22 entries is constructed. Then, the first 6 entries from this general MPM list are included in the PMPM list, and the remaining entries form the SMPM list. The first entry in the general MPM list is a flat pattern. The remaining entries are... Figure 5 The intra-frame patterns of the left (L), top (A), bottom left (BL), top right (AR), and top left (AL) neighboring blocks, as well as DIMD patterns arranged in ascending order of SAD cost, are shown. A maximum of five patterns with the lowest SAD cost are added. The SAD cost is calculated between the predicted and reconstructed samples of the template. The sorted directional patterns with added offsets are added to the general MPM list, followed by the default pattern, until a general MPM list with 22 entries is constructed.

[0069] If the CU block is vertical, the order of the neighboring blocks is A, L, BL, AR, AL; otherwise, it is L, A, BL, AR, AL.

[0070] Figure 5 The neighboring blocks (L, A, BL, AR, AL) used in the derivation of the general MPM list are shown.

[0071] The MPM list is evenly divided into four groups, and the group index is parsed first. Then, the pattern index is further parsed to indicate which pattern in the selected group is used.

[0072] 2.1.4 Reference Sample Interpolation and Smoothing for Intra-Frame Prediction As described in JVET-D0119, the 4-tap cubic interpolation is replaced by a 6-tap cubic interpolation filter for deriving predicted samples from reference samples.

[0073] For reference sample filtering, a 6-tap Gaussian filter is applied to larger blocks (W>= 32 and H>= 32), otherwise the existing VVC 4-tap Gaussian interpolation filter is applied. Extended intra-frame reference samples are derived using a 4-tap interpolation filter instead of nearest-neighbor rounding.

[0074] 2.1.5 Decoder-side Intra-Frame Mode Derivation (DIMD) When DIMD is applied, at most five intra-frame modes are derived from the reconstructed neighboring samples, and these five predictions are combined with planar mode predictions, as described in JVET-O0449, where weights are derived from the gradient histogram. The division operations in the weight derivation are performed using the same lookup table (LUT)-based integerization scheme used by CCLM. For example, division operations in direction calculation.

[0075] It is computed using the following LUT-based scheme: x = Floor( Log2( Gx ) ) normDiff = ( ( Gx<<4 )>>x )&15 x +=( 3 + ( normDiff != 0 ) ? 1 : 0 ) Orient = (Gy (DivSigTable[normDiff] | 8) + (1<<(x-1)))>>x in DivSigTable

[16] = {0, 7, 6, 5,5, 4, 4, 3, 3, 2, 2, 1, 1, 1, 1, 0}.

[0076] For size If the magnitude of either the upper or left histogram is twice that of the other, then the weights for each of the five derived patterns are modified. In this case, the weights depend on the position and are calculated as follows: If the histogram above is twice the size of the histogram on the left, then: ; If the histogram on the left is twice the size of the histogram on the top, then: , in It is the unmodified uniform weight of the DIMD selected as in JVET-O0449. It is predefined and set to 10.

[0077] The derived intra-frame modes are included in the main list of most probable intra-frame modes (MPMs), so the DIMD process is performed before the MPM list is built. The main derived intra-frame modes of the DIMD block are stored with the block and used for the construction of the MPM lists of neighboring blocks.

[0078] Finally, note that compared to the JVET-O0449 method, the region used to compute the gradient histogram for neighboring reconstructed samples is modified, depending on the availability of reconstructed samples. If available, the region of the decoded reference sample for the current WxH brightness CB is expanded upwards and to the right, with a maximum of W additional columns. If available, it is expanded downwards and to the left, with a maximum of H additional rows.

[0079] 2.1.5.1 DIMD Chroma Mode The DIMD chromaticity mode uses the DIMD derivation method, based on, for example, Figure 6 The reconstructed Y, Cb, and Cr samples in the second nearest row and column are used to derive the chroma intra-prediction mode for the current block. Specifically, the horizontal and vertical gradients are calculated for each co-located reconstructed luminance sample and the reconstructed Cb and Cr samples of the current chroma block to construct the HoG. The intra-prediction mode with the largest histogram amplitude value is then used to perform chroma intra-prediction for the current chroma block.

[0080] Figure 6 The neighboring reconstructed samples used for the DIMD chromaticity mode are shown.

[0081] When the intra-prediction mode derived from the DIMD chroma mode is the same as the intra-prediction mode derived from the DM mode, the intra-prediction mode with the second largest histogram amplitude value is used as the DIMD chroma mode. A CU level flag is transmitted via signal transmission to indicate whether the proposed DIMD chroma mode is applied.

[0082] Finally, compared to JVET-O0449, the luminance regions of the reconstructed samples used to compute the gradient histogram of the chromaticity DIMD mode are modified. For the WxH chromaticity CB pair to be predicted, in order to construct the gradient histogram associated with the corresponding luminance CB, the vertical and horizontal gradient pairs are extracted from the second and third rows of that luminance CB, instead of from the regular set of decoded DIMD reference samples surrounding that luminance CB.

[0083] 2.1.6 Fusion of Chroma Intra-Frame Prediction Modes In ECM, two chroma intra-prediction signals can be fused together. One of the two chroma intra-prediction signals is predicted using DM mode, DIMD chroma mode, and one of the four default modes (non-LM mode). The other chroma intra-prediction signal is predicted using the cross-component linear prediction mode (LM mode). Two different methods are supported.

[0084] In the first method, the LM mode can be either MM-CCLM or MM-CCCM, and the final predicted value is derived as follows: , in These are predicted values ​​obtained by applying a non-LM model. These are predicted values ​​obtained by applying the LM model, and This is the final predicted value for the current chroma block. Two weights. and Determined by the intra-prediction mode of adjacent chroma blocks, and It is set to equal to 2. Specifically, when both the upper adjacent block and the left adjacent block are encoded and decoded using LM mode, { }={1, 3}; When both the upper adjacent block and the left adjacent block are encoded and decoded using non-LM mode, { }={3, 1}; otherwise, { }={2, 2}. Two template costs are calculated by fusing the angular chromaticity prediction with either MM-CCLM or MM-CCCM, and one of the two CCPs providing the smaller template cost is used to derive the... .

[0085] In the second method, the LM mode can be either MMLM or CCLM, and the final predicted value is derived as follows: , in These are predicted values ​​obtained by applying a non-LM model. It is the set of reconstructed brightness samples downsampled at the same location, and It is the final predicted value of the current chroma block. It is a fixed value, and is set to 512 for 10-bit content. Three weights. , and The LDL is derived from adjacent luminance and chrominance samples using the same LDL derivation method as in CCCM.

[0086] For syntax design, an index is transmitted via signaling to indicate whether fusion is applied and which method is used. It's important to note that for I-stripes, non-LM modes can be DM mode, DIMD chroma mode, and four default modes. For non-I-stripes, only DIMD chroma mode is allowed to be fused with LM mode.

[0087]

[0088] 2.1.7 Intra-frame template matching Intra-Template Matching Prediction (IntraTMP) is a special intra-prediction mode that copies the best prediction block from the reconstructed portion of the current frame, whose L-shaped template matches the current template. For a predefined search range, the encoder searches the reconstructed portion of the current frame for the template most similar to the current template and uses the corresponding block as the prediction block. The encoder then transmits the use of this mode via signal transmission and performs the same prediction operation on the decoder side.

[0089] By comparing the L-shape of the current block with, only the top or only the left causal nearest neighbors Figure 7 The prediction signal is generated by matching another block within a predefined search region. There are 6 predefined search regions, namely... Figure 7 R1 to R6 in the CTU contain reconstruction samples from the top CTU and the left CTU, as well as partial reconstruction samples located above, to the left, below the left, and above the right of the current block within the current CTU.

[0090] The sum of absolute differences (SAD) is used as the cost function.

[0091] Using a given search order across six regions, R4, R5, R6, R1, R2, and R3, the decoder constructs a candidate list of up to 19 template-matched block vectors within each region, ordered in ascending order based on template cost (SAD). The following modes are supported: 1- Single Predicted Value: Select a single predicted value from the candidate list.

[0092] 2- Fusion of Multiple Predictions: Multiple predictions are blended multiple times to derive the final prediction block. The blending weights are either calculated from the template matching cost of each prediction or using a weight derivation method based on Wiener filters.

[0093] 3- Subpixel precision: When using a single prediction value, subpixel precision can be used at 1 / 2 pixel precision, 1 / 4 pixel precision, and 3 / 4 pixel precision, with each precision having 8 possible directions.

[0094] 4- Linear Filter Model: A linear filter can be learned between the reference template and the current template, and the linear model is applied to the reference block. This mode can be used for a single prediction when sub-pixel precision is not used.

[0095] The dimensions of all regions (SearchRange_w, SearchRange_h) are set to be proportional to the block dimensions (BlkW, BlkH) to have a fixed number of SAD comparisons per pixel. That is: SearchRange_w = min(64,a BlkW) SearchRange_h = min(64,a BlkH), in" "" is a constant that controls the trade-off between gain and complexity. In practice, " "Equals 5."

[0096] To accelerate the template matching process, the search range of all search regions is downsampled by a factor of 3. After finding the best match, a refinement process is performed. Refinement is accomplished through a second template matching search around the best match with a reduced range.

[0097] For CUs with width and height dimensions less than or equal to 64, the intra-frame template matching tool is enabled. This maximum CU size used for intra-frame template matching is configurable.

[0098] When DIMD is not used for the current CU, the intra-template matching prediction mode is transmitted at the CU level via a dedicated flag.

[0099] 2.1.7.1 Block Vector Candidates Derived for IntraTMP in IBC In this method, the block vector (BV) derived from IntraTMP (Intra-Temporal Matching Prediction) is used for IntraBlock Copy (IBC). The stored IntraTMP BV and IBC BV of neighboring blocks are used as spatial BV candidates in the construction of the IBC candidate list.

[0100] The IntraTMP block vector is stored in the IBC block vector cache, and the current IBC block can use both the IBC BV and the IntraTMP BV of neighboring blocks as BV candidates for the IBC BV candidate list, such as... Figure 8 As shown.

[0101] IntraTMP block vectors are added to the IBC block vector candidate list as spatial domain candidates.

[0102] 2.1.8 Fusion for Template-Based Intra-Frame Mode Derivation (TIMD) For each intra-prediction mode in the MPM, and for the wide-angle mode if upper-right and / or lower-left reference samples are available, the SATD between the predicted and reconstructed samples of the template is calculated. The top two intra-prediction modes with the minimum SATD are selected as TIMD modes. These two TIMD modes are fused using weights after applying the PDPC procedure, and this weighted intra-prediction is used for encoding and decoding the current CU. Position-dependent intra-prediction combination (PDPC) is included in the derivation of the TIMD modes.

[0103] The costs of the two selected modes are compared with a threshold, and a cost factor of 2 is applied in the test, as shown below: costMode2<2 costMode1.

[0104] If the condition is true, then apply fusion; otherwise, use only mode 1.

[0105] The weights of the patterns are calculated from their SATD costs as follows: weight1 = costMode2 / (costMode1+ costMode2) weight2 = 1 - weight1.

[0106] Division operations are performed using the same lookup table (LUT)-based integerization scheme used by CCLM.

[0107] 2.1.9 Intra-frame prediction fusion This intra-frame prediction method derives predicted samples as a weighted combination of multiple predicted values ​​generated from different reference rows. In this process, multiple intra-frame predicted values ​​are generated and then fused by a weighted average. The process of deriving the predicted values ​​to be used in the fusion process is described below: • For the intra-angle prediction mode in the single-mode case including TIMD and DIMD, the proposed method improves upon the representation as... Intra-prediction is derived by weighting the intra-prediction obtained from multiple reference rows, where It is an intra-frame prediction from the default reference line, and This is a prediction from the row above the default reference row. The weights are set to... and .

[0108] • For TIMD modes with hybrid patterns, Used in the first mode ( ),and Used in the second mode ( ).

[0109] • For DIMD patterns with a mixture, the number of forecasts selected for weighted averaging increases from 3 to 6.

[0110] When the intra-frame prediction mode has a non-integer slope (required reference sample interpolation) and the block size is greater than 16, the intra-frame prediction fusion method is applied to the luma block, used together with MRL, but not to the ISP-encoded block. In the method studied in subtest a, PDPC is applied to the intra-frame prediction mode using the reference line closest to the current block.

[0111] 2.1.10 Combination of CIIP with TIMD and TM Merge In CIIP mode, prediction samples are generated by weighting the inter-frame prediction signal using CIIP-TM Merge candidate prediction and the intra-frame prediction signal using the intra-frame prediction mode derived using TIMD. This method is only applied to codec blocks with an area less than or equal to 1024.

[0112] The TIMD derivation method was used to derive intra-prediction modes in CIIP. Specifically, the intra-prediction mode with the smallest SATD value was selected from the TIMD mode list and mapped to one of 67 regular intra-prediction modes.

[0113] Furthermore, it is proposed that if the derived intra-prediction mode is an angled mode, the weights (wIntra, wInter) for the two tests should be modified. For near-horizontal mode (2 <= angled mode index < 34), the current block is vertically divided, as shown below. Figure 9A As shown; for near-vertical mode (34 <= angle mode index <= 66), the current block is divided horizontally, as... Figure 9B As shown.

[0114] For different sub-blocks (wIntra, wInter) such as Figure 9A and Figure 9B As shown.

[0115] Table 1. Weights used for modifications to the angle mode.

[0116]

[0117] Using CIIP-TM, a CIIP-TM Merge candidate list is constructed for the CIIP-TM pattern. Merge candidates are refined through template matching. CIIP-TM Merge candidates are also reordered as regular Merge candidates using the ARMC method. The maximum number of CIIP-TM Merge candidates is 2.

[0118] 2.1.11 Extended Multi-Reference Row (MRL) List The MRL list in VVC is expanded to include more reference lines for intra-frame prediction. The expanded reference line list consists of line indices {1, 3, 5, 7, 12}, as shown below. Figure 10 As shown in the figure. For template-based intra-frame mode derivation (TIMD), only the first two reference row candidates (i.e., {1, 3}) are used, instead of the complete MRL candidate list.

[0119] 2.1.12 Template-based multi-reference row intra-frame prediction Template-based multi-reference line intra-prediction (TMRL) mode combines reference lines and prediction modes, using template matching to construct a candidate combination list. The indices of the candidate combination list are encoded / decoded to indicate which reference line and prediction mode to use when encoding / decoding the current block. Regular multi-reference lines (MRL) in non-TIMD portions are replaced by the TMRL mode.

[0120] The TMRL mode expands the reference line candidate list and the intra-prediction mode candidate list. The expanded reference line candidate list is {1, 3, 5, 7, 12}. The restriction on the top CTU line remains unchanged. The size of the intra-prediction mode candidate list is 10. The construction of the intra-prediction mode candidate list is similar to that of MPM, except that planar modes are excluded from the intra-prediction mode candidate list, DC modes are added after the modes of the 5 neighboring PUs and DIMD modes (if they are not included), and has a range from... arrive An incremental angle pattern (compared to existing angle patterns in the intra-prediction pattern candidate list) has been added. The accuracy of angle prediction has been expanded from 65 to 129. Additionally, non-adjacent locations have been added as candidates when constructing the intra-prediction pattern candidate list. If adjacent or non-adjacent blocks are encoded / decoded using SGPM or GPM modes, the block's intra-prediction pattern is replaced by the split angle.

[0121] The TMRL candidates are constructed as follows. For each block, there are 5 x 10 = 50 combinations of extended reference lines and allowed intra-prediction modes. Since the extended reference lines start from reference line 1, the region covered by reference line 0 is used for template matching. The calculation between prediction (generated from the 50 combinations) and reconstruction is performed within the template region (see...). Figure 11 The SAD cost on the TMRL is calculated. The 20 combinations with the lowest SAD cost are selected in ascending order to form the TMRL candidate list.

[0122] For TMR signaling, instead of directly encoding and decoding the reference line and intra-frame mode, the index of the TMRL candidate list is encoded and decoded to indicate which combination of reference line and prediction mode is used to encode and decode the current block.

[0123] 2.1.13 Convolutional Cross-Component Intra-Frame Prediction Model In this method, a convolutional cross-component model (CCCM) is applied to predict chroma samples from reconstructed luminance samples, similar in spirit to what the current CCLM model does. As with CCLM, when chroma downsampling is used, the reconstructed luminance samples are downsampled to match a lower-resolution chroma grid. Similar to CCLM, top, left, or top and left reference samples are used as templates for model derivation.

[0124] Additionally, similar to CCLM, there are options for single-model or multi-model variants using CCCM. The multi-model variant uses two models: one derived for samples above the average luminance reference value, and the other for the remaining samples (following the spirit of the CCLM design). The multi-model CCCM mode can be selected for PUs with at least 128 available reference samples.

[0125] 2.1.13.1 Convolution Filter The convolutional 7-tap filter consists of a 5-tap plus-shaped spatial component, a nonlinear term, and a bias term. The input of the 5-tap spatial component of the filter consists of the center (C) luminance sample that is in the same position as the chrominance sample to be predicted, and its upper / north (N), lower / south (S), left / west (W), and right / east (E) neighbors, as shown in Figures 9(a) and 9(b).

[0126] The nonlinear term P is expressed as the square of the center luminance sample C and scaled to the range of sample values ​​for the content: P = (C C + midVal )>>bitDepth.

[0127] That is, for 10 bits of content, it is calculated as: P = (C C + 512)>>10.

[0128] The bias term B represents the scalar offset between the input and output (similar to the offset term in CCLM) and is set to an intermediate chroma value (512 for 10-bit content).

[0129] The output of the filter is calculated as the filter coefficients c. i The convolution with the input values ​​is then limited to the range of valid chromaticity samples: predChromaVal = c0C + c1N + c2S + c3E + c4W + c5P + c6B.

[0130] 2.1.13.2 Calculation of Filter Coefficients Filter coefficients c i The mean squared error (MSE) is calculated by minimizing the difference between the predicted chromaticity samples and the reconstructed chromaticity samples in the reference region. Figure 13 A reference region is shown, consisting of 2 or 6 rows of chroma samples above and to the left of the PU. Whether to use 6 or 2 rows of neighboring samples to derive CCCM model parameters in a single-model CCCM is determined by the template cost. Similarly, for a multi-model CCCM mode, two candidates are used to derive the mean that separates the samples into two groups using 6 rows of neighboring luma samples or luma samples co-located with the current chroma block. The cost is derived by applying the candidate CCP (2 or 6 rows) to the template and calculating the sum of absolute differences (SAD) between the CCP prediction samples and the reconstructed samples in the template.

[0131] The reference region extends one PU width to the right of the PU boundary and one PU height below it. The region is adjusted to include only available samples. An extension of the blue region is needed to support the "side samples" of the plus-shaped spatial filter and is used to fill in unavailable areas.

[0132] MSE minimization is performed by calculating the autocorrelation matrix for the luma input and the cross-correlation vector between the luma input and the chromaticity output. The autocorrelation matrix is ​​decomposed using LDL, and the final filter coefficients are calculated using inverse substitution. This process largely follows the calculation of ALF filter coefficients in ECM; however, LDL decomposition is chosen instead of Cholesky decomposition to avoid the use of square root operations.

[0133] The autocorrelation matrix is ​​calculated using reconstructed values ​​from luma and chromaticity samples. These samples are full-range (e.g., between 0 and 1023 for 10-bit content), resulting in relatively large values ​​in the autocorrelation matrix. This requires high-bit-depth operations during model parameter calculation. A proposed solution is to remove a fixed offset from the luma and chromaticity samples in each PU for each model. This reduces the magnitude of the values ​​used in model creation and allows for a reduction in the precision required for fixed-point arithmetic. Therefore, a 16-bit decimal precision is proposed instead of the 22-bit precision of the original CCCM implementation.

[0134] For simplicity, the reference sample values ​​immediately outside the top-left corner of the PU are used as offsets (offsetLuma, offsetCb, and offsetCr). The sample values ​​used in both model creation and final prediction (i.e., luminance and chrominance in the reference region, and luminance in the current PU) are reduced by these fixed values, as follows: C' = C – offsetLuma N' = N – offsetLuma S' = S – offsetLuma E' = E – offsetLuma W' = W – offsetLuma P' = nonLinear(C') B = midValue = 1<<(bitDepth - 1), Furthermore, the chromaticity values ​​are predicted using the following equation, where offsetChroma is equal to offsetCr and offsetCb for the Cr and Cb components, respectively: predChromaVal = c0C' + c1N' + c2S' + c3E' + c4W' + c5P' + c6B + offsetChroma.

[0135] To avoid any additional sample-level operations, the luminance offset is removed during luminance reference sample interpolation. This can be done, for example, by replacing the rounding term used in luminance reference sample interpolation with an updated offset that includes both the rounding term and offsetLuma. The chrominance offset can be removed by subtracting it directly from the reference chrominance sample. Alternatively, the effect of the chrominance offset can be removed from the cross-component vector, yielding the same result. To add the chrominance offset back to the output of the convolution prediction operation, it is added to the bias term of the convolution model.

[0136] The calculation of CCCM model parameters requires division operations. Division operations are not always considered implementation-friendly. Division operations are replaced by multiplication (using scaling factors) and shift operations, where the scaling factor and the number of shifts are calculated based on the denominator, similar to the method used in the calculation of CCLM parameters.

[0137] 2.1.13.3 Gradient Linear Model For the YUV 4:2:0 color format, the Gradient Linear Model (GLM) method can be used to predict chromaticity samples from the luminance sample gradient. Two modes are supported: two-parameter GLM mode and three-parameter GLM mode.

[0138] Compared to CCLM, two-parameter GLM uses the gradient of luminance samples to derive a linear model, rather than downsampled luminance values. Specifically, when two-parameter GLM is applied, the input to the CCLM process (i.e., downsampled luminance samples) is... Gradient of brightness sample points Replacement. Other parts of CCLM (e.g., parameter derivation, linear transformation of prediction samples) remain unchanged.

[0139]

[0140] In a three-parameter GLM, chromaticity samples can be predicted based on both the gradient of luminance samples with different parameters and the downsampled luminance values. The model parameters of the three-parameter GLM are derived from 6 rows and 6 columns of adjacent samples using an MSE minimization method based on LDL decomposition, as used in CCCM.

[0141]

[0142] For signaling, when CCLM mode is enabled for the current CU, a flag is signaled to indicate whether GLM is enabled for both Cb and Cr components; if GLM is enabled, another flag is signaled to indicate which of the two GLM modes is selected, and a syntax element is also signaled to select one of the four gradient filters for gradient calculation.

[0143] • Enable four gradient filters for GLM, such as Figure 14 As shown.

[0144] 2.1.13.4 CCCM Signaling The use of this mode is signaled via the PU-level flag of the CABAC codec. A new CABAC context is included to support this. When signaling is involved, CCCM is considered a sub-mode of CCLM. That is, the CCCM flag is signaled only when the intra-frame prediction mode is LM_CHROMA.

[0145] 2.1.13.5 CCCM using non-downsampled luminance samples The CCCM mode uses a 3x2 filter that employs unsampled luminance samples, consisting of a 6-tap spatial term, four nonlinear terms, and a bias term. The 6-tap spatial term corresponds to the 6 neighboring luminance samples (i.e., , C) around the chrominance sample to be predicted (i.e., C). L 0 、L 1 、…、L 5) Four nonlinear terms from the sample points L 0 、L 1 、L 2 and L 3 is derived, such as Figure 15 As shown.

[0146] , in It is a coefficient. This is an offset. Similar to existing CCCM designs, up to 6 rows / columns of chroma samples above and to the left of the current CU are applied to derive the filter coefficients. The filter coefficients are derived based on the same LDL decomposition method used in CCCM. In addition to the existing model, the proposed method is transmitted as an supplementary CCCM model via signaling. When CCCM is selected, a single flag is transmitted via signaling and applied to both chroma components to indicate whether the default CCCM model or the proposed CCCM model is applied. Furthermore, SPS signaling is introduced to indicate whether CCCM using unsampled luminance samples is enabled.

[0147] 2.1.13.6 Block Vector Guided CCCM (BVG-CCCM) When in-segment luminance prediction is encoded and decoded using IBC or IntraTMP (Intra-Template Matching Prediction), the BVG-CCCM mode can be used. In this mode, the block vectors of the in-segment luminance blocks encoded and decoded in IBC or IntraTMP mode are used to determine the reference region for calculating CCCM parameters. Prediction is performed using the calculated model parameters and in-segment luminance samples. Figure 16 The reference region in the BVG-CCCM method is shown.

[0148] The BVG-CCCM mode uses an 11-tap filter for cross-component prediction, as shown below: predChromaVal = c0C + c1N + c2S + c3E + c4W + c5P(C) + c6P(N) + c7P(S) +c8P(W) + c9P(E)+ c 10 B.

[0149] The input of the spatial domain 5-tap component of the filter consists of the center (C) luminance sample that is co-located with the chrominance sample to be predicted, and its upper / north (N), lower / south (S), left / west (W), and right / east (E) neighbors, as shown below. Figure 1 As shown.

[0150] The nonlinear term P is represented as the square of the corresponding brightness sample, and B is the bias term.

[0151] Similar to the direct block vector (DBV), five locations in the same luminance block region are scanned, and the associated block vector is then used to determine the reference region for parameter calculation in the BVG-CCCM method.

[0152] 2.1.13.7 Gradient- and Position-Based Convolutional Cross-Component Model (GL-CCCM) This method uses a filter to map luminance values ​​to chrominance values. The filter input consists of a spatial luminance sample, two gradient values, two location information terms, a nonlinear term, and a bias term. The GL-CCCM method uses the gradient and location information instead of the four spatial neighbor samples used in the CCCM filter. The GL-CCCM filter used for prediction is:

[0153] Among them G y and G x These are the vertical gradient and the horizontal gradient, respectively, and are calculated as follows:

[0154]

[0155] In addition, Y and X are the spatial coordinates of the center brightness sample point.

[0156] The remaining parameters are the same as those in the CCCM tool. The reference area for parameter calculation is the same as that in the CCCM method.

[0157] The use of this mode is achieved through signal transmission of the PU-level flag encoded and decoded in CABAC. When signaling is involved, GL-CCCM is considered a sub-mode of CCCM. That is, the GL-CCCM flag is only transmitted through signal transmission if the original CCCM flag is true.

[0158] Similar to CCCM, the GL-CCCM tool has six modes for calculating parameters: • Single-model GL-CCCM from the templates above and to the left; • Single-model GL-CCCM from the template above; • Single-model GL-CCCM from the template on the left; • Multi-model GL-CCCM from the top and left templates; • Multi-model GL-CCCM from the template above; • Multi-model GL-CCCM from the left template.

[0159] The encoder performs a SATD search on the six GL-CCCM patterns and existing CCCM patterns to find the best candidate for the full RD test.

[0160] 2.1.13.8 CCCM with Multiple Downsampling Filters Multiple downsampling filters are applied to a set of reconstructed luminance samples in the CCCM. A linear combination of these downsampled reconstructed samples is multiplied by derived filter coefficients to form the final chromaticity prediction. The horizontal or vertical position of the center luminance sample is also considered in the tested model. The cross-component model shown below was tested as an additional CCCM mode, where the mode index is transmitted via the signal in the bitstream: (1) Model 1: predChroma = c0 H(C) + c1 G1(C) + c2 G2(C) + c3 G3(C) + c4 P(H(C)) + c5 P(G1(C)) + c6 P(G2(C)) + c7 X + c8 Y + c9 B; (2) Model 2: predChroma = c0 H(C) + c1 H(W) + c2 H(E) + c3 G1(C) + c4 G1(W) + c5 G1(E) + c6 P(H(C)) + c7 P(H(W)) + c8 P(H(E)) + c9 X + c10 B; (3) Model 3: predChroma = c0 H(C) + c1 H(NE) + c2 H(SW) + c3 G3(C) + c4 G3(NE) + c5 G3(SW) + c6 P(H(C)) + c7 P(H(NE)) + c8 P(H(SW)) + c9 Y + c10 B; Where H(·), G1(·), G2(·), and G3(·) are... Figure 18 The various downsampling filters shown are illustrated, where C represents the current chroma sample location, and N, S, W, E, NE, and SW are the locations surrounding C. i These are the filter coefficients, P and B are the nonlinear and bias terms, and X and Y are the horizontal and vertical positions of the center brightness sample relative to the top-left corner of the block.

[0161] 2.1.14 Local Enhancement Cross-Component Prediction (LB-CCP) Predicted samples in MM-CCLM / MM-CCCM can be filtered along with neighboring samples. For example... Figure 19 As shown, a 3×3 low-pass filter is applied to filter the predicted samples generated by MM-CCLM / MM-CCCM. For samples at the top / left boundary, the filtering window may include neighboring reconstructed samples. For inner samples, the filtering window only includes the predicted samples, which may be filled. A signal transmission flag indicates whether filtering is applied to blocks encoded / decoded using MM-CCLM / MM-CCCM.

[0162] 2.1.15 Cross-Component Prediction (CCP) Merge (also known as Non-local CCP) Mode For chroma encoding / decoding, a signal transmission flag indicates whether to use CCP mode (including CCLM, CCCM, GLM, and their variants) or a non-CCP mode (a fusion of regular chroma intra-prediction mode and chroma intra-prediction mode). If CCP mode is selected, another signal transmission flag indicates how the CCP type and parameters are derived, i.e., whether derived from the CCPmerge list or on-the-fly via signal transmission / derivation. The CCP Merge candidate list is constructed from spatially adjacent, temporal, spatially non-adjacent, and history-based m- or shifted temporal candidates. After including these candidates, a default model is further included to fill the remaining empty positions in the Merge list. To remove redundant CCP models from the list, a deduplication operation is applied. After the list is constructed, the CCP models in the list are reordered according to the SAD cost obtained using the neighbor templates of the current block. More details are described below.

[0163] Candidates for adjacent and non-adjacent airspace The positions and inclusion order of spatially adjacent and non-adjacent candidates are the same as those defined in ECM for regular inter-frame Merge prediction candidates.

[0164] Time-domain and shifted time-domain candidates Temporal candidates are selected from co-located images. The positions and inclusion order of the temporal candidates are the same as those defined in ECM for regular inter-frame merge prediction candidates. Shifted temporal candidates are also selected from co-located images. The positions of the temporal candidates are shifted by the selected motion vectors derived from the motion vectors of neighboring blocks.

[0165] Candidates based on history A history-based table is maintained to include the most recently used CCP models, and this table is reset at the beginning of each CTU row. If the current list is not full after including both spatially adjacent and non-adjacent candidates, a CCP model from the history-based table is added to the list.

[0166] Default Candidate CCLM candidates with default scaling parameters are considered only if the list is insufficient after considering candidates that are spatially adjacent, spatially non-adjacent, or based on history. If the current list does not have candidates using the single-model CCLM mode, the default scaling parameter is {0, 1 / 8, -1 / 8, 2 / 8, -2 / 8, 3 / 8, -3 / 8, 4 / 8, -4 / 8, 5 / 8, -5 / 8, 6 / 8}. Otherwise, the default scaling parameter is {0, the scaling parameter of the first CCLM candidate + {1 / 8, -1 / 8, 2 / 8, -2 / 8, 3 / 8, -3 / 8, 4 / 8, -4 / 8, 5 / 8, -5 / 8, 6 / 8}}.

[0167] A signal transmission flag indicates whether CCP Merge mode is applied. If CCP Merge mode is applied, a signal transmission index indicates which candidate mode is used by the current block. Additionally, CCP Merge mode is not allowed for the current chroma codec block when the current CU is encoded / decoded via intra-fractional segmentation (ISP) with a single tree, or when the current chroma codec block size is less than or equal to 16.

[0168] 2.1.16 Spatial Geometric Partitioning Model (SGPM) SGPM is an intra-frame mode of inter-frame coding / decoding tools similar to GPM, where two prediction parts are generated from the intra-frame prediction process. In this mode, a candidate list is constructed, where each entry contains a segmentation partition and two intra-frame prediction modes, such as... Figure 20 As shown, 26 segmentation modes and 3 intra-frame prediction modes were used to form a combination. The length of the candidate list was set to 16. The selected candidate indices were transmitted via signaling.

[0169] The list is reordered using a template. Figure 21 The SAD between the template's prediction and reconstruction was used for sorting. The template size was fixed at 1.

[0170] For each segmentation pattern, the same intra-to-inter-frame GPM list derivation (Section 2.2.21) is used to derive the IPM list for each segment. The IPM list size is set to 3. In the list, the TIMD-derived pattern is replaced by two derived patterns with horizontal and vertical directions.

[0171] The SGPM pattern is applied with limited block sizes: 4 <= width <= 64, 4 <= height <= 64, width < height. 8. Height < Width 8. Width Height >= 32.

[0172] The PPS flag is encoded and decoded to indicate whether mixing of two intra-frame predictions is allowed. When the PPS flag is set to false, the following adaptive mixing is also applied to the spatial GPM, where... Figure 22 The mixing depth τ shown is derived as follows: • If min(width, height) == 4, then choose 1 / 2 τ; • Otherwise, if min(width, height) == 8, then choose τ; • Otherwise, if min(width, height) == 16, then choose 2τ; • Otherwise, if min(width, height) == 32, then choose 4τ; • Otherwise, choose 8τ.

[0173] Otherwise (with the PPS flag set to true), 1 / 4 τ is always used for blocks encoded via spatial GPM to ensure that blending is not used when SGPM blocks have perfectly horizontal or vertical split angles, and a much narrower blending width is used when SGPM blocks have other split angles. It should be noted that this flag is set to true in the current Common Test Conditions (CTC) for screen content video.

[0174] 2.1.17 Plane Mode with Direction Two additional planar modes are used, where either horizontal interpolation only or vertical interpolation only is used to obtain the predicted samples.

[0175] For the planar horizontal mode, horizontal linear interpolation is performed based solely on the left reference sample and the upper right reference sample to predict the current sample as:

[0176] For the planar vertical mode, vertical linear interpolation is performed only based on the upper reference sample and the lower left reference sample to predict the current sample: . Transform kernel selection for horizontal and vertical planar modes, as follows: Figure 23 As shown. If the intra-prediction mode of the current block is planar vertical mode, then the horizontal intra-prediction mode is used to derive the transform kernels in the MTS set and LFNST set. Conversely, if the intra-prediction mode of the current block is planar horizontal mode, then the vertical intra-prediction mode is used to derive the transform kernels in the MTS set and LFNST set.

[0177] 2.1.18 Direct block vector of chroma blocks Direct block vectors are used for chroma blocks in a dual-tree stripe. When the chroma dual-tree is activated, a signal transmission flag indicates whether the chroma blocks are being encoded / decoded using IBC mode. Figure 24 If one of the luma blocks in the five locations shown is encoded or decoded in IBC or IntraTMP mode, its block vector is scaled and used as the block vector for the chroma block. Template matching is used to perform the block vector scaling.

[0178] 2.2 Inter-frame prediction 2.2.1 Local Illumination Compensation (LIC) LIC is an inter-frame prediction technique used to model the local illumination variation between the current block and its predicted block as a function of the local illumination variation between the current block template and the reference block template. The parameters of this function can be scaled... α and offset β This means that they form linear equations, i.e. α p[x]+ β To compensate for lighting changes, p[x] is a reference sample point at position x on a reference image pointed to by MV. When surround motion compensation is enabled, MV must be clipped to account for surround offset. Because α and β It can be derived based on the current block template and the reference block template, so there is no signaling overhead for them except for signaling the LIC flag to indicate the use of LIC for AMVP mode.

[0179] The local illumination compensation proposed in JVET-O0066 was used for inter-frame CUs and the following modifications were made.

[0180] • Intra-frame neighbor samples can be used for LIC parameter derivation; • For blocks with fewer than 32 luminance samples, LIC is disabled; • For both non-subblock mode and affine mode, the LIC parameter derivation is performed based on the template block sample corresponding to the current CU, rather than based on the partial template block sample corresponding to the first upper-left 16x16 cell; • The samples of the reference block template are generated by using a MC with block MV, without rounding them to integer pixel precision.

[0181] For bidirectional prediction inter-frame CUs, two sets of LIC parameters are derived separately for L0 and L1 prediction samples. An iterative method for deriving the L0 and L1 LIC parameters is applied. Specifically, the L0 LIC parameters are first derived by minimizing the difference between the L0 template prediction T0 and the template T, and the samples in T are updated by subtracting the corresponding samples in T0. Then, the L1 parameters are calculated to minimize the difference between the L1 template prediction T1 and the updated template. Finally, the L0 parameters are refined again in the same manner.

[0182] 2.2.2 Non-adjacent airspace candidates For example, in JVET-L0399, non-adjacent spatial merge candidates are inserted after the temporal motion vector prediction (TMVP) in the regular merge candidate list. The style of the spatial merge candidates is as follows: Figure 25 As shown. The distance between non-adjacent spatial domain candidates and the current codec block is based on the width and height of the current codec block. Line buffer limits are not applied.

[0183] 2.2.3 Derivation of Time-Domain Motion Information In VVC, temporal motion vector prediction (TMVP) for AMVP and Merge modes is derived by acquiring motion information from the center or lower right of the co-occurrence block in the co-occurrence image transmitted via signal transmission. Similarly, for sub-block-based temporal motion vector prediction (SbTMVP) mode, motion information from the left neighboring location is used as motion displacement, which is then used to obtain the TMVP at the sub-CU level.

[0184] In ECM, two aspects were modified to further improve the encoding and decoding efficiency of TMVP. First, two co-located images are used, which are two reference frames with the minimum POC distance relative to the frame to be encoded / decoded. Second, the motion displacement for locating the TMVP is adaptively determined from multiple positions based on template cost. More specifically, two motion displacement candidate lists are constructed for each of the two co-located frames. The motion displacement with the minimum template matching cost is used to derive either SbTMVP or TMVP candidates. The sub-block-based merge list includes a maximum of four SbTMVP candidates. The SbTMVP candidate with the minimum template matching cost derived from the first co-located frame is placed at the first entry without reordering, while other SbTMVP candidates are ordered together with affine candidates. Furthermore, the prediction direction of each sub-block template is determined based on the central sub-block. Figure 26As shown, if the central sub-block is unidirectionally predicted, then all sub-block templates are unidirectionally predicted, and vice versa. If the motion vector of the corresponding adjacent sub-block at a defined reference list is unavailable for a sub-block template, then zero MV is used for that sub-block template.

[0185] 2.2.4 Template Matching (TM) Template matching (TM) is a decoder-side MV derivation method used to refine the motion information of the current CU by finding the closest match between a template in the current image (i.e., the top and / or left neighboring blocks of the current CU) and a block in the reference image (i.e., of the same size as the template). For example... Figure 27 As shown, within the search range of [-8, +8] pixels, a better MV is searched around the initial motion of the current CU. The template matching method in JVET-J0021 is used, with the following modifications: the search step size is determined based on the AMVR mode, and in Merge mode, the TM can be cascaded with the bilateral matching process.

[0186] In AMVP mode, MVP candidates are determined based on template matching error to select the one that achieves the minimum difference between the current block template and the reference block template. Then, TM is performed only for that specific MVP candidate for MV refinement. TM refines the MVP candidate by using an iterative 16-point diamond search, starting with full-pixel MVD precision (or 4 pixels for 4-pixel AMVR mode) within a search range of [-8, +8] pixels. AMVP candidates can be further refined using a cross search with full-pixel MVD precision (or 4 pixels for 4-pixel AMVR mode), followed by half-pixels and quarter-pixels sequentially according to the AMVR mode specified in Table 2. This search process ensures that the MVP candidate maintains the same MV precision as indicated by the AMVR mode after the TM process. During the search process, if the difference between the previous minimum cost and the current minimum cost in the iteration is less than or equal to a threshold of the block area, the search process terminates.

[0187] Table 2. AMVR search styles and Merge mode with AMVR.

[0188]

[0189] In Merge mode, a similar search method is applied to the Merge candidates indicated by the Merge index. As shown in Table 2, TM can proceed up to 1 / 8 pixel MVD precision, or skip those precisions beyond half-pixel MVD precision, depending on whether an alternative interpolation filter is used based on the merged motion information (i.e., used when AMVR is in half-pixel mode). Furthermore, when TM mode is enabled, template matching can operate as a standalone process or as an additional MV refinement process between block-based and sub-block-based bilateral matching (BM) methods, depending on whether BM can be enabled according to its enable condition check.

[0190] When TM is applied to bidirectional prediction blocks, an iterative process is used. Specifically, the initial motion vectors of L0 and L1 are first refined, and the TM costs Cost0 and Cost1 are calculated for L0 and L1 respectively. When Cost0 is greater than Cost1, the refined motion vector of L1 ( ) was used to derive the further refined motion vector of L0 ( ).Then, use It is further refined. Similarly, when Cost0 is not greater than Cost1, the refined motion vector of L0 ( ) was used to derive the further refined motion vector of L1 ( ),and use It is further refined. In addition, TM for bidirectional prediction is enabled when the DMVR condition is met.

[0191] 2.2.5 Multi-pass decoder-side motion vector refinement Multi-pass decoder-side motion vector refinement is applied. In the first pass, bilateral matching (BM) is applied to the codec block. In the second pass, BM is applied to each 16x16 sub-block within the codec block. In the third pass, the motion vectors (MVs) in each 8x8 sub-block are refined by applying bidirectional optical flow (BDOF). The refined MVs are stored for both spatial and temporal motion vector prediction.

[0192] 2.2.5.1 First pass – Block-based bilateral matching MV refinement In the first pass, the refined MV is derived by applying the BM to the codec block. Similar to decoder-side motion vector refinement (DMVR), in the bidirectional prediction operation, the refined MV is searched around the two initial MVs (MV0 and MV1) in the reference picture lists L0 and L1. The refined MVs (MV0_pass1 and MV1_pass1) are derived around the initial MVs based on the minimum bilateral matching cost between the two reference blocks in L0 and the two reference blocks in L1.

[0193] BM performs a local search to derive the integer sample precision intDeltaMV. The local search applies a 3×3 square search pattern to iterate through the search range [-sHor, sHor] in the horizontal direction and [-sVer, sVer] in the vertical direction, where the values ​​of sHor and sVer are determined by the block dimension, and the maximum value of sHor and sVer is 8.

[0194] The cost of bilateral matching is calculated as: bilCost = mvDistanceCost + sadCost. When the block size is cbW... When cbH is greater than 64, the mean-removed SAD (MRSAD) cost function is applied to remove the DC effect of distortion between reference blocks. The intDeltaMV local search terminates when bilCost at the center point of the 3×3 search pattern has the minimum cost. Otherwise, the current minimum cost search point becomes the new center point of the 3×3 search pattern, and the search for the minimum cost continues until it reaches the end of the search range.

[0195] The existing fractional sample refinement is further applied to derive the final deltaMV. The refined MV after the first pass is then derived as: · MV0_pass1 = MV0 + deltaMV, · MV1_pass1 = MV1 – deltaMV.

[0196] 2.2.5.2 Second pass – Sub-block-based bilateral matching MV refinement In the second pass, the refined MV is derived by applying the BM to 16×16 grid sub-blocks. For each sub-block, a refined MV is searched around the two MVs (MV0_pass1 and MV1_pass1) obtained in the first pass in the reference image lists L0 and L1. The refined MVs (MV0_pass2(sbIdx2) and MV1_pass2(sbIdx2)) are derived based on the minimum bilateral matching cost between the two reference sub-blocks in L0 and L1.

[0197] For each subblock, BM performs a full search to derive the integer sample precision intDeltaMV. The full search has a search range of [-sHor, sHor] in the horizontal direction and [-sVer, sVer] in the vertical direction, where the values ​​of sHor and sVer are determined by the block dimension, and the maximum value of sHor and sVer is 8.

[0198] The bilateral matching cost is calculated by applying a cost factor to the SATD cost between the two reference subblocks, as follows: bilCost = satdCost costFactor. Search area (2) sHor + 1) (2) sVer + 1) is divided into Figure 28 A maximum of 5 diamond-shaped search regions are shown. Each search region is assigned a costFactor, which is determined by the distance (intDeltaMV) between each search point and the starting MV, and each diamond region is processed sequentially starting from the center of the search region. Within each region, search points are processed in raster scan order, starting from the top left corner and ending at the bottom right corner. The minimum bilCost within the current search region is less than or equal to sbW. When the threshold of sbH is reached, the integer pixel full search is terminated; otherwise, the integer pixel full search continues to the next search region until all search points have been checked. Additionally, the search process terminates if the difference between the previous minimum cost and the current minimum cost in the iteration is less than or equal to a threshold of the block area.

[0199] The existing VVC DMVR fractional sample refinement is further applied to derive the final deltaMV(sbIdx2). The MV of the second refinement is then derived as: · MV0_pass2(sbIdx2) = MV0_pass1 + deltaMV(sbIdx2), · MV1_pass2(sbIdx2) = MV1_pass1 – deltaMV(sbIdx2).

[0200] 2.2.5.3 Third pass – Sub-block based bidirectional optical flow MV refinement In the third pass, the refined MV is derived by applying BDOF to the 8×8 grid sub-blocks. For each 8×8 sub-block, BDOF refinement is applied to derive scaled Vx and Vy, starting from the refined MV of the parent-child blocks in the second pass and without clipping. The derived bioMv(Vx, Vy) is rounded to 1 / 16 sample precision and clipped between -32 and 32.

[0201] The third refined MV (MV0_pass3(sbIdx3) and MV1_pass3(sbIdx3)) is derived as follows: · MV0_pass3(sbIdx3) = MV0_pass2(sbIdx2) + bioMv, · MV1_pass3(sbIdx3) = MV0_pass2(sbIdx2) – bioMv.

[0202] 2.2.5.4 Fourth pass – Bidirectional optical flow MV refinement based on adaptive sub-blocks In the fourth pass, the refined MV is derived by applying BDOF to 4×4 or 8×8 grid sub-blocks. 4×4 grid sub-blocks are used when the block is less than 1024 pixels. Otherwise, 8×8 grid sub-blocks are used. The MV of each sub-block is refined in the same way as used in the third pass.

[0203] In all of the aforementioned sub-entries, when surround motion compensation is enabled, the motion vector must be limited in taking surround offset into account.

[0204] 2.2.6 Adaptive Decoder-Side Motion Vector Refinement The adaptive decoder-side motion vector refinement method is an extension of multi-pass DMVR, consisting of two new merge modes to refine the motion vector only in one direction (L0 or L1) of the bidirectional prediction of the merge candidates that satisfy the DMVR conditions. The multi-pass DMVR process is applied to the selected merge candidates to refine the motion vector; however, in the first pass (i.e., PU level) of DMVR, either MVD0 or MVD1 is set to zero.

[0205] The merge candidates in the new Merge mode are similar to those in the regular Merge mode, derived from spatially adjacent encoded / decoded blocks, TMVP, non-adjacent blocks, HMVP, and paired candidates. The difference is that only those satisfying the DMVR conditions are added to the candidate list. Both new Merge modes use the same merge candidate list. If the BM candidate list contains inherited BCW weights, the DMVR process remains unchanged, except for the use of MRSAD or MRSATD for distortion calculation when the weights are unequal and bidirectional prediction is weighted with BCW weights. The encoding and decoding of the merge index is the same as in the regular Merge mode.

[0206] 2.2.7 OBMC When OBMC is applied, the top and left boundary pixels of the CU are refined using motion information from neighboring blocks with weighted prediction, as described in JVET-L0101.

[0207] The following conditions should not be used for OBMC: • When OBMC is disabled at the SPS level; • When the current block has intra-frame mode or IBC mode; • When the current luminance block area is less than or equal to 32.

[0208] In addition, OBMC is adaptively controlled at the block level as follows: • For affine Merge mode, the OBMC flag is inherited from the neighboring affine block.

[0209] • If there are neighboring blocks encoded or decoded using IBC, Palette, or BDPCM modes, OBMC is not applied to the block.

[0210] • When OBMC is applied to a block, a block boundary check is also performed based on the reference samples of the current block to determine whether OBMC is applied to the boundary. If any absolute difference between the predicted sample and the non-interpolated (integer pixel) reference sample is greater than a threshold, OBMC is not applied to that boundary.

[0211] Sub-block boundary OBMC performs this by applying the same blending to the top, left, bottom, and right sub-block boundary pixels using motion information from neighboring sub-blocks. Its sub-block-based encoding / decoding tools are enabled. • Affine AMVP mode; • Affine Merge pattern and sub-block-based temporal motion vector prediction (SbTMVP). • Bilateral matching based on sub-blocks.

[0212] When OBMC mode is used together with LMCS in CIIP mode, inter-frame blending is performed before the LMCS mapping of the inter-frame samples. LMCS is applied to the blended inter-frame samples, which are combined with the intra-frame samples to which LMCS is applied in CIIP mode.

[0213]

[0214] in This represents the sample points predicted by the motion of the current block in the original domain. This represents the sample points predicted in the mapping domain. This represents the sample points predicted by the motion of neighboring blocks in the original domain, and and It's the weight.

[0215] When the OBMC mode is used for LIC-encoded blocks, the LIC parameters are applied to generate corresponding prediction samples for the OBMC of the LIC-encoded block. Furthermore, to reduce complexity, OBMC is only applied to the top CU boundary and the left CU boundary, while the boundaries of internal sub-blocks of the LIC-encoded block are always disabled.

[0216] 2.2.8 Template Matching-Based OBMC In the template matching-based OBMC scheme, the method for deriving the predicted values ​​of CU boundary samples is determined based on the template matching cost, rather than directly using weighted prediction. This includes using only the motion information of the current block, or using the motion information of neighboring blocks in a hybrid mode.

[0217] In this scheme, for each block with a size of 4×4 at the top CU boundary, the template size above it is equal to 4×1. If N If two adjacent blocks have the same motion information, the size of the template above is increased to 4. N ×1, because the MC operation can be processed in one go. For each left block with a size of 4×4 at the left CU boundary, the left stencil size is equal to 1×4 or 1×4. N ( Figure 29 ).

[0218] For each 4×4 top block (or N (For each 4×4 block group), the predicted values ​​of the boundary samples are derived by following these steps.

[0219] With blocks A As the current block and its neighboring blocks AboveNeighbor_A For example, the operations on the left-hand block are performed in the same way.

[0220] First, the cost of three template matching methods ( , , The SAD between the reconstructed sample points of the template and their corresponding reference sample points derived through the MC process based on the following three types of motion information is measured: according to A The motion information is calculated.

[0221] according to AboveNeighbor_A The motion information is calculated.

[0222] according to A motion information and AboveNeighbor_A The weighted prediction of motion information is calculated, where the weighting factors are respectively and .

[0223] Second, through comparison Cost1 , Cost2 and Cost 3 Choose a method to calculate the final prediction results for the boundary samples.

[0224] The original MC result using the motion information of the current block is represented as: And the MC results using the motion information of neighboring blocks are represented as The final prediction result is expressed as .

[0225] - if Cost1 Minimum, then .

[0226] - if( Cost2 + ( Cost2 >>2) + ( Cost2 >>3))<= Cost1 If so, then use mixed mode 1.

[0227] For a luminance block, the number of mixed pixel rows is 4.

[0228] ; ; ; .

[0229] For a chroma block, the number of mixed pixel rows is 1.

[0230] .

[0231] - if Cost1 <= Cost2 If so, then use hybrid mode 2.

[0232] For a luminance block, the number of mixed pixel rows is 2.

[0233] , .

[0234] For a chroma block, the number of mixed pixel rows / columns is 1.

[0235] . - Otherwise, use mixed mode 3.

[0236] For a luminance block, the number of mixed pixel rows is 4.

[0237] ; ; .

[0238] For a chroma block, the number of mixed pixel rows is 1.

[0239] .

[0240] 2.2.9 Affine Model Inheritance and Non-Adjacent Affine Modes Based on Historical Parameters History-based Affine Model Inheritance (HAMI) allows affine models to be inherited from previously affine-encoded blocks that may not be adjacent to the current block. Similar to the enhanced regular Merge mode, Non-Adjacent Affine Mode (NA-AFF) is introduced.

[0241] The first history parameter table (HPT) is established. Each entry in the first HPT stores a set of affine parameters: a , b , c and d Each affine parameter is represented by a 16-bit signed integer. Entries in the HPT are categorized by reference list and reference index. Each reference list in the HPT supports 5 reference indices. The HPT category (denoted as HPTCat) is calculated in a formulaic manner as follows: HPTCat (RefList, RefIdx) = 5×RefList + min (RefIdx, 4), RefList and RefIdx represent the list of reference images (0 or 1) and the reference index, respectively. A maximum of 7 entries can be stored for each category, resulting in a total of 70 entries in the HPT. At the beginning of each CTU line, the number of entries for each category is initialized to zero. Decoding involves using the reference list RefList... cur and RefIdx cur After the affine codec (CU), the affine parameters are used to update the category HPTCat(RefList) in a manner similar to HMVP table updates. cur RefIdx cur ) entries.

[0242] Candidates based on historical affine parameters (HAPC) from... Figure 29 The MV of one of the seven neighboring 4×4 blocks, represented as A0, A1, A2, B0, B1, B2, or B3, and a set of affine parameters stored in the corresponding entry in the first HPT are derived. The MV of the neighboring 4×4 blocks is used as the base MV. In a formulaic manner, the current block at position ( x , y The MV at point ) is calculated as: , in( mv h base , mv v base ) represents the MV of the nearest 4×4 block, ( xbase , y base () indicates the center position of the nearest 4x4 block. x , y The MV can be the top left, top right, or bottom left corner of the current block to obtain the corner position MV (CPMV) for the current block, or it can be the center of the current block to obtain the regular MV for the current block.

[0243] A second history parameter table (HPT) containing basic MV information is also appended. The second HPT contains nine entries, each including the basic MV, reference index, four affine parameters for each reference list, and the base position. An additional MergeHAPC can be generated from the second HPT containing the basic MV information and corresponding affine models stored in the entries. The differences between the first and second HPTs are as follows: Figure 30 As shown.

[0244] Furthermore, paired affine merge candidates are generated from two historically derived or non-historically derived affine merge candidates. Paired affine merge candidates are generated by averaging the CPMV of existing affine merge candidates in the list.

[0245] In response to the introduction of the new HAPC, the size of the sub-block-based Merge candidate list was increased from 5 to 15, all of which are involved in the ARMC process.

[0246] In NA-AFF, the style for obtaining the nearest neighbors of non-adjacent spatial domains is as follows: Figure 6 As shown. Similar to existing non-adjacent regular Merge candidates, the distance between non-adjacent spatial neighbors in NA-AFF and the current codec block is also defined based on the width and height of the current CU.

[0247] Figure 6 Motion information of non-adjacent spatial neighbors in the VVC is used to generate additional inherited and constructed affine Merge / AMVP candidates. Specifically, for inherited candidates, the same derivation process for inherited affine Merge / AMVP candidates in the VVC remains unchanged, except that CPMV inherits from non-adjacent spatial neighbors. Non-adjacent spatial neighbors are checked based on their distance from the current block (i.e., from nearest to farthest). At a specific distance, for the candidate derivation of inheritance, only the first available neighbor (encoded in affine mode) from each side (e.g., left and top) of the current block is included. Figure 31A As shown by the red dashed arrows in the image, the order of checking the nearest neighbors on the left and top is from bottom to top and from right to left, respectively.

[0248] For candidates of the first type of construction, such as Figure 31BAs shown, the positions of non-adjacent spatial neighbors on the left and top are first determined independently; then, the positions of the upper left neighbors can be determined accordingly, which can form a rectangular virtual block together with the non-adjacent neighbors on the left and top. Then, as... Figure 32 As shown, motion information from three non-adjacent nodes is used to form a CPMV at the top left (A), top right (B), and bottom left (C) of the virtual block, which is then projected onto the current CU to generate corresponding candidate builds.

[0249] NA-AFF candidates are inserted into the existing affine Merge candidate list and affine AMVP candidate list in the following order: Affine Merge Mode: 1. SbTMVP candidate, if available; 2. Inherited from the nearest neighbor; 3. Inherited from non-adjacent neighbors; 4. Build from neighboring neighbors; 5. Affine candidates constructed from the first type of non-adjacent elements; 6. Zero MV.

[0250] Affine AMVP mode: 1. Inherited from the nearest neighbor; 2. Build from neighboring neighbors; 3. Translation MV from neighboring units; 4. Translation MV from temporal nearest neighbors; 5. Inherited from non-adjacent neighbors; 6. Affine candidates from the first type of construction that are not adjacent to each other; 7. Zero MV.

[0251] The size of the affine Merge candidate list increases from 5 to 15 due to the inclusion of additional candidates generated by NA-AFF. The size of the subgroups of the ARMC in the affine Merge pattern increases from 3 to 15.

[0252] In NA-AFF: 1. Regions from which non-adjacent neighbors originate are restricted to the current CTU (i.e., there are no additional storage requirements for the row cache).

[0253] 2. The storage granularity for affine motion information (including CPMV and reference index) is reduced from 8x8 to 16x16 (i.e., only affine motion from the top-left 8x8 block is saved). Additionally, the saved CPMV is projected onto each 16x16 block before storage, making position and size information unnecessary.

[0254] 3. Only the top left and top right CPMVs are stored (i.e., always use a 4-parameter affine model for NA-AFF).

[0255] 2.2.10 Sample-based BDOF In sample-based BDOF, motion refinement (Vx, Vy) is not derived on a block-by-block basis, but is performed on a per-sample basis.

[0256] The encoding / decoding block is divided into 8×8 sub-blocks. For each sub-block, whether to apply BDOF is determined by checking the SAD between two reference sub-blocks against a threshold. If it is decided to apply BDOF to the sub-block, a sliding 5×5 window is used for each sample in the sub-block, and the existing BDOF procedure is applied to derive Vx and Vy for each sliding window. The derived motion refinement (Vx, Vy) is applied to adjust the bidirectional predicted sample values ​​for the center sample of the window.

[0257] 2.2.11 Interpolation The 8-tap interpolation filter used in VVC has been replaced by a 12-tap filter. The interpolation filter is derived from the sinc function, and its frequency response is cut off at the Nyquist frequency and limited by a cosine window function. Table 3 shows the filter coefficients for all 16 phases. Figure 33 The frequency responses of the interpolation filter and the VVC interpolation filter were compared, and all frequency responses were at half-pixel phase.

[0258] Table 3. Filter coefficients of the 12-tap interpolation filter

[0259] For chroma interpolation, an additional, longer 6-tap filter is used. The filter coefficients are listed in Table 4.

[0260] Table 4. Coefficients of the 6-tap interpolation filter for the chromaticity component.

[0261]

[0262] 2.2.12 Multiple Hypothesis Prediction (MHP) In the multi-hypothesis inter-frame prediction mode (JVET-M0425), in addition to the regular bidirectional prediction signal, one or more additional motion-compensated prediction signals are transmitted via signal transmission. The resulting overall prediction signal is obtained by weighted superposition of samples. Utilizing the bidirectional prediction signal... and the first additional inter-frame prediction signal / hypothesis The generated prediction signal The following was obtained: .

[0263] Weighting factors The new syntax element add_hyp_weight_idx is specified according to the following mapping:

[0264] Similar to the above, more than one additional prediction signal can be used. The resulting overall prediction signal is iteratively accumulated using each additional prediction signal.

[0265] .

[0266] The resulting overall prediction signal is obtained as the last one. (i.e., has the largest index) of Within this EE, a maximum of two additional prediction signals can be used (i.e., Limited to 2).

[0267] The motion parameters for each additional prediction hypothesis can be transmitted via signaling by explicitly specifying the reference index, motion vector prediction index, and motion vector difference, or by implicitly specifying the merge index. A separate multi-hypothesis merge flag distinguishes between these two signaling modes.

[0268] For inter-frame AMVP mode, MHP is applied only when unequal weights are selected in BCW in bidirectional prediction mode.

[0269] Combining MHP and BDOF is possible; however, BDOF is only applied to the bidirectional prediction signal portion of the predicted signal (i.e., the ordinary first two assumptions).

[0270] 2.2.13 Pixel-based Affine Motion Compensation For the luminance and chrominance components, the minimum affine sub-block size changes from 4x4 to 1x1, and the 1x1 sub-block size allows for pixel-based affine MC. PROF is disabled when the affine sub-block width or height is less than 4.

[0271] 2.2.14 Adaptive Reordering of Merge Candidates Using Template Matching (ARMC-TM) Merge candidates are adaptively reordered using template matching (TM). The reordering method is applied to the regular Merge pattern, TM Merge pattern, and affine Merge pattern (excluding SbTMVP candidates). For the TM Merge pattern, Merge candidates are reordered before the refinement process.

[0272] The initial Merge candidate list is first constructed according to a given inspection order, such as spatial, TMVP, non-adjacent, HMVP, paired, and dummy Merge candidates. Then, the candidates in the initial list are divided into several subgroups. For Template Matching (TM) Merge mode and Adaptive DMVR mode, each Merge candidate in the initial list is first refined using TM / multi-pass DMVR. The Merge candidates in each subgroup are reordered to generate a reordered Merge candidate list, and the reordering is based on the cost of template matching. The indices of the selected Merge candidates in the reordered Merge candidate list are signaled to the decoder. For simplicity, the Merge candidates in the last subgroup (not the first subgroup) are not reordered. During the construction of the Merge motion vector candidate list, all zero candidates from the ARMC reordering process are excluded. For the regular Merge mode and TM Merge mode, the subgroup size is set to 5. For the affine Merge mode, the subgroup size is set to 3.

[0273] • Cost Calculation During the reordering process, the template matching cost of the merge candidate is measured by the SAD between the samples of the current block's template and their corresponding reference samples. The template includes a set of reconstructed samples adjacent to the current block. The reference samples of the template are located using the motion information of the merge candidate. When the merge candidate utilizes bidirectional prediction, the reference samples of the merge candidate's template are also generated through bidirectional prediction, such as... Figure 34 As shown.

[0274] • Refinement of the initial Merge candidate list When multi-pass DMVR is used to derive refined motion for the initial Merge candidate list, only the first pass of the multi-pass DMVR (i.e., PU level) is applied in the reordering. When template matching is used to derive refined motion, the template size is set to 1. When the block is a flat block with a width greater than twice its height or a narrow block with a height greater than twice its width, only the top or left template is used during TM motion refinement. TM is expanded to perform 1 / 16 pixel MVD precision. The first four Merge candidates are reordered in TM Merge mode using refined motion.

[0275] For a sub-block-based merge candidate with a sub-block size equal to Wsub × Hsub, the upper template includes several sub-templates of size Wsub × 1, and the left template includes several sub-templates of size 1 × Hsub. For example... Figure 35 As shown, the motion information of the sub-blocks in the first row and first column of the current block is used to derive the reference sample points of each sub-template.

[0276] • Reordering criteria During the reordering process, if the cost difference between a candidate and its predecessor is less than λ, for example, |D1-D2|<λ (where D1 and D2 are the costs obtained during the first ARMC sorting, and λ is the Lagrange parameter used in the RD criterion on the encoder side), the candidate is considered redundant.

[0277] The proposed algorithm is defined as follows: - Determine the minimum cost difference between a candidate and its predecessor among all candidates in the list. • If the minimum cost difference is greater than or equal to λ, the list is considered sufficiently diverse, and reordering stops.

[0278] • If the minimum cost difference is less than λ, the candidate is considered redundant and is moved to another position in the list. This other position is the first position where the candidate is sufficiently diverse compared to the previous candidate.

[0279] - The algorithm stops after a finite number of iterations (if the minimum cost difference is not less than λ).

[0280] This algorithm has been applied to regular, TM, BM, and affine Merge modes. Similar algorithms have been applied to Merge MMVD and Symbolic MVD prediction methods, which also use ARMC for reordering.

[0281] The value of λ is set to be equal to the rate-distortion criterion used to select the best Merge candidate on the encoder side in the low-latency configuration, and is also set to be equal to the value λ corresponding to another QP in the random access configuration. For QP offsets that are not present in the SPS, a set of λ values ​​corresponding to each QP offset transmitted through the signal is provided in the SPS or strip header.

[0282] • Extensions to the AMVP pattern The ARMC design also applies to the AMVP mode, where AMVP candidates are reordered based on TM costs. For the Template Matching (TM-AMVP) mode for advanced motion vector prediction, an initial AMVP candidate list is constructed, followed by refinement from the TM to construct a refined AMVP candidate list. Additionally, MVP candidates with TM costs greater than a threshold (which is equal to five times the cost of the first MVP candidate) are skipped.

[0283] Note that when surround motion compensation is enabled, MV candidates must be clipped while taking surround offset into account.

[0284] 2.2.15 ARMC based on MV candidate types Merge candidates for a single candidate type (e.g., TMVP or Non-Adjacent MVP (NA-MVP)) are reordered based on ARMC™ cost values. The reordered candidates are then added to the Merge candidate list. For the TMVP candidate type, more TMVP candidates with more temporal locations and different inter-frame prediction directions are added to perform reordering and selection. Furthermore, the NA-MVP candidate type is expanded using more spatially non-adjacent locations. The target reference image for a TMVP candidate can be selected from any reference image in the list based on a scaling factor. The selected reference image is the one with a scaling factor closest to 1.

[0285] 2.2.16 TM-based reordering for MMVD and affine MMVD The MMVD offset is extended for both MMVD and affine MMVD modes. For example... Figure 36 As shown, additional refinement positions along a diagonal angle of k×π / 8 are added, increasing the number of directions from 4 to 16. Second, all possible MMVD refinement positions (16×6) for each basic candidate are reordered based on the SAD cost between the template (the row above and column to the left of the current block) and its reference for each refinement position. Finally, the top 1 / 8 refinement positions with the minimum template SAD cost are retained as available positions for MMVD index encoding and decoding. The MMVD index is binary-coded using rice code with a parameter equal to 2. The affine MMVD reordering is extended, with additional refinement positions along a diagonal angle of k×π / 4 being added. After reordering, the top 1 / 2 refinement positions with the minimum template SAD cost are retained.

[0286] The top N motion candidates in the candidate list before reordering are used as the base candidates for both MMVD and affine MMVD. For MMVD, N equals 3, and for affine MMVD, N equals [1, 3] (depending on the affine flag of the neighboring blocks). Two ways are allowed to add MMVD offsets, including "both sides" and "one side," depending on whether the offsets of the other reference picture list are mirrored or directly set to zero. Which method is applied to a block depends on the TM cost.

[0287] 2.2.17 Derivation of Affine Candidates Based on Regression The regression-based motion vector field (RMVF) derivation method provides a novel sub-block-based merge candidate. For example... Figure 37 As shown, the motion vectors and center positions of the neighboring sub-blocks from the current CU are used as inputs to the linear regression process to derive a set of linear model parameters.

[0288] The sub-block motion field from the previously encoded and decoded affine CU and the motion vectors of neighboring sub-blocks from the current CU are used as inputs to the regression process. The predicted CPMV for the current block is derived as the output.

[0289] Regression-based affine merge candidates are derived and added to the affine merge list. The sub-block motion fields from previously encoded and decoded affine CUs and the motion information of neighboring sub-blocks from the current CU are used as inputs to the regression process to derive the proposed affine candidates.

[0290] Previously encoded and decoded affine CUs can be identified by scanning non-adjacent positions and the affine HMVP table.

[0291] The information of adjacent sub-blocks of the current CU is as follows: Figure 37 The 4x4 sub-blocks, represented by the gray area, are captured. For each sub-block, given a reference list, the corresponding motion vector and center coordinates of the sub-block can be used.

[0292] For each affine CU, at most two affine candidates can be derived. One has neighboring subblock information, and the other does not. All candidates generated by linear regression are deduplicated and collected into a candidate subgroup. When ARMC is enabled, the ARMC process based on TM cost is applied. Afterward, when N affine CUs are found, at most N linear regression-generated candidates are added to the affine merge list. The number of affine candidates for ARMC is 30, and the output list size is 15.

[0293] 2.2.18 Geometric Partitioning Mode (GPM) with Merge Motion Vector Difference (MMVD) GPM in VVC is extended by applying motion vector refinement on top of the existing unidirectional MV of GPM. First, a flag is transmitted to the GPMCU to specify whether to use this mode. If this mode is used, each geometric segment of the GPM CU can further determine whether to transmit MVD via signal transmission. If MVD is transmitted via signal transmission for a geometric segment, the segment's motion is further refined using the transmitted MVD information after selecting a GPM Merge candidate. All other processes remain the same as in GPM.

[0294] Similar to MMVD, MVD is transmitted as a pair of distance and direction via signals. GPM with MMVD (GPM-MMVD) involves nine candidate distances (¼ pixel, ½ pixel, 1 pixel, 2 pixels, 3 pixels, 4 pixels, 6 pixels, 8 pixels, 16 pixels) and eight candidate directions (four horizontal / vertical directions and four diagonal directions). Additionally, when pic_fpel_mmvd_enabled_flag equals 1, MVD is shifted left by 2, as in MMVD.

[0295] 2.2.19 Features an adaptive blending geometric segmentation mode (GPM) In VVC, the final predicted samples are generated by weighted averaging of the predictions from the two predicted signals. Two integer mixing matrices ( W 0 and W 1) Used. The weights in the GPM blending matrix are derived from the ramp function based on the displacement from the predicted sample location to the GPM segmentation boundary. The blending region size is fixed at 2 (2 samples on each side of the GPM segmentation boundary).

[0296] The blending process in ECM is improved by adding four additional blending region sizes (one-quarter, half, twice, and four times the existing region size), such as Figure 38 As shown. The CU level flags are encoded and decoded to accommodate the selected mixing region size via signal transmission. Furthermore, extended weighted precision is utilized, where the maximum weight value changes from 8 (in VVC) to 32 to accommodate the extended mixing region size.

[0297] 2.2.20 Geometric Segmentation Pattern (GPM) with Template Matching (TM) Template matching is applied to GPM. When GPM mode is enabled for CU, the CU level flag is signaled to indicate whether TM is applied to the two geometric segments. Motion information for each geometric segment is refined using TM. When TM is selected, a template is constructed using left, top, or left and top neighboring samples based on the segmentation angle, as shown in Table 5. Motion is then refined by minimizing the difference between the current template and the template in the reference image using the same search style in Merge mode with half-pixel interpolation filters disabled.

[0298] Table 5. Templates for the first and second geometric segments, where A indicates the use of the upper sample points, and L indicates the use of the upper sample points. Use the left sample point, and L+A indicates that both the left and top samples are used.

[0299]

[0300] The GPM candidate list is constructed as follows: 1. The interleaved list 0 MV candidates and list 1 MV candidates are derived directly from the regular merge candidate list, where list 0 MV candidates have a higher priority than list 1 MV candidates. A deduplication method with an adaptive threshold based on the current CU size is applied to remove redundant MV candidates.

[0301] 2. The interleaved list 1 MV candidates and list 0 MV candidates are also derived directly from the regular merge candidate list, where list 1 MV candidates have a higher priority than list 0 MV candidates. The same deduplication method with an adaptive threshold is also applied to remove redundant MV candidates.

[0302] 3. Zero MV candidates are filled until the GPM candidate list is full.

[0303] GPM-MMVD and GPM-TM are exclusively enabled to a single GPM CU. This is accomplished first by signaling the GPM-MMVD syntax. When both GPM-MMVD control flags are false (i.e., GPM-MMVD is disabled for both GPM segments), the GPM-TM flag is signaled to indicate whether template matching is applied to both GPM segments. Otherwise (if at least one GPM-MMVD flag is true), the value of the GPM-TM flag is presumed to be false.

[0304] 2.2.21 GPM with inter-frame and intra-frame prediction In GPM with inter-frame and intra-frame prediction, the final prediction samples are generated by weighting the inter-frame and intra-frame prediction samples for the regions separated for each GPM. Inter-frame prediction samples are derived from the inter-frame GPM, while intra-frame prediction samples are derived from the intra-frame prediction mode (IPM) candidate list and the index from the encoder transmitted through the signal. The IPM candidate list size is predefined as 3. Available IPM candidates are parallel angle modes (parallel mode) relative to the GPM block boundary, vertical angle modes (vertical mode) relative to the GPM block boundary, and planar modes, such as... Figures 39A to 39C As shown. Furthermore, as... Figure 39D The GPM with intra-frame and intra-frame prediction shown is limited to reduce signaling overhead for IPM and avoid increasing the size of intra-frame prediction circuitry on the hardware decoder. Additionally, direct motion vectors and IPM storage on the GPM mixing region are introduced to further improve encoding / decoding performance.

[0305] In IPM derivation based on DIMD and neighboring modes, parallel modes are registered first. Therefore, if no identical IPM candidates exist in the list, up to two IPM candidates derived from the decoder-side intra-frame pattern derivation (DIMD) method and / or neighboring block derivation can be registered. As for neighboring mode derivation, up to five locations are available for neighboring blocks, but they are limited by the angle of the GPM block boundary, as shown in Table 6, and they have been used for GPM with template matching (GPM-TM).

[0306] Table 6. Locations of available neighboring blocks for IPM candidate derivation based on the angles of GPM block boundaries. A and L represent... The top and left sides of the prediction block.

[0307]

[0308] GPM-intraframe can be combined with GPM with Merge and Motion Vector Difference (GPM-MMVD). TIMD is used as an IPM candidate for GPM-intraframe to further improve encoding and decoding performance. Parallel modes can be registered first, followed by TIMD, DIMD, and IPM candidates from neighboring blocks.

[0309] 2.2.22 Template-matching-based reordering for GPM partitioning patterns In template-match-based reordering for GPM partitioning patterns, given the motion information of the current GPM block, the corresponding TM value of the GPM partitioning pattern is calculated. All GPM partitioning patterns are then reordered in ascending order based on their TM values. Instead of sending the GPM partitioning patterns, an index using Golomb-Rice codes is transmitted via signaling, indicating the exact location of the GPM partitioning pattern in the reordering list.

[0310] The GPM partitioning pattern reordering method is a two-step process performed after the corresponding reference templates for the two GPM partitions in the encoding / decoding unit are generated, as follows: • Extend the GPM segmentation edge to the reference templates of the two GPM segments to generate 64 reference templates, and compute the corresponding TM cost for each of the 64 reference templates. • The TM generation value based on the GPM partitioning pattern is reordered in ascending order for the GPM partitioning patterns, and the best 32 partitioning patterns are marked as available partitioning patterns.

[0311] like Figure 40 As shown, the edges on the template are extended from the edges of the current CU, but the GPM blending process is not applied to the template regions across the edges.

[0312] After ascending reordering using the TM cost, the index is transmitted via signaling.

[0313] 2.2.23 Two-way prediction of GPM The GPM design in VVC relies on unidirectional predicted motion vectors to generate motion-compensated predicted samples for each inter-frame GPM segment. In ECM, this design has been extended to allow the use of bidirectional predicted motion vectors.

[0314] When constructing the GPM candidate list, the extraction process for extracting unidirectional predicted motion vectors from the initial Merge list is invoked only for small 8x8, 16x8, and 8x16 blocks. For larger blocks, the extraction process is bypassed, allowing the initial Merge list (which can contain bidirectional MVs of the Merge) to be directly used as the final GPM Merge list. The generation of the initial Merge list is the same as before (i.e., normal Merge list generation without any candidate reordering), except that when generating the initial Merge list for larger blocks (i.e., blocks where the extraction process is bypassed), the motion vector difference threshold used to control whether a candidate can be added to the list is increased to a full sample distance.

[0315] When generating motion-compensated prediction samples, BDOF-based motion vector refinement, as in multi-pass DMVR, is used.

[0316] When GPM-MMVD is used for GPM segmentation and its underlying motion vectors are bidirectional predictions, for low-latency images, as in existing Merge MMVD designs, the MVD transmitted via signaling is applied over the L0 and L1 motion vectors. For non-low-latency images, the bidirectional prediction motion vectors are first converted to unidirectional prediction motion vectors, and then the MVD is applied on top.

[0317] 2.2.24 Two-sided matching AMVP-Merge mode The bidirectional prediction consists of an AMVP prediction in one direction and a Merge prediction in the other. This mode can be enabled to the codec block when the selected Merge and AMVP predictions satisfy the DMVR condition, where there is at least one reference image from the past and one reference image from the future relative to the current image, and both reference images are equidistant from the current image. Bidirectional matching MV refinement is applied to the Merge MV candidate and the AMVP MVP as starting points. Otherwise, if template matching is enabled, template matching MV refinement is applied to either the Merge or AMVP prediction, which has a higher template matching cost.

[0318] The AMVP portion of this pattern is transmitted via signaling as a regular one-way AMVP, i.e., the reference index and MVD are transmitted via signaling, and it has a derived MVP index if template matching is used, or the MVP index is transmitted via signaling when template matching is disabled.

[0319] For the AMVP direction LX, where X can be 0 or 1, the Merge portion in the other direction (1-LX) is implicitly derived by minimizing the bilateral matching cost between the AMVP prediction and the Merge prediction, i.e., for a pair of AMVP and Merge motion vectors. For each Merge candidate in the Merge candidate list with that other direction (1-LX) motion vector, the bilateral matching cost is computed using the Merge candidate MV and the AMVP MV. The Merge candidate with the minimum cost is selected. Bilateral matching refinement is applied to the codec block, with the selected Merge candidate MV and AMVP MV serving as the starting point.

[0320] The third pass of the multi-pass DMVR (i.e., sub-PU BDOF refinement of the multi-pass DMVR) is enabled for blocks encoded and decoded in AMVP-Merge mode. The sub-PU size of the BDOF is adaptively selected based on width × height. For blocks smaller than 256, a 4×4 sub-block size is used; otherwise, an 8×8 sub-block size is used. Additionally, the following high-precision equation for deriving the BDOF MV refinement parameters is utilized:

[0321] Where Gx / Gy is the sum of the two horizontal / vertical gradients derived for each reference block.

[0322] sum It is a weighted sum, where the weights depend on the location within the target region. In other cases, the weights can also be applied to derive vx / vy.

[0323] This mode is indicated by a flag, and if this mode is enabled, the AMVP direction LX is further indicated by a flag.

[0324] When the bilateral matching (BM) AMVP-Merge pattern is used for the current block and template matching is enabled, MVD is not signaled. An additional pair of AMVP-Merge MVPs is introduced. The Merge candidate list is sorted in ascending order based on the BM cost. An index (0 or 1) is signaled to indicate which Merge candidate from the sorted Merge candidate list is used. When there is only one candidate in the Merge candidate list, a pair of AMVP MVPs and MergeMVPs without bilateral matching MV refinement are populated.

[0325] 2.2.25 IBC Merge / AMVP List Construction Compared to VVC, the IBC Merge / AMVP list construction has been modified as follows: • It can be inserted into the IBC Merge / AMVP candidate list only if the IBC Merge / AMVP candidate is valid.

[0326] • Airspace candidates in the upper right, lower left, and upper left (belonging to the adjacent airspace candidate category) and a pairwise average candidate can be added to the IBC Merge / AMVP candidate list.

[0327] • Template-based adaptive reordering (ARMC-TM) is applied to the IBC Merge list.

[0328] • Candidates from non-adjacent spatial neighbor blocks (also known as non-adjacent candidates) can be added to the candidate lists for IBC Merge mode and IBC AMVP. These non-adjacent candidates are inserted between the adjacent spatial candidate and HBVP candidate lists for both IBC Merge and IBC AMVP. The same reference region for non-adjacent merges in regular inter-frame mode is reused for IBC.

[0329] • Removed the restriction that adjacent airspace candidates could not be used for IBC Merge in 4x4 CUs.

[0330] The HMVP table size for IBC is increased to 25. After deriving a maximum of 20 IBC Merge candidates using full deduplication, they are reordered together. After reordering, the top 6 candidates with the lowest template matching cost are selected as the final candidates in the IBC Merge list.

[0331] The zero vector candidate used to populate the IBC Merge / AMVP list is replaced with a set of BVP candidates located in the IBC reference region. The zero vector is invalid as a block vector in the IBC Merge mode, and therefore, it is discarded as a BVP in the IBC candidate list.

[0332] Three candidates are located at the nearest corner of the reference region, and three additional candidates are determined in the middle of the three sub-regions (A, B, and C), as follows. Figure 41 As shown, its coordinates are determined by the width and height of the current block, as well as the ΔX and ΔY parameters.

[0333] During IBC AMVP list construction, clustering of BVP candidates can be applied when both BV candidate components are not zero. If there are more than 2 valid BV candidates and a maximum of 6 candidates are clustered, then... Figure 42 The L2 distance clustering shown is applied, and the cluster radius is defined as follows:

[0334] Clustering methods are applied sequentially from the candidate list, and candidates assigned to groups are removed from the list for subsequent clustering. Within each group, the BVP with the lowest TM cost is selected as the representative candidate for that group. Finally, the representative candidates from the first two groups are selected as candidates for the IBC AMVP list.

[0335] Furthermore, if one of the BV candidate components is zero or the block is encoded / decoded in RIBC, a flag is transmitted via signaling to indicate this, with a directional flag indicating whether the horizontal or vertical component is non-zero. Instead of the usual IBCAMVP list, two new BVP candidates are derived, and the sign of the non-zero BV component is derived on the decoder side. AMVP BVP0 is set to the nearest valid position of the current block (-cbWidth or -cbHeight), so non-zero BVD is always negative, pointing to the left for a BV with zero vertical component, or pointing upwards for a BV with zero horizontal component. Similarly, AMVP BVP1 is set to the position furthest from the current block in the valid reference region, i.e., the left or top boundary of the IBC search region. Therefore, if BVP1 is selected, BVD is always positive, pointing to the right for a BV with zero vertical component, or pointing to the bottom for a BV with zero horizontal component.

[0336] The optimal IBC AMVP index is transmitted via signaling, which allows the sign of the non-zero BVD component to be derived on the decoder side. The absolute amplitude of the non-zero BVD component is also transmitted via signaling. In RRIBC, the direction of the flip mode is derived from the direction marker transmitted via signaling.

[0337] 2.2.26 IBC with Template Matching Template matching is used in both the IBC Merge mode and the IBC AMVP mode in IBC.

[0338] Compared to the IBC-TM Merge list used in the regular IBC Merge mode, the IBC-TM Merge list is modified so that candidates are selected based on a deduplication method, where the motion distance between candidates is the same as in the regular TM Merge mode. The zero-motion padding at the end is replaced by motion vectors pointing to the left (-W, 0), top (0, -H), and upper left (-W, -H), where W is the width of the current CU and H is the height of the current CU.

[0339] In IBC-TM Merge mode, the selected candidate is refined using a template matching method before the RDO or decoding process. IBC-TM Merge mode competes with the regular IBC Merge mode, and the TM-Merge flag is transmitted via signaling.

[0340] In the IBC-TM AMVP mode, up to three candidates are selected from the IBC-TM Merge list. Each of these three selected candidates is refined using a template matching method and ranked according to the template matching cost it produces. Then, as usual, only the top two are considered in the motion estimation process.

[0341] Template matching refinement for both IBC-TM Merge and AMVP modes is very simple because the IBC motion vector is constrained to (i) integers, and (ii) in the form of... Figure 43 Within the reference region shown. Therefore, in IBC-TM Merge mode, all thinning is performed with integer precision, and in IBC-TM AMVP mode, they are performed with either integer precision or 4-pixel precision, depending on the AMVR value. Such thinning only accesses samples that are not interpolated. In both cases, the motion vectors for thinning and the template used in each thinning step must respect the constraints of the reference region.

[0342] 2.2.27 IBC Reference Area The IBC reference area is extended to the two CTU rows above. Figure 44 The reference region used for encoding and decoding CTUs (m, n) is shown. Specifically, for a CTU (m, n) to be encoded and decoded, the reference region includes CTUs with indices (m-2, n-2)…(W, n-2), (0, n-1)…(W, n-1), (0, n)…(m, n), where W represents the maximum horizontal index within the current slice, strip, or picture. When the CTU size is 256, the reference region is limited to the uppermost CTU row. This setting ensures that for CTUs of size 128 or 256, the IBC does not require additional memory in the current ETM platform. The block vector search (or local search) range per sample is limited to horizontal [-(C<<1), C>>2] and vertical [-C, C>>2] to accommodate the reference region expansion, where C represents the CTU size.

[0343] 2.2.28 Fractional Pixel IBC The block vector resolution options have been expanded to include quarter-pixel resolution in addition to full-pixel and 4-pixel resolutions. Similar to the inter-frame AMVR syntax, the first bit is signaled to indicate whether the BV is at quarter-pixel resolution, and the second bit is signaled to switch between full-pixel and 4-pixel resolutions. Interpolation filters are applied to the luma (8-tap) and chroma (6-tap existing inter-frame interpolation) components of the IBC block. For template-based IBC tools, a 2-tap bilinear interpolation filter is applied to generate the template prediction block. Reference sample padding is performed when some of the reference samples are outside the IBC reference region. This is performed first in the horizontal direction and then in the vertical direction, if necessary.

[0344] 2.2.29 Filtered IBC Prediction An additional filtered IBC mode is introduced, in which a filter is applied to the IBC prediction, which is derived by minimizing the MSE between the current template and the reference template.

[0345] The filter output is calculated as follows: predLumaVal = c0C + c1N + c2S + c3E + c4W + c5P + c6B.

[0346] The nonlinear term P is represented as the square of the center sample C and scaled to the range of sample values ​​of the content: P = (C C + midVal )>>bitDepth.

[0347] The bias term B represents the scalar offset between the input and output and is set to the intermediate brightness value (512 for 10-bit content).

[0348] This filtered mode is used as an additional mode for non-Merge IBC blocks and is not used with IBC-LIC, IBC-CIIP, or RR-IBC. For IBC Merge modes, this filtered mode is inherited when the Merge mode list is constructed. The mode flag is transmitted via signaling before the IBC-LIC flag.

[0349] 2.2.30MVD Prediction In this method, possible combinations of MVD symbols and the first six most important suffix bits of the MVD magnitude are sorted according to the template matching cost, and the indices corresponding to the true MVD symbols and MVD magnitudes are derived and context-coded. On the decoder side, the MVD is derived as follows: 1. Analyze the amplitude of MVD components; 2. Parse the context-encoded MVD prediction index; 3. Construct MV candidates by creating combinations between possible symbols and possible MVD magnitudes, and add them to the MV predictions; 4. Based on template matching cost derivation, derive and sort the MVD prediction cost for each derived MV; 5. Use the index transmitted via signal to select the true MVD.

[0350] MVD prediction is applied to inter-frame AMVP, affine AMVP, MMVD, and affine MMVD modes. Note that when surround motion compensation is enabled, MV candidates must be clipped to account for surround offset.

[0351] 2.2.31 BVD Prediction Similar to MVD prediction, the possible BVD symbol combinations for the IBC mode are sorted according to the template matching cost. Furthermore, the first four most important suffix bits of the exponential Golomb code used to represent the BVD magnitude are also sorted according to the TM cost. An example is provided in... Figure 45 The template matching operation is shown in the figure. It is used to determine the BVD candidate with the best cost and to indicate in the bitstream whether the best candidate was correctly predicted.

[0352] 2.2.32 Enhanced bidirectional motion compensation In bidirectional motion compensation, out-of-bounds (OOB) predicted samples are discarded, and only non-OOB predicted values ​​(when available) are used to generate the final predicted value. Specifically, this makes... and This indicates the location of a predicted sample point in the current block. and (x = 0, 1) represents the MV of the current block; , , and These are the positions of the four boundaries of the image. A predicted sample point is considered out-of-bounds (OOB) when at least one of the following conditions is met: ( )>( + half_pixel ), ( )<( - half_pixel ), ( )>( + half_pixel ), ( )<( - half_pixel ), in half_pixel It equals 8, which represents the distance of half a pixel sample in 1 / 16 pixel sample precision.

[0353] After checking the OOB conditions for each sample point, the final predicted sample points for a bidirectional block are generated as follows: if It is OOB and Yes or No OOB ; else if Is it non-OOB and It's OOB ; else

[0354] When BCW is enabled, the OOB check process also applies.

[0355] Finally, note that this sample-point adaptive bidirectional prediction process only applies when at least one reference block is initially detected as a prediction cell that is partially or completely outside the boundary. Therefore, the block-level OOB criterion is checked first. If both prediction blocks are non-OOB, then the usual bidirectional prediction occurs.

[0356] 2.2.33 Image boundary filling with motion compensation Samples outside the image boundaries are derived through motion compensation, rather than simply using repeating fill. In the implementation, the total fill area size is increased by 16 compared to repeating fill. This is to maintain the MV limit when implementing repeating fill, such as... Figure 46 As shown.

[0357] For motion-compensated fill, the MV of the 4×4 boundary block is used to derive the M×4 or 4×M fill block. The value M is derived as the distance from the reference block to the image boundary, such as... Figure 47 As shown. Furthermore, once the motion vector points to a position inside the boundary of the reference image, M is set to at least 4. If the boundary block is intra-frame encoded, MV is unavailable, and M is set to 0. If M is less than 16, the remainder of the filled area is filled with repeating samples.

[0358] In the case of bidirectional inter-frame prediction, only one prediction direction is used in MC boundary padding, which has a motion vector pointing to a pixel position in the reference image that is farther from the image boundary in the padding direction.

[0359] Pixels in the MC fill block are corrected using an offset equal to the difference between the DC value of the reconstructed boundary block and its corresponding reference block.

[0360] 2.2.34 Reordering of the block-level reference image list A block-level reference picture reordering method based on template matching is used. For the one-way prediction AMVP mode, reference pictures in list 0 and list 1 are interleaved to generate a joint list. For each hypothesis of the reference pictures in the joint list, template matching is performed to compute the cost. The joint list is reordered in ascending order based on the template matching cost. The index of the selected reference picture in the reordered joint list is transmitted via signaling in the bitstream. For the two-way prediction AMVP mode, a list of reference picture pairs from list 0 and list 1 is generated and similarly reordered based on the template matching cost. The index of the selected pair is transmitted via signaling.

[0361] 2.2.35 Reference Image Resampling (RPR) Reference image resampling is inherited from VVC. Compared to the filter lengths in VVC (e.g., 8, 6, and 4 taps for luma affine, luma non-affine, and chroma blocks, respectively), the corresponding RPR filters in ECM increase to 12, 10, and 6 taps.

[0362] 2.2.36 Reconstructing the Reordered IBC (RR-IBC) The Reconstruction-Reordered IBC (RR-IBC) mode allows for the reconstruction of reordered IBC blocks after IBC encoding and decoding. When RR-IBC is applied, samples in the reconstructed block are flipped according to the flip type of the current block. On the encoder side, the original block is flipped before motion search and residual calculation, while the predicted block is derived without flipping. On the decoder side, the reconstructed block is flipped back to recover the original block.

[0363] Two flipping methods are supported for blocks encoded with RR-IBC: horizontal flipping and vertical flipping. First, for blocks encoded with IBC AMVP, a syntax flag is signaled indicating whether the reconstruction has been flipped. If flipped, another flag is further signaled specifying the flipping type. For IBC Merge, the flipping type is inherited from the neighboring block without syntax signaling. Considering horizontal or vertical symmetry, the current block and the reference block are typically aligned horizontally or vertically. Therefore, when a horizontal flip is applied, the vertical component of the BV is not signaled and is presumed to be 0. Similarly, when a vertical flip is applied, the horizontal component of the BV is not signaled and is presumed to be 0.

[0364] To better utilize symmetry, a flip-aware BV adjustment method is applied to refine block vector candidates. For example, as Figure 48A and Figure 48B As shown, ( x nbr ,y nbr )and( x cur , y cur () represents the coordinates of the center sample points of the neighboring blocks and the current block, respectively. BV nbr and BV cur These represent the BV of the neighboring block and the current block, respectively. When the neighboring block is horizontally flipped encoding / decoding, instead of directly inheriting the BV from the neighboring block, BV cur The horizontal component is transmitted through the direction of BV nbr The horizontal component (represented as) BV nbr h The added motion displacement is calculated, that is, BV cur h =2( x nbr - x cur ) + BV nbr h Similarly, in the case where adjacent blocks are encoded and decoded using vertical flipping, BV cur The vertical component is through the direction BV nbr The vertical component (represented as) BV nbr v The added motion displacement is calculated, that is, BV cur v =2( y nbr - y cur ) + BV nbr v .

[0365] 2.2.37 Combination of IBC with other codec tools 2.2.37.1 IBC Merge Mode with Block Vector Difference (IBC-MBVD) Affine-MMVD and GPM-MMVD have been adopted as extensions to the regular MMVD mode in ECM. Extending the MMVD mode to the IBC Merge mode is a natural progression.

[0366] In IBC-MBVD, the distance set is {1 pixel, 2 pixels, 4 pixels, 8 pixels, 12 pixels, 16 pixels, 24 pixels, 32 pixels, 40 pixels, 48 ​​pixels, 56 pixels, 64 pixels, 72 pixels, 80 pixels, 88 pixels, 96 pixels, 104 pixels, 112 pixels, 120 pixels, 128 pixels}, and the BVD direction is two horizontal directions and two vertical directions.

[0367] The basic candidate is selected from the top five candidates in the reordered IBC Merge list. And based on the SAD cost between the template (the row above and column to the left of the current block) and its reference for each basic candidate, all possible MBVD refinement positions (20×4) are reordered. Finally, the top 8 refinement positions with the lowest template SAD cost are retained as available positions for MBVD index encoding and decoding. The MBVD index is binary-coded using rice code with a parameter equal to 1.

[0368] In the IBC-MBVD list derivation, adaptive BVD offsets along the MVBD direction are enabled for the IBC MBVD mode. The MBVD candidate search is a two-step process that begins by checking the template SAD cost of the offsets added to the BVP at 1-pixel intervals along each direction. The second step of the search checks the template SAD cost of candidates surrounding the selected candidate from the first step at 1 / 4-pixel intervals. For integer MBVDs (where ph_fpel_mbvd_enabled_flag is 0 when present in the ECM), these intervals are multiplied by 4. The candidate with the lowest TM cost is included in the final MBVD list.

[0369] Blocks encoded and decoded by IBC-MBVD do not inherit the flip type from neighboring blocks encoded and decoded by RR-IBC.

[0370] 2.2.37.2 Combination of Intra-Block Copying and Intra-Prediction Intra-Block Copy and Intra-Prediction Combination (IBC-CIIP) is an encoding / decoding tool for CUs that uses IBC and intra-prediction to obtain two prediction signals, which are then weighted and summed to generate the final prediction, as shown below: , in and This represents the IBC prediction signal and the intra-frame prediction signal. For IBC Merge mode and IBCAMVP mode, It was set to equal to (13, 4) and (1, 1).

[0371] An intra-prediction mode (IPM) candidate list is used to generate the intra-prediction signal, and the IPM candidate list size is predefined to 2. The IPM index is transmitted via signaling to indicate which IPM to use.

[0372] 2.2.37.3 IBC with geometric segmentation Intra-Block Copy with Geometric Partitioning (IBC-GPM) is an encoding / decoding tool that geometrically partitions the CU into two sub-segments. Predictions for both sub-segments are generated using IBC and intra-prediction. IBC-GPM can be applied to either the regular IBC Merge mode or the IBC™ Merge mode. The Intra-Prediction Mode (IPM) candidate list is constructed using the same method as GPM with inter-frame and intra-frame prediction for intra-prediction, and the IPM candidate list size is predefined to 3. A total of 48 geometric partitioning modes are available, divided into two geometric partitioning mode sets, as follows: Table 7: Geometric segmentation patterns in the first set of geometric segmentation patterns

[0373] Table 8: Geometric segmentation patterns in the second set of geometric segmentation patterns

[0374] When using IBC-GPM, the IBC-GPM geometric segmentation mode set flag is signaled to indicate whether a first or second geometric segmentation mode set is selected, followed by the geometric segmentation mode index. The IBC-GPM intra-frame flag is signaled to indicate whether intra-frame prediction is used for the first sub-segmentation. When intra-frame prediction is used for sub-segmentation, the intra-frame prediction mode index is signaled. When IBC is used for sub-segmentation, the merge index is signaled.

[0375] In bidirectional prediction IBC GPM, two flags are transmitted via signaling to indicate the prediction modes of the two segments. The first flag indicates whether the first segment is intra-predicted; if not, the second flag is transmitted via signaling to indicate whether intra-prediction was used for the second segment. This method is only applied to SCC.

[0376] 2.2.37.4 IBC BVP-Merge and IBC Merge Based on Bidirectional Prediction IBC-BVP-Merge is similar to AMVP-Merge, deriving a BV from the IBC block vector prediction (BVP) and a second BV from the IBC merge to form a bidirectional prediction for IBC. The two distinct indices for the IBC BVP candidate and the IBC merge candidate are transmitted via signaling.

[0377] Bidirectional predictive IBC Merge is enabled along with MBVD and unidirectional Merge. In bidirectional predictive IBC Merge, two BVs from the existing IBC Merge candidate list are derived using two different indices transmitted via signaling. Bidirectional predictive IBC Merge is applied to both IBC regular Merge and IBC MBVD. For non-SCC categories, bidirectional predictive IBC Merge, IBC MBVD, and IBC unidirectional Merge are enabled.

[0378] 2.2.37.5 IBC MBVD List Derivation In Test 2.4a, adaptive BVD offsets along the MVBD direction are enabled for the IBC MBVD mode. The MBVD candidate search is a two-step process that begins by checking the template SAD cost of the offsets added to the BVP at 1-pixel intervals along each direction. The second step of the search checks the template SAD cost of candidates surrounding the selected candidate from the first step at 1 / 4-pixel intervals. For integer MBVDs (where ph_fpel_mbvd_enabled_flag is 0 when present in the ECM), these intervals are multiplied by 4. The candidate with the lowest TM cost is included in the final MBVD list.

[0379] 2.2.37.6 IBC with local lighting compensation Intra-Block Copy with Local Illumination Compensation (IBC-LIC) is an encoding / decoding tool that uses linear equations to compensate for local illumination variations within an image between an IBC-encoded CU and its predicted blocks. Except for the use of block vectors to generate a reference template in IBC-LIC, the derivation of the parameters of the linear equations is the same as for LIC used for inter-frame prediction. IBC-LIC can be applied to both IBC AMVP and IBC Merge modes. For IBC AMVP mode, an IBC-LIC flag is signaled to indicate the use of IBC-LIC. Top, left, or L-shaped templates are allowed for deriving single-model parameters. MMLM is extended to IBC-LIC, allowing IBC-LIC to have two linear models within a CU. Only L-shaped templates are used in IBC-LIC MMLM. The mode index is signaled. For IBC Merge mode, the IBC-LIC flag is presumed from the Merge candidate. The IBC-LIC flag is inherited from the IBC HMVP candidate to reconcile IBC HMVP and IBC-LIC in a manner similar to that of inter-frame LIC.

[0380] 2.2.38 Derivation of Template Matching-Based BCW Index for Merge Pattern The BCW index of the merged CU is derived based on template matching cost, rather than from neighboring blocks. Given a selected merge candidate, the TM cost is calculated using different bidirectional prediction weights, and then the bidirectional prediction weight with the minimum TM cost value is used to predict the merged CU.

[0381] When calculating the TM cost for weights in bidirectional prediction, the following rules apply: - Because the inherited bidirectional prediction weights may have higher accuracy than other weights, only the inherited bidirectional prediction weights and their two nearest neighbor weights (i.e., ±1) are considered. For example, if the inherited bidirectional prediction weight is 4, then only three weights {3, 4, 5} are involved in the TM cost calculation.

[0382] - The TM cost of the inherited BCW index is multiplied by 0.90625, which means the cost is reduced by 3 / 32.

[0383] - The cost of TM with equal weights is multiplied by 0.90625 because bidirectional prediction samples benefit BDOF, and BDOF is only applied to CUs with equal weights.

[0384] Template matching-based BCW index derivation was applied to CUs encoded and decoded using regular merge, template matching, adaptive decoder-side motion vector refinement, and MMVD modes.

[0385] Furthermore, the bidirectional prediction weights for the Merge mode are expanded from {-2, 3, 4, 5, 10} to {1, 2, 3, 4, 5, 6, 7}. Additionally, the negative bidirectional prediction weights {-2, 10} for non-Merge modes are replaced by positive weights {1, 7}.

[0386] 2.2.39 DMVR for blocks encoded / decoded via affine Merge When the DMVR condition is met, DMVR is applied to blocks encoded and decoded via affine Merge and blocks encoded and decoded via affine MMVD. It is also extended to the adaptive BM Merge mode.

[0387] The affine motion field is modeled as follows (6-parameter affine case):

[0388] ,in It is a location The motion vector at that location, and It is the fundamental MV representing the translational motion of an affine model. Parameters , , and This represents non-translation parameters (rotation, scaling).

[0389] Motion vector and The control point motion vector (CPMV) of the affine codec unit under consideration is used. During the DMVR process applied to the affine, the bilateral matching cost is calculated for each sub-block. The sub-block bilateral matching cost and the refined sub-block MV are then used to determine the overall optimal refined CPMV for the affine block. More specifically, the CPMV is refined according to the following steps: 1) Perform integer-pixel bilateral matching on sub-blocks. Accumulate the sub-block bilateral matching cost to determine the optimal integer-pixel MV offset.

[0390] 2) Perform a half-pixel bilateral matching search using the best integer MV offset as the initial offset, and output the best MV offset that minimizes the bilateral matching cost for the same set of sub-blocks in step 1.

[0391] 3) Perform linear regression using the refined sub-block MV from step 1 as input and output a set of control point motion vectors.

[0392] 4) Compare the bilateral matching costs of the outputs of steps 2 and 3 to select the one with the minimum cost.

[0393] Furthermore, after the basic MV is determined, the non-translation parameters of the affine model are refined. Each of the CPMVs is sequentially fixed to the basic MV, and an offset is added to the non-translation parameters of the affine model by minimizing the bilateral matching cost. Then, two more CPMVs are calculated based on the basic MV and the refined non-translation parameters.

[0394] For both affine Merge and affine MMVD modes, both CPMV refinement and non-translation parameter refinement are applied. When applied to the affine MMVD mode, if the basic candidate satisfies the affine DMVR refinement condition, the MMVD offset is added to the affine Merge basic candidate refined by the affine DMVR. For the adaptive BM Merge mode, an affine Merge list containing only affine Merge candidates that satisfy the affine DMVR condition is constructed, and then CPMV refinement and non-translation parameter refinement are applied.

[0395] 2.2.40 Inter-frame CCCM When the CU uses inter-frame prediction or intra-frame block copy (IBC), inter-frame CCCM applies the CCCM method to predict chromaticity samples from reconstructed luminance samples. Figure 49The decoder side of the method is shown. A cross-component filter is derived using prediction blocks for both luma and chroma. The derived filter is applied to reconstruct the luma block and mixed with the chroma prediction block to produce the final chroma prediction block. During the mixing process, the filtered reconstructed luma block uses a mixing weight of 0.75, and the chroma prediction block uses a mixing weight of 0.25.

[0396] An 8-tap filter consists of 6 spatial brightness samples, a nonlinear term, and a bias term. For example... Figure 50 As shown, the spatial luminance samples (L0, ..., L5) are obtained from the six luminance samples closest to the chromaticity position C selected from the luminance grid, without downsampling. The predicted chromaticity values ​​are obtained as follows: predChromaVal = c 0 L0+ c 1 L1 + c 2 L2 + c 3 L3 + c 4 L4 + c 5 L5 + c 6 nonlinear ((L0+L3+1)>>1) + c 7 B, in nonlinear `B` is the nonlinear operator of CCCM, and `B` is the bias. The filter coefficients are derived using ECM's division-free Gaussian elimination, and the necessary offset is applied to the samples before the filter derivation. The offset for division-free Gaussian elimination is obtained by averaging four points from the luma and chroma prediction blocks, where the four points correspond to the top-left, top-right, bottom-left, and bottom-right corners of the block. For the filter coefficient derivation, a maximum of 256 chroma samples are used.

[0397] The use of this mode is achieved through signal transmission of the TU level flag via CABAC encoding / decoding. A new CABAC context is included to support this. The inter-frame CCCM flag is only transmitted via signal transmission when the TU's luminance Cbf is not zero and the CU's predMode is MODE_INTER or MODE_IBC.

[0398] When the luminance Cbf is not zero and the predMode of the CU is MODE_INTER or MODE_IBC, the encoder performs RD decisions in the transform selection loop for the chrominance component.

[0399] 2.2.41 Chain motion vector prediction Chained MV prediction (CMVP) is introduced into the construction of the inter-frame merge candidate list.

[0400] like Figure 51As shown, CMVP candidates can be derived as the accumulation of MV and BV based on the recursive tracing of MVs for a pre-derived inter-frame merge candidate list. For example, CMVP candidates, a set of motion vector MVs... k / m and reference image RefPic k / m It can be derived using the following formula: MV k / m =MV k(0) + BV k(0) +MV k(1) +MV k(2) + … + MV k(m) , RefPic k / m =RefPic k(m) , Where k and m indicate the number of Merge indices and the tracking depth of the CMVP.

[0401] When deriving MV k / m At that time, MV k(m) By checking all five positions of the current block (such as...) Figure 52 The existence of the MV or BV in the corresponding MV / BV storage (as shown) (i.e., the center, top left, top right, bottom left, and bottom right of the current block) is found.

[0402] When the pre-derived Merge candidate for the CMVP candidate has two MVs, the MV is deduced for each Merge index, each list (i.e., L0 and L1), and each tracking depth. k / m At most two MVs can be derived for each list and each tracking depth, and the MV sets are sequentially inserted into the inter-frame merge candidate list.

[0403] Traceable reference images are only available within the reference image list.

[0404] For regular Merge and TM Merge, the CMVP candidate is inserted after the HMVP candidate.

[0405] When deriving CMVP candidates, hpelIfIdx, bcwIdx, licFlag, and mhpFlag are not inherited.

[0406] When TMVP is disabled, CMVP candidates are not inferred.

[0407] 3 questions In ECM, time-domain cache access is not always restricted to no more than the co-located CTU plus the four sample columns to the right of the co-located CTU, as shown in the table below.

[0408] Table 3-1. Temporal buffer requirements for encoding / decoding tools in ECM-11.0

[0409] 4 Detailed Solutions The detailed embodiments described below should be considered as examples for explaining general concepts. These embodiments should not be interpreted in a narrow sense. Furthermore, these embodiments can be combined in any way.

[0410] The terms “video unit” or “code-decoder unit” or “block” can refer to a picture, strip, slice, code-decoder tree block (CTB), code-decoder tree unit (CTU), code-decoder block (CB), CU, PU, ​​TU, PB, or TB.

[0411] The term "prediction cell" can refer to a prediction block or a prediction sample.

[0412] The term "CCP" can refer to any cross-component prediction method, such as any kind of LM / intra-frame CCLM / inter-frame CCCM / MMLM / CCCM / GLM / GL-CCCM / intra-frame CCP Merge / inter-frame CCP Merge. It can be used for intra-frame blocks, inter-frame blocks, or IBC blocks. It can be a CCP-based fusion mode.

[0413] It should be noted that the terms mentioned below are not limited to the specific terms defined in existing standards. Any changes to encoding / decoding tools also apply.

[0414] Time-domain cache, CTU cache 1) Temporal buffering limits can be applied to video units.

[0415] a) Time-domain cache can store: i) MV information; ii) CCP model information; iii) Intra-frame prediction mode; iv) Affine model information.

[0416] b) Temporal buffering limits can be imposed based on limiting the location of temporal blocks to a specific area of ​​the reference image.

[0417] i) For example, the position of the time-domain block can be limited to a position within the same CTU line.

[0418] ii) For example, the position of a time-domain block can be bounded to the position within the same CTU and the N (e.g., N=4) columns to the right of the same CTU.

[0419] iii) For example, the location of a time-domain block can be bounded to a location within a co-located CTU and within X (e.g., X=1 or X>1) CTUs to the right of the co-located CTU and Y (e.g., Y=1 or Y>1) CTUs to the left of the co-located CTU.

[0420] iv) For example, the position of a temporal block can be bound to a reference picture, or a co-position / reference subpicture, or a co-position / reference strip, or a position within a co-position / reference slice.

[0421] c) To identify time-domain candidates, time-domain cache limits can be imposed based on conditional checks of the restricted valid regions.

[0422] i) For example, when examining a predefined location of a temporal candidate in a temporal reference picture (e.g., for inter-frame CCPmerge mode, etc.), a temporal candidate may be considered unavailable if the location is outside the restricted valid area.

[0423] ii) For example, alternatively, when examining a predefined location of a temporal candidate in a temporal reference picture (e.g., for inter-frame CCP Merge mode, etc.), if the location is outside the restricted valid area, a second location within the restricted valid area may be used instead.

[0424] (1) For example, instead of searching for a predefined location of a time-domain candidate (in which case the predefined location is outside the restricted valid area), one can search for the nearest location within the restricted valid area.

[0425] iii) For example, the restricted effective area may be related to time-domain cache limitations, such as those related to co-occurring CTUs (e.g., co-occurring CTU rows, or co-occurring CTUs plus the four columns to the right, etc.).

[0426] d) In order to identify / store candidates for inter-frame propagation / chaining / cascading / automatic relocation, temporal buffer limits can be imposed based on conditional checks of the restricted valid regions.

[0427] i) For example, when examining the location of a candidate for inter-frame propagation / chaining / cascading / automatic relocation in a temporal reference image, if the location is outside the restricted valid area, the candidate can be considered unavailable.

[0428] ii) For example, alternatively, when examining candidate locations for inter-frame propagation / chaining / cascading / automatic relocation in a temporal reference picture, if the location is outside the restricted valid area, a second location within the restricted valid area may be used instead.

[0429] (1) For example, the nearest location within a limited valid area can be used.

[0430] iii) For example, the restricted effective area may be related to time-domain cache limitations, such as those related to co-occurring CTUs (e.g., co-occurring CTU rows, or co-occurring CTUs plus the four columns to the right, etc.).

[0431] e) For example, motion / CCP / mode information of time-domain blocks can be stored in a time-domain cache.

[0432] f) For example, temporal motion / CCP / mode candidates can be derived based on temporal blocks.

[0433] g) For example, motion / CCP / mode candidates for inter-frame propagation can be derived based on temporal blocks.

[0434] i) For example, inter-frame propagation candidates can refer to motion / CCP / mode candidates associated with a reference block (or a reference block of the current reference block, etc.).

[0435] h) For example, propagation / cascade / chain / automatic relocation motion / CCP / mode candidates can be derived based on more than one (or at least one) temporal block.

[0436] i) For example, candidates for propagation / cascade / chaining / automatic relocation can refer to motion candidates derived from the sum of MV / BVs of recursive tracing, where the MV / BVs of recursive tracing can be derived from pre-derived MV / BVs.

[0437] (1) For example, the length of the depth of recursive tracing can be equal to N (such as N=1 or N>1).

[0438] (2) For example, the constraint on the length of the depth of the recursive tracing can be left unapplied.

[0439] ii) For example, temporal cache limits can be imposed on each temporal block based on the traced MV / BV derivation.

[0440] (1) For example, assuming the length of the depth of the recursive tracing is N, all the traced temporal blocks in the traced reference images (e.g., different reference images for the traced MV, the same reference image for the traced BV) can be required to comply with temporal cache constraints.

[0441] (a) For example, the location of all tracked time-domain blocks may be required not to exceed the constraint area (e.g., based on co-location CTU rows or co-location CTUs, or reference images, or co-location / reference sub-images, or co-location / reference strips, or co-location / reference slices, etc.).

[0442] (b) For example, the location of all tracked temporal blocks can be bounded to a location within a constrained region (e.g., based on a co-location CTU row or co-location CTU, or a reference image, or a co-location / reference sub-image, or a co-location / reference strip, or a co-location / reference slice, etc.).

[0443] i) For example, whether and how temporal buffer limits are applied to video units can be transmitted via signaling or predefined.

[0444] j) Information regarding whether and how time-domain caching limits are applied can be transmitted via signals in VPS / SPS / PPS / sequence header / image header / strip header / CTU line / CTU / CU, etc.

[0445] i) Signaling can be performed conditionally.

[0446] (1) For example, information can only be transmitted via signals if the encoding and decoding tools are applicable.

[0447] (a) The encoding / decoding tool can be a temporal motion vector prediction tool.

[0448] (b) The encoding / decoding tool can be an inter-frame CCP.

[0449] k) For example, whether to apply time-domain buffer limits and / or how large a time-domain buffer is allowed for video units that can be transmitted via signal in the bitstream.

[0450] i) For example, whether temporal buffering limits are applied to video units can be transmitted via signals.

[0451] ii) For example, how much time-domain buffer is allowed to be used can be transmitted via signaling.

[0452] iii) For example, whether to apply time-domain buffer limits and / or how large a time-domain buffer is allowed for a sequence (or group of pictures) that can be transmitted via signaling in SPS (or PPS).

[0453] iv) For example, whether to apply time-domain buffer limits and / or how large a time-domain buffer is allowed for an image to be transmitted via signaling in the image header.

[0454] (1) For example, it can be based on low-latency (e.g., low-latency B and / or low-latency P) images.

[0455] (2) For example, it can be based on time-domain information.

[0456] v) For example, whether to apply time-domain buffer limits and / or how large a time-domain buffer is allowed for a stripe to be transmitted via signaling in the stripe header.

[0457] (1) For example, it can be based on the strip type (e.g., B strip, P strip).

[0458] vi) For example, whether to apply time-domain buffer limits and / or how large a time-domain buffer is allowed for blocks that can be signaled at the block level.

[0459] (1) For example, it can be based on prediction modes (e.g., intra-frame, inter-frame, IBC, PLT, CCP, etc.).

[0460] (2) For example, it can be based on prediction methods (e.g., intra-frame CCP, intra-frame CCP Merge, inter-frame CCP, inter-frame CCP Merge, template-based methods, GPM, affine, CIIP, etc.).

[0461] (3) For example, it can be based on block information (e.g., block height / width / width-to-height ratio / block area, etc.).

[0462] l) For example, whether to apply time-domain buffer limits and / or how large a time-domain buffer is allowed for video units can be predefined in the codec (i.e., without explicit signaling).

[0463] i) For example, predefined rules can be established to indicate which tool(s) should follow specific time-domain caching limits, and which tool(s) can be applied without time-domain caching limits.

[0464] (1) For example, CCP mode may not follow line cache limits, while non-CCP mode should follow line cache limits.

[0465] ii) For example, predefined rules can be established to indicate which tool(s) must comply with the first type of time-domain caching restrictions and which tool(s) must comply with the second type of time-domain caching restrictions.

[0466] (1) For example, different time-domain cache limits can define different regions of the time-domain cache.

[0467] iii) For example, predefined rules can be established to indicate which information must comply with the first type of time-domain caching restrictions and which information must comply with the second type of time-domain caching restrictions.

[0468] (1) For example, the information can be MV information / CCP model information / intra-frame prediction mode / affine model information.

[0469] iv) For example, predefined rules can be defined based on the type of video unit / encoding / decoding tool / encoding method, such as: (1) For example, it can be based on low-latency (e.g., low-latency B and / or low-latency P) images.

[0470] (2) For example, it can be based on time-domain information.

[0471] (3) For example, it can be based on the strip type (e.g., B strip, P strip).

[0472] (4) For example, it can be based on prediction modes (e.g., intra-frame, inter-frame, IBC, PLT, CCP, etc.).

[0473] (5) For example, it can be based on prediction methods (e.g., intra-frame CCP, intra-frame CCP Merge, inter-frame CCP, inter-frame CCP Merge, template-based methods, GPM, affine, CIIP, etc.).

[0474] (6) For example, it can be based on block information (e.g., block height / width / width-to-height ratio / block area, etc.).

[0475] 2) Encoding / decoding tools / methods may be required to follow predefined or time-domain buffering constraints via signal transmission.

[0476] a) For example, time-domain cache limits can be defined as one or a specific combination of the following: i) The position of the time-domain block can be required to not exceed the same time-domain CTU line.

[0477] ii) The position of the time-domain block can be required to not exceed the time-domain co-located CTU and N (e.g., N=4) columns to the right of the time-domain co-located CTU.

[0478] iii) The position of the time-domain block can be required to not exceed the time-domain co-located CTU and X (such as X=1 or X>1) CTUs to the right of the time-domain co-located CTU and Y (such as Y=1 or Y>1) CTUs to the left of the time-domain co-located CTU.

[0479] iv) The position of a temporal block may be required not to exceed the reference picture, or the same / reference subpicture, or the same / reference strip, or the same / reference slice.

[0480] v) For example, the position of a time-domain block can be bound to a position within the same CTU line.

[0481] vi) For example, the position of a time-domain block can be bounded to the position within the same CTU and the N (e.g., N=4) columns to the right of the same CTU.

[0482] vii) For example, the position of a time-domain block can be bounded to a position within a co-located CTU and within X (e.g., X=1 or X>1) CTUs to the right of the co-located CTU and Y (e.g., Y=1 or Y>1) CTUs to the left of the co-located CTU.

[0483] viii) For example, the position of a temporal block can be bound to a reference picture, or a co-position / reference subpicture, or a co-position / reference strip, or a position within a co-position / reference slice.

[0484] b) For example, all encoding / decoding tools / methods in a video codec may be required to comply with temporal buffering limits.

[0485] c) For example, alternatively, the following codec tools / methods may be required to comply with time-domain buffering constraints: i) Intra-frame CCP Merge (1) For example, the temporal candidate of the intra-frame CCP Merge mode can be required not to break the predefined temporal buffer limit.

[0486] ii) Inter-frame CCP Merge (1) For example, the temporal candidate of the inter-frame CCP Merge mode can be required not to break the predefined temporal buffer limit.

[0487] iii) MV information.

[0488] iv) Intra-frame mode propagation.

[0489] v) Affine model propagation.

[0490] 3) For example, the range / position / coordinates of temporal candidates / blocks (or inter-frame propagation candidates, or propagation / cascade / chain / automatic relocation motion / CCP / mode candidates, etc.) can be imposed without limitation.

[0491] a) For example, temporal candidates (or inter-frame propagation candidates) can be derived from temporal blocks located anywhere within a reference image.

[0492] b) For example, for the storage / derivation of inter-frame propagation candidates, the inter-frame propagation candidates can refer to any location within the reference image.

[0493] c) For example, the location of the tracked temporal block for propagation / cascade / chain / automatic relocation motion / CCP / pattern candidate can be located anywhere within the reference image.

[0494] 4) Specific encoding / decoding tools / methods can access the temporal buffer of an area greater than the co-occurrence CTU plus K (e.g., K=4) columns to the right of the co-occurrence CTU.

[0495] a) In addition, for example, it can access time-domain caches with an area greater than that of the corresponding CTU row.

[0496] b) In addition, for example, it can access the temporal cache of the entire image range.

[0497] c) For example, it can access the temporal cache of a range of co-position / reference sub-pictures, or co-position / reference strips, or co-position / reference slices.

[0498] 5) The range / location / coordinates of temporal candidates / blocks (or inter-frame propagation candidates) can be constrained to not exceed a specific region.

[0499] a) For example, a specific region can be defined as a reference picture, or a parietal / reference subpicture, or a parietal / reference strip, or a parietal / reference slice.

[0500] b) For example, a specific region can be defined as the same CTU line and the X line above the same CTU line.

[0501] i) For example, X can be equal to a constant (such as X=0 or X>0).

[0502] c) For example, for storing inter-frame propagation candidates, a candidate can only be considered a valid inter-frame propagation candidate for storage if the position referenced by the candidate does not exceed the limited range in the reference image.

[0503] i) Otherwise, the default value can be stored as a candidate for inter-frame propagation.

[0504] d) For example, for storing inter-frame propagation candidates, if the location referenced by the inter-frame propagation candidate exceeds the restricted range in the reference image, it can be bounded to the restricted range for storage.

[0505] e) For example, for storing candidates for inter-frame propagation, the method in b) is used for storing at least one codec tool / mode / information. And for at least one other codec tool / mode / information, the method in c) is used for storing.

[0506] 6) The location of the temporal block used to derive propagation / cascade / chain / automatic relocation motion / CCP / mode candidate can be constrained to a specific region.

[0507] a) For example, a constraint can be defined as limiting the location of a time-domain block to the effective region.

[0508] i) For example, the effective area can be defined as the same CTU line and the X line above the same CTU line.

[0509] (1) For example, X can be equal to a constant (such as X=0 or X>0).

[0510] ii) For example, the effective region can be defined as a reference picture, or a parietal / reference subpicture, or a parietal / reference strip, or a parietal / reference slice.

[0511] b) For example, a constraint can be defined as such that a temporal candidate can be considered invalid if the location of the temporal block is outside the valid region.

[0512] i) For example, the effective area can be defined as the same CTU line and the X line above the same CTU line.

[0513] (1) For example, X can be equal to a constant (such as X=0 or X>0).

[0514] ii) For example, the effective region can be defined as a reference picture, or a parietal / reference subpicture, or a parietal / reference strip, or a parietal / reference slice.

[0515] c) For example, a time-domain block can refer to a reference block derived based on the traced MV / BV.

[0516] i) For example, more than one temporal block can be derived (e.g., from different reference images identified by the tracked MV, or from sample reference images identified by the tracked BV, etc.).

[0517] d) For example, all time-domain blocks along the tracing path can be required to comply with time-domain cache limits.

[0518] e) For example, the final temporal block derived from the sum of MV / BV based on recursive tracing can be required to comply with temporal cache constraints.

[0519] 7) For candidate derivations of temporal / inter-frame propagation / chaining / cascading / automatic relocation, and / or for the storage of encoding and decoding information in temporal / inter-frame propagation / chaining / cascading / automatic relocation, no co-location constraints can be applied.

[0520] a) For example, for candidate derivation of temporal / inter-frame propagation / chaining / cascading / automatic relocation, such candidates can be obtained from previously encoded / decoded images, which may not be co-located images of the current image.

[0521] i) For example, candidates for temporal / inter-frame propagation / chaining / cascading / automatic relocation do not necessarily need to be obtained from the stored information of the co-located images.

[0522] ii) For example, candidates for temporal / inter-frame propagation / chaining / cascading / automatic relocation can be obtained from the stored information of any reference image of the current image.

[0523] iii) For example, candidates for temporal / inter-frame propagation / chaining / cascading / automatic relocation can be obtained from the storage information of any available image in the decoded image buffer (DPB).

[0524] iv) For example, a temporal cache can be maintained to store the encoding and decoding information for each available image in the Decoded Image Cache (DPB).

[0525] v) For example, for each available image in the DPB, the checklist / order of potential temporal candidates can be predefined.

[0526] b) For example, for candidate storage of inter-frame propagation / chaining / cascading / automatic relocation, if the motion vector used to identify such a candidate points to a reference image that is not a co-located image of the current image, then the encoding and decoding information of such a candidate can be acquired and stored.

[0527] 8) For candidate derivations of temporal / inter-frame propagation / chaining / cascading / automatic relocation, no motion vector scaling process can be applied.

[0528] a) For example, candidates for temporal / inter-frame propagation / chaining / cascading / automatic relocation can point to any reference image (which is not constrained to a co-located image).

[0529] b) For example, if the candidate pointer for temporal / inter-frame propagation / chaining / cascading / automatic relocation is not a reference image of the co-image of the current image, then the motion information of such candidate pointers (e.g., motion vectors, reference indices, etc.) can be used as is without scaling to the co-image.

[0530] 9) For example, the above time-domain candidates could be: a) Temporal mode candidates (such as intra / inter / IBC / PLT / BDPCM mode, SCC mode, CCP mode, GPM / SGPM / affine / CIIP / MHP, etc.).

[0531] b) Temporal motion candidates (such as MV / BV / refIdx, etc.).

[0532] i) For example, inter-frame merge TMVP candidates; ii) For example, inter-frame BM Merge TMVP candidates; iii) For example, inter-frame AMVP and TMVP candidates; iv) For example, affine Merge TMVP candidates; v) For example, BM Merge affine TMVP candidate; vi) For example, affine AMVP and TMVP candidates; vii) For example, sbTMVP motion displacement candidate.

[0533] c) Temporal model candidates (such as CCP models).

[0534] i) For example, intra-frame CCP model candidates.

[0535] ii) For example, inter-frame CCP model candidates.

[0536] 10) For example, the candidates for the above inter-frame propagation could be: a) For example, intra-prediction modes propagated between frames (e.g., for TIMD candidates, or SGPM / GPM candidates, etc.). b) For example, CCP models for inter-frame propagation (e.g., for inter-frame CCP merge candidates, etc.) c) For example, SCC modes propagated across frames (e.g., for OBMC block-level SCC detection, etc.) 11) For example, the candidates for the above propagation / cascade / chain / automatic relocation could be: a) Motion vector derived from the sum of MV / BV based on recursive tracing of more than one temporal block.

[0537] b) A CCP model derived from more than one temporal block identified by the tracked MV / BV.

[0538] c) Intra-frame / inter-frame / IBC / PLT / SCC / transform mode information derived from more than one temporal block identified by the tracked MV / BV.

[0539] General aspects : 12) The disclosed methods can be used in a single tree.

[0540] 13) The disclosed methods can be used in two trees.

[0541] 14) The disclosed method can be used for chroma encoding and decoding.

[0542] 15) The disclosed method can be used for luminance encoding and decoding.

[0543] 16) The disclosed method can be used for intra-frame block encoding and decoding.

[0544] 17) The disclosed method can be used for inter-frame block encoding and decoding.

[0545] 18) The disclosed method can be used for IBC block encoding and decoding.

[0546] 19) The disclosed method can be used in inter-frame (such as B or P) stripes.

[0547] 20) The disclosed method can be used in intra-frame (such as I) stripes.

[0548] 21) Whether and / or how the methods disclosed above can be applied can be transmitted via signaling at the sequence level / picture group level / picture level / strip level / piece group level, such as in the sequence header / picture header / SPS / VPS / DPS / DCI / PPS / APS / strip header / piece group header.

[0549] 22) Whether and / or how the methods disclosed above can be applied to transmit signals at PB / TB / CB / PU / TU / CU / VPDU / CTU / CTU lines / strips / films / sub-images / other types of areas containing more than one sample point or pixel.

[0550] 23) Whether and / or how the methods disclosed above are applied may depend on the encoded / decoded information, such as block size, color format, single / double tree segmentation, color components, and stripe / picture type.

[0551] Figure 53 A flowchart of a method 5300 for video processing according to an embodiment of the present disclosure is shown. Method 5300 is implemented during the conversion between video units of a video and a bitstream of a video.

[0552] At box 5310, for the conversion between video units and video bitstreams, it is determined whether to apply at least one isotopic image constraint or motion vector scaling procedure to the candidate derivation or codec information storage associated with the video unit. For example, candidate derivation may include at least one of the following: temporal candidate derivation, inter-frame propagation candidate derivation, chained candidate derivation, cascaded candidate derivation, or automatic relocation candidate derivation. As another example, codec information storage may include at least one of the following: temporal codec information storage, inter-frame propagation codec information storage, chained codec information storage, cascaded codec information storage, or automatic relocation codec information storage.

[0553] At block 5320, a conversion is performed based on a determination. In some embodiments, the conversion may include encoding video units into a bitstream. Alternatively or additionally, the conversion may include decoding video units from the bitstream. Method 5300 can advantageously improve encoding / decoding efficiency and quality.

[0554] In some embodiments, the absence of a co-location image constraint can be applied to candidate derivation or codec information storage. In some embodiments, for candidate derivation, candidates can be obtained from previously encoded / decoded images. In this case, the previously encoded / decoded image is not a co-location image of the current image, and the candidates include at least one of the following: temporal candidates, inter-frame propagation candidates, chained candidates, cascaded candidates, or automatically relocated candidates.

[0555] In some embodiments, candidates may not be obtained from the storage information of co-located images. In other words, candidates do not necessarily need to be obtained from the storage information of co-located images. In some other embodiments, candidates may be obtained from the storage information of reference images of the current image, such as any reference image of the current image. Alternatively, candidates may be obtained from the storage information of available images in the Decoded Image Buffer (DPB). In some embodiments, the temporal buffer may be used to store the encoding and decoding information of available images in the DPB. In some other embodiments, the checklist of potential temporal candidates and / or the order of potential temporal candidates may be predetermined for available images in the DPB.

[0556] In some other embodiments, for candidate storage, if the motion vector used to identify a candidate points to a reference image that is not a co-located image of the current image, then the candidate's encoding / decoding information can be obtained and stored. In this case, the candidate includes at least one of the following: inter-frame propagation candidates, chained candidates, cascaded candidates, or automatically relocated candidates.

[0557] In some embodiments, the motion vector scaling process may not be applied to candidate derivation. In some embodiments, the candidate may point to a reference image. In this case, the candidate includes at least one of the following: temporal candidate, inter-frame propagation candidate, chained candidate, cascaded candidate, or auto-relocation candidate. Additionally, the reference image is not constrained to a co-located image. In some other embodiments, if the candidate points to a reference image that is not a co-located image of the current image, the motion information of the candidate can be used as is without scaling to the co-located image. In this case, the candidate includes at least one of the following: temporal candidate, inter-frame propagation candidate, or chained candidate. For example, the motion information may include at least one of a motion vector or a reference index.

[0558] In some embodiments, temporal buffer restrictions can be applied to video units. In some embodiments, temporal buffer restrictions can be imposed to identify temporal candidates based on conditional checks of restricted valid regions.

[0559] In some embodiments, if a predetermined location (such as an inter-frame CCP Merge pattern) of a temporal candidate in a temporal reference image is determined to be outside a restricted valid region, the temporal candidate may be unavailable. Alternatively, if a predetermined location (such as an inter-frame CCP Merge pattern) of a temporal candidate in a temporal reference image is determined to be outside a restricted valid region, another location within the restricted valid region may be used. For example, the nearest location within the restricted valid region may be searched instead of the predetermined location of the temporal candidate (in this case, the predetermined location is outside the restricted valid region). In some other embodiments, the restricted valid region may relate to limitations of the temporal buffer, for example, limitations related to code-decoder tree units (CTUs) (such as co-occurring CTU rows, co-occurring CTUs plus the four columns to the right, etc.).

[0560] In some embodiments, based on a conditional check of the restricted valid region, a temporal cache constraint may be imposed to identify or store candidates including at least one of the following: inter-frame propagation candidates, chained candidates, cascaded candidates, or automatically relocated candidates. In some embodiments, if the location of a candidate in the temporal reference image is determined to be outside the restricted valid region, the candidate may be unavailable. Alternatively, if the location of a candidate in the temporal reference image is determined to be outside the restricted valid region, another location within the restricted valid region may be used. For example, the nearest location within the restricted valid region may be used. In some other embodiments, the restricted valid region may be related to a temporal cache constraint, for example, a constraint related to a codec tree unit (CTU) (such as a co-occurring CTU row, a co-occurring CTU plus the four columns to the right, etc.).

[0561] In some embodiments, the temporal candidate can be at least one of the following: temporal mode candidate, temporal motion candidate (such as MV, BV, refIdx, etc.), or temporal model candidate (such as CCP model, etc.). For example, the temporal mode candidate can include at least one of the following: intra-frame mode candidate, inter-frame mode candidate, palette mode candidate, block-based differential pulse codec modulation (BDPCM) mode candidate, screen content codec (SCC) mode candidate, CCP mode candidate, geometric segmentation mode (GPM) candidate, spatial geometric segmentation mode (SGPM) candidate, affine mode candidate, inter-frame and intra-frame joint prediction (CIIP) mode candidate, or multiple hypothesis prediction (MHP) mode candidate. In some embodiments, temporal motion candidates may include at least one of the following: inter-frame merge temporal motion vector prediction (TMVP) candidates, inter-frame bilateral matching (BM) merge TMVP candidates, inter-frame advanced motion vector prediction (AMVP) TMVP candidates, affine merge TMVP candidates, BM merge affine TMVP candidates, affine AMVP TMVP candidates, or sub-block-based temporal motion vector prediction (SbTMVP) motion displacement candidates. In some embodiments, temporal model candidates may include at least one of the following: intra-frame cross-component prediction (CCP) model candidates, or inter-frame CCP model candidates.

[0562] In some embodiments, inter-frame propagation candidates can be at least one of the following: inter-frame propagation intra-prediction mode candidates, inter-frame propagation CCP model candidates, or inter-frame propagation SCC mode candidates. For example, inter-frame propagation intra-prediction mode candidates can include at least one of the following: template-based intra-mode derivation (TIMD) candidates, SGPM candidates, or GPM candidates. As another example, inter-frame propagation CCP model candidates can include inter-frame CCP merge candidates. In some examples, inter-frame propagation SCC mode candidates can include Overlapping Block Motion Compensation (OBMC) block-level SCC detection candidates.

[0563] In some embodiments, candidates can be one of the following: propagated candidates, cascaded candidates, chained candidates, or automatic relocation candidates, and candidates can include at least one of the following: motion vectors, CCP models, intra-frame mode information, inter-frame mode information, intra-block copy (IBC) mode information, palette (PLT) mode information, SCC mode information, or transform mode information. For example, motion vectors can be derived based on the sum of motion vectors and / or block vectors recursively tracked across multiple temporal blocks. In some examples, CCP models can be derived based on multiple temporal blocks identified by tracked motion vectors or block vectors. As an example, at least one of the following: intra-frame mode information, inter-frame mode information, IBC mode information, PLT mode information, SCC mode information, or transform mode information can be derived based on multiple temporal blocks identified by tracked motion vectors or block vectors.

[0564] In some embodiments, the method of performing a conversion based on determining whether to apply at least one isotopic picture constraint or motion vector scaling procedure to the candidate derivation or codec information storage associated with the video unit can be used in one of the following ways: single-tree, dual-tree, inter-frame stripe (such as B-strip and / or P-strip), or intra-frame stripe (such as I-strip). In some embodiments, the method of performing a conversion based on determining whether to apply at least one isotopic picture constraint or motion vector scaling procedure to the candidate derivation or codec information storage associated with the video unit can be used in one of the following ways: chroma codec, luma codec, intra-frame block codec, inter-frame block codec, or intra-frame block copy (IBC) block codec.

[0565] In some embodiments, the method of performing a transformation based on determining whether to apply at least one isotopic picture constraint or motion vector scaling procedure to the candidate derivation or codec information storage associated with the video unit may be indicated at one of the following: sequence level, picture group level, picture level, strip level, or slice group level. In some embodiments, the method of performing a transformation based on determining whether to apply at least one isotopic picture constraint or motion vector scaling procedure to the candidate derivation or codec information storage associated with the video unit may be indicated at one of the following: sequence header, picture header, sequence parameter set (SPS), video parameter set (VPS), dependency parameter set (DPS), decoding capability information (DCI), picture parameter set (PPS), adaptive parameter set (APS), strip header, or slice group header.

[0566] In some embodiments, the method of performing a transformation based on determining whether to apply at least one isotopic image constraint or motion vector scaling process to a candidate derivation or codec information storage associated with a video unit may be indicated by one of the following: prediction block (PB), transform block (TB), codec block (CB), prediction unit (PU), transform unit (TU), codec unit (CU), virtual pipeline data unit (VPDU), codec tree unit (CTU), CTU row, strip, slice, sub-picture, or region containing more than one sample point or pixel.

[0567] In some embodiments, the method of performing the conversion based on determining whether to apply at least one isotopic image constraint or motion vector scaling process to the candidate derivation or codec information storage associated with the video unit may depend on the codec information. For example, the codec information may include at least one of the following: block size, color format, single-tree segmentation, dual-tree segmentation, color components, stripe type, or picture type.

[0568] According to another embodiment of this disclosure, a non-transitory computer-readable recording medium is provided. This non-transitory computer-readable recording medium stores a bitstream of video generated by a method performed by a video processing apparatus. The method includes: determining whether to apply at least one isotopic picture constraint or motion vector scaling process to candidate derivation or encoding / decoding information storage associated with video units of the video; and generating a bitstream based on the determination.

[0569] According to further embodiments of this disclosure, a method for storing a bitstream of video is provided. The method includes: determining whether to apply at least one isotopic image constraint or motion vector scaling process to candidate derivation or encoding / decoding information storage associated with video units of the video; generating a bitstream based on the determination; and storing the bitstream in a non-transitory computer-readable recording medium.

[0570] The embodiments of this disclosure can be described according to the following entries, and their features can be combined in any reasonable manner.

[0571] Item 1. A video processing method, comprising: for a conversion between a video unit of a video and a bitstream of the video, determining whether to apply at least one isotopic image constraint or motion vector scaling process to a candidate derivation or encoding / decoding information storage associated with the video unit; and performing the conversion based on the determination.

[0572] Item 2. The method according to Item 1, wherein the candidate derivation includes at least one of the following: temporal candidate derivation, inter-frame propagation candidate derivation, chained candidate derivation, cascaded candidate derivation, or automatic relocation candidate derivation, and wherein the codec information storage includes at least one of the following: temporal codec information storage, inter-frame propagation codec information storage, chained codec information storage, cascaded codec information storage, or automatic relocation codec information storage.

[0573] Item 3. The method according to Item 1, wherein no co-position image constraint is applied to the candidate derivation or the encoding / decoding information storage.

[0574] Item 4. The method according to Item 3, wherein for the candidate derivation, the candidate is obtained from a previously encoded / decoded image, wherein the previously encoded / decoded image is not a co-located image of the current image, and the candidate includes at least one of the following: a temporal candidate, an inter-frame propagation candidate, a chained candidate, a cascaded candidate, or an automatic relocation candidate.

[0575] Item 5. The method according to Item 4, wherein the candidate is not obtained from the storage information of the co-position image.

[0576] Item 6. The method according to Item 4, wherein the candidate is obtained from the storage information of the reference image of the current image.

[0577] Item 7. The method according to Item 4, wherein the storage information of the candidate available images from the Decoded Image Buffer (DPB) is obtained.

[0578] Item 8. The method according to Item 4, wherein the temporal buffer is used to store encoding and decoding information of available images in the DPB.

[0579] Item 9. The method according to Item 4, wherein for available images in the DPB, the checklist of potential temporal candidates and / or the order of the potential temporal candidates are predetermined.

[0580] Item 10. The method according to Item 3, wherein, for candidate storage, if the motion vector used to identify a candidate points to a reference image that is not a co-located image of the current image, the encoding and decoding information of the candidate is obtained and stored, the candidate including at least one of the following: inter-frame propagation candidate, chained candidate, cascaded candidate, or automatic relocation candidate.

[0581] Item 11. The method according to Item 1, wherein the motion vector scaling process is not applied to the candidate derivation.

[0582] Item 12. The method according to Item 11, wherein the candidate points to a reference image, the candidate including at least one of the following: a temporal candidate, an inter-frame propagation candidate, a chained candidate, a cascaded candidate, or an automatic relocation candidate, wherein the reference image is not constrained to a co-located image.

[0583] Item 13. The method according to Item 11, wherein if a candidate points to a reference image that is not a co-located image of the current image, the motion information of the candidate is used without scaling to the co-located image, the candidate including at least one of the following: a temporal candidate, an inter-frame propagation candidate, or a chained candidate.

[0584] Item 14. The method according to Item 13, wherein the motion information includes at least one of a motion vector or a reference index.

[0585] Item 15. The method according to Item 1, wherein a temporal buffer constraint is applied to the video unit.

[0586] Item 16. The method according to Item 15, wherein the restriction of the time-domain cache is imposed to identify time-domain candidates based on a condition check of the restricted valid region.

[0587] Item 17. The method according to Item 16, wherein the time-domain candidate is unavailable if the predetermined position of the time-domain candidate in the time-domain reference picture is determined to be outside the restricted effective region.

[0588] Item 18. The method according to Item 16, wherein if a predetermined position of the time-domain candidate in the time-domain reference picture is determined to be outside the restricted effective region, then another position within the restricted effective region is used.

[0589] Item 19. The method according to Item 18, wherein the nearest location within the restricted effective area is searched.

[0590] Item 20. The method according to Item 16, wherein the restricted valid region relates to the limitation of the time-domain cache, wherein the limitation relates to the code-decode tree unit (CTU).

[0591] Item 21. The method according to Item 15, wherein the limitation of the temporal buffer is imposed to identify or store candidates including at least one of the following, based on a condition check of the restricted valid region: inter-frame propagation candidates, chained candidates, cascaded candidates, or automatic relocation candidates.

[0592] Item 22. The method according to Item 21, wherein if the location of the candidate in the time-domain reference picture is determined to be outside the restricted effective region, the candidate is unavailable.

[0593] Item 23. The method according to Item 21, wherein if the candidate location in the time-domain reference image is determined to be outside the restricted valid region, then another location within the restricted valid region is used.

[0594] Item 24. The method according to Item 23, wherein the nearest location within the restricted effective area is used.

[0595] Item 25. The method according to Item 21, wherein the restricted valid region relates to the limitation of the time-domain cache, wherein the limitation relates to the code-decode tree unit (CTU).

[0596] Item 26. The method according to any one of Items 1 to 25, wherein the time-domain candidate is at least one of the following: a time-domain pattern candidate, a time-domain motion candidate, or a time-domain model candidate.

[0597] Item 27. The method according to Item 26, wherein the temporal mode candidate includes at least one of the following: intra-frame mode candidate, inter-frame mode candidate, palette mode candidate, block-based differential pulse codec modulation (BDPCM) mode candidate, screen content codec (SCC) mode candidate, CCP mode candidate, geometric segmentation mode (GPM) candidate, spatial geometric segmentation mode (SGPM) candidate, affine mode candidate, inter-frame and intra-frame joint prediction (CIIP) mode candidate, or multiple hypothesis prediction (MHP) mode candidate.

[0598] Item 28. The method according to Item 26, wherein the temporal motion candidate includes at least one of the following: inter-frame merge temporal motion vector prediction (TMVP) candidate, inter-frame bilateral matching (BM) merge TMVP candidate, inter-frame advanced motion vector prediction (AMVP) TMVP candidate, affine merge TMVP candidate, BM merge affine TMVP candidate, affine AMVP TMVP candidate, or sub-block-based temporal motion vector prediction (SbTMVP) motion displacement candidate.

[0599] Item 29. The method according to Item 26, wherein the temporal model candidate includes at least one of the following: an intra-frame cross-component prediction (CCP) model candidate, or an inter-frame CCP model candidate.

[0600] Item 30. The method according to any one of Items 1 to 25, wherein the inter-frame propagation candidate is at least one of the following: an intra-frame prediction mode candidate for inter-frame propagation, a CCP model candidate for inter-frame propagation, or an SCC mode candidate for inter-frame propagation.

[0601] Item 31. The method according to Item 30, wherein the inter-frame propagation intra-prediction mode candidate includes at least one of the following: template-based intra-frame mode derivation (TIMD) candidate, SGPM candidate or GPM candidate, and / or wherein the inter-frame propagation CCP model candidate includes an inter-frame CCP Merge candidate, and / or wherein the inter-frame propagation SCC mode candidate includes an Overlapping Block Motion Compensation (OBMC) block-level SCC detection candidate.

[0602] Item 32. The method according to any one of items 1 to 25, wherein the candidate is one of the following: the propagation candidate, the cascaded candidate, the chained candidate, or the motion candidate of the automatic relocation, and the candidate includes at least one of the following: motion vector, CCP model, intra-frame mode information, inter-frame mode information, intra-block copy (IBC) mode information, palette (PLT) mode information, SCC mode information, or transform mode information.

[0603] Item 33. The method according to Item 32, wherein the motion vector is derived based on the sum of motion vectors and / or block vectors recursively traced across multiple time-domain blocks.

[0604] Item 34. The method according to Item 32, wherein the CCP model is derived based on multiple temporal blocks identified by tracked motion vectors or block vectors.

[0605] Item 35. The method according to Item 32, wherein at least one of the following: the intra-frame mode information, the inter-frame mode information, the IBC mode information, the PLT mode information, the SCC mode information, or the transform mode information is derived based on a plurality of temporal blocks identified by tracked motion vectors or block vectors.

[0606] Item 36. The method according to any one of items 1 to 35, wherein the method of performing the transformation based on determining whether to apply at least one co-location picture constraint or motion vector scaling process to the candidate derivation or codec information storage associated with the video unit is used in one of the following: single-tree, dual-tree, inter-frame striping, or intra-frame striping.

[0607] Item 37. The method according to any one of items 1 to 35, wherein the method of performing the conversion based on determining whether to apply at least one isotopic picture constraint or motion vector scaling process to a candidate derivation or codec information storage associated with the video unit is used for one of: chroma codec, luma codec, intra-block codec, inter-block codec, or intra-block copy (IBC) block codec.

[0608] Item 38. The method according to any one of items 1 to 37, wherein the method of performing the transformation based on determining whether to apply at least one isotopic picture constraint or motion vector scaling process to the candidate derivation or codec information storage associated with the video unit is indicated at one of the following: sequence level, picture group level, picture level, strip level, or slice group level.

[0609] Item 39. The method according to any one of items 1 to 37, wherein the method of performing the transformation based on determining whether to apply at least one isotopic picture constraint or motion vector scaling process to a candidate derivation or codec information storage associated with the video unit is indicated by one of the following: sequence header, picture header, sequence parameter set (SPS), video parameter set (VPS), dependency parameter set (DPS), decoding capability information (DCI), picture parameter set (PPS), adaptive parameter set (APS), strip header, or slice header.

[0610] Item 40. The method according to any one of items 1 to 37, wherein the method of performing the transformation based on determining whether to apply at least one isotopic picture constraint or motion vector scaling process to a candidate derivation or codec information storage associated with the video unit is indicated in one of the following: prediction block (PB), transform block (TB), codec block (CB), prediction unit (PU), transform unit (TU), codec unit (CU), virtual pipeline data unit (VPDU), codec tree unit (CTU), CTU row, strip, slice, sub-picture, or region containing more than one sample point or pixel.

[0611] Item 41. The method according to any one of items 1 to 37, wherein the method of performing the conversion based on determining whether to apply at least one co-position image constraint or motion vector scaling process to candidate derivation or codec information storage associated with the video unit depends on the codec information.

[0612] Item 42. The method according to Item 41, wherein the encoded / decoded information includes at least one of the following: block size, color format, single-tree segmentation, dual-tree segmentation, color components, stripe type, or image type.

[0613] Item 43. The method according to any one of items 1 to 42, wherein the conversion includes encoding the video unit into the bitstream.

[0614] Item 44. The method according to any one of items 1 to 42, wherein the conversion includes decoding the video unit from the bitstream.

[0615] Item 45. An apparatus for video processing, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform a method according to any one of items 1 to 44.

[0616] Item 46. A non-transitory computer-readable storage medium storing instructions that cause a processor to execute the method according to any one of items 1 to 44.

[0617] Item 47. A non-transitory computer-readable recording medium storing a bitstream of video generated by a method performed by a video processing apparatus, wherein the method includes: determining whether to apply at least one isotopic picture constraint or motion vector scaling process to candidate derivation or encoding / decoding information storage associated with video units of the video; and generating the bitstream based on the determination.

[0618] Item 48. A method for storing a bitstream of video, comprising: determining whether to apply at least one isotopic image constraint or motion vector scaling process to candidate derivation or encoding / decoding information storage associated with video units of the video; generating the bitstream based on the determination; and storing the bitstream in a non-transitory computer-readable recording medium.

[0619] Example device Figure 54 A block diagram of a computing device 5400 in which various embodiments of the present disclosure may be implemented is shown. The computing device 5400 may be implemented as a source device 110 (or video encoder 114 or 200) or a destination device 120 (or video decoder 124 or 300), or may be included in a source device 110 (or video encoder 114 or 200) or a destination device 120 (or video decoder 124 or 300).

[0620] It should be understood that, Figure 54 The computing device 5400 shown is for illustrative purposes only and is not intended to imply any limitation on the functionality and scope of the embodiments of this disclosure.

[0621] like Figure 54 As shown, computing device 5400 includes general-purpose computing device 5400. Computing device 5400 may include at least one or more processors or processing units 5410, memory 5420, storage unit 5430, one or more communication units 5440, one or more input devices 5450, and one or more output devices 5460.

[0622] In some embodiments, the computing device 5400 can be implemented as any user terminal or server terminal with computing capabilities. The server terminal can be a server provided by a service provider, a large computing device, etc. The user terminal can be, for example, any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, stations, units, devices, multimedia computers, multimedia tablet computers, internet nodes, communicators, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, and includes accessories and peripherals of these devices, or any combination thereof. It is conceivable that the computing device 5400 can support any type of interface to the user (such as "wearable" circuitry devices, etc.).

[0623] Processing unit 5410 can be a physical processor or a virtual processor, and can perform various processes based on programs stored in memory 5420. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capabilities of computing device 5400. Processing unit 5410 may also be referred to as a central processing unit (CPU), microprocessor, controller, or microcontroller.

[0624] Computing device 5400 typically includes various computer storage media. Such media can be any media accessible by computing device 5400, including but not limited to volatile and non-volatile media, or removable and non-removable media. Memory 5420 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (such as read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), or flash memory) or any combination thereof. Storage cell 5430 can be any removable or non-removable media and may include machine-readable media, such as memory, flash drives, disks, or other media that can be used to store information and / or data and can be accessed within computing device 5400.

[0625] The computing device 5400 may also include additional removable / non-removable storage media, volatile / non-volatile storage media. Although in Figure 54 Not shown, but may provide disk drives for reading from and / or writing to removable non-volatile disks, and optical disc drives for reading from and / or writing to removable non-volatile optical discs. In this case, each drive may be connected to a bus (not shown) via one or more data media interfaces.

[0626] Communication unit 5440 communicates with another computing device via a communication medium. Furthermore, the functionality of components in computing device 5400 can be implemented by a single computing cluster or by multiple computing machines communicating via communication connections. Therefore, computing device 5400 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or other general-purpose network nodes.

[0627] Input device 5450 can be one or more of various input devices, such as a mouse, keyboard, trackball, voice input device, etc. Output device 5460 can be one or more of various output devices, such as a monitor, speaker, printer, etc. With the aid of communication unit 5440, computing device 5400 can also communicate with one or more external devices (not shown), such as storage devices and display devices. Computing device 5400 can also communicate with one or more devices that enable a user to interact with computing device 5400, or any device that enables computing device 5400 to communicate with one or more other computing devices (e.g., network card, modem, etc.), if needed. Such communication can be performed via an input / output (I / O) interface (not shown).

[0628] In some embodiments, some or all components of computing device 5400 may not be integrated into a single device, but may be deployed in a cloud computing architecture. In a cloud computing architecture, components may be provided remotely and may work together to achieve the functionality described herein. In some embodiments, cloud computing provides computing, software, data access, and storage services without requiring end users to know the physical location or configuration of the systems or hardware providing these services. In various embodiments, cloud computing provides services via a wide area network (WAN), such as the Internet, using suitable protocols. For example, a cloud computing provider offers applications via a WAN that can be accessed through a web browser or any other computing component. The software or components of the cloud computing architecture, along with the corresponding data, may be stored on servers at remote locations. Computing resources in a cloud computing environment may be consolidated or distributed at locations in remote data centers. Cloud computing infrastructure may provide services through shared data centers, although they may appear as a single access point for users. Therefore, cloud computing architectures can be used to provide the components and functionality described herein from service providers at remote locations. Alternatively, they may be provided from conventional servers or directly or otherwise installed on client devices.

[0629] The computing device 5400 can be used to implement video encoding / decoding in the embodiments of this disclosure. The memory 5420 may include one or more video encoding / decoding modules 5425 having one or more program instructions. These modules can be accessed and executed by the processing unit 5410 to perform the functions of the various embodiments described herein.

[0630] In an example embodiment of performing video encoding, input device 5450 may receive video data as input 5470 to be encoded. The video data may be processed, for example, by video codec module 5425 to generate an encoded bitstream. The encoded bitstream may be provided as output 5480 via output device 5460.

[0631] In an example embodiment of performing video decoding, input device 5450 may receive an encoded bitstream as input 5470. The encoded bitstream may be processed, for example, by a video codec module 5425 to generate decoded video data. The decoded video data may be provided as output 5480 via output device 5460.

[0632] While this disclosure has been specifically shown and described with reference to preferred embodiments, those skilled in the art will understand that various changes in form and detail may be made without departing from the spirit and scope of this application as defined by the appended claims. These changes are intended to be covered by the scope of this application. Therefore, the foregoing description of embodiments of this application is not intended to be limiting.

Claims

1. A video processing method, comprising: For the conversion between video units and the bitstream of the video, determine whether to apply at least one co-position image constraint or motion vector scaling process to the candidate derivation or encoding / decoding information storage associated with the video unit; as well as The conversion is performed based on the determination.

2. The method according to claim 1, wherein the candidate derivation includes at least one of the following: temporal candidate derivation, inter-frame propagation candidate derivation, chained candidate derivation, cascaded candidate derivation, or automatic relocation candidate derivation, and The codec information storage includes at least one of the following: time-domain codec information storage, inter-frame propagation codec information storage, chained codec information storage, cascaded codec information storage, or automatic relocation codec information storage.

3. The method of claim 1, wherein no co-position image constraint is applied to the candidate derivation or the encoding / decoding information storage.

4. The method of claim 3, wherein for the candidate derivation, the candidate is obtained from a previously encoded / decoded image, wherein the previously encoded / decoded image is not a co-located image of the current image, and the candidate includes at least one of the following: a temporal candidate, an inter-frame propagation candidate, a chained candidate, a cascaded candidate, or an auto-relocation candidate.

5. The method of claim 4, wherein the candidate is not obtained from the storage information of the co-position image.

6. The method of claim 4, wherein the candidate is obtained from the storage information of a reference image of the current image.

7. The method of claim 4, wherein the storage information of available images from the candidate decoded image cache (DPB) is obtained.

8. The method of claim 4, wherein the temporal cache is used to store encoding and decoding information of available images in the DPB.

9. The method of claim 4, wherein for available images in the DPB, the checklist of potential temporal candidates and / or the order of the potential temporal candidates are predetermined.

10. The method of claim 3, wherein, for candidate storage, if the motion vector used to identify a candidate points to a reference image that is not a co-located image of the current image, then the encoding / decoding information of the candidate is obtained and stored, the candidate including at least one of the following: inter-frame propagation candidate, chained candidate, cascaded candidate, or automatic relocation candidate.

11. The method of claim 1, wherein the motion vector scaling process is not applied to the candidate derivation.

12. The method of claim 11, wherein the candidate points to a reference image, the candidate including at least one of the following: a temporal candidate, an inter-frame propagation candidate, a chained candidate, a cascaded candidate, or an auto-relocation candidate, wherein the reference image is not constrained to a co-located image.

13. The method of claim 11, wherein if the candidate points to a reference image that is not a co-located image of the current image, the motion information of the candidate is used without scaling to the co-located image, and the candidate includes at least one of the following: a temporal candidate, an inter-frame propagation candidate, or a chained candidate.

14. The method of claim 13, wherein the motion information includes at least one of a motion vector or a reference index.

15. The method of claim 1, wherein a temporal buffer constraint is applied to the video unit.

16. The method of claim 15, wherein the restriction on the time-domain cache is imposed to identify time-domain candidates based on a condition check of the restricted valid region.

17. The method of claim 16, wherein the time-domain candidate is unavailable if a predetermined position of the time-domain candidate in the time-domain reference picture is determined to be outside the restricted effective region.

18. The method of claim 16, wherein if a predetermined position of the temporal candidate in the temporal reference image is determined to be outside the restricted effective region, then another position within the restricted effective region is used.

19. The method of claim 18, wherein the nearest location within the restricted effective area is searched.

20. The method of claim 16, wherein the restricted valid region relates to the limitation of the time-domain cache, wherein the limitation relates to a codec tree unit (CTU).

21. The method of claim 15, wherein the limitation of the temporal cache is imposed based on a condition check of the restricted valid region to identify or store candidates including at least one of the following: inter-frame propagation candidates, chained candidates, cascaded candidates, or automatic relocation candidates.

22. The method of claim 21, wherein if the location of the candidate in the time-domain reference image is determined to be outside the restricted effective region, the candidate is unavailable.

23. The method of claim 21, wherein if the candidate location in the time-domain reference image is determined to be outside the restricted effective region, then another location within the restricted effective region is used.

24. The method of claim 23, wherein the nearest location within the restricted effective area is used.

25. The method of claim 21, wherein the restricted effective region relates to the limitation of the time-domain cache, wherein the limitation relates to a codec tree unit (CTU).

26. The method according to any one of claims 1 to 25, wherein the time-domain candidate is at least one of the following: Temporal pattern candidates, Temporal motion candidates, or Temporal model candidates.

27. The method of claim 26, wherein the time-domain pattern candidate comprises at least one of the following: Intra-frame mode candidates, Inter-frame mode candidates Palette mode candidates, Block-based differential pulse codec modulation (BDPCM) mode candidates, Screen Content Codec (SCC) mode candidates, CCP model candidate Geometric Partitioning Pattern (GPM) Candidates Candidate spatial geometric partitioning patterns (SGPM), Affine mode candidates, Inter-frame and intra-frame joint prediction (CIIP) mode candidates, or Multiple hypothesis prediction (MHP) model candidates.

28. The method of claim 26, wherein the time-domain motion candidate comprises at least one of the following: Inter-frame Merge Temporal Motion Vector Prediction (TMVP) Candidates Inter-frame bilateral matching (BM) Merge TMVP candidates Inter-frame advanced motion vector prediction (AMVP) TMVP candidate, Affine Merge TMVP candidate BM Merge Affine TMVP Candidate Affine AMVP TMVP candidate, or Sub-block-based temporal motion vector prediction (SbTMVP) motion displacement candidates.

29. The method of claim 26, wherein the time-domain model candidate comprises at least one of the following: Intra-frame cross-component prediction (CCP) model candidates, or Inter-frame CCP model candidates.

30. The method according to any one of claims 1 to 25, wherein the candidate for inter-frame propagation is at least one of the following: Intra-prediction mode candidates for inter-frame propagation. Candidate CCP models for inter-frame propagation, or Candidate SCC modes for inter-frame propagation.

31. The method of claim 30, wherein the inter-frame propagation intra-prediction mode candidate includes at least one of the following: template-based intra-frame mode derivation (TIMD) candidate, SGPM candidate, or GPM candidate, and / or The inter-frame propagation CCP model candidates include inter-frame CCP merge candidates, and / or The inter-frame propagation SCC mode candidates include Overlapping Block Motion Compensation (OBMC) block-level SCC detection candidates.

32. The method according to any one of claims 1 to 25, wherein the candidate is one of the following: the propagation candidate, the cascaded candidate, the chained candidate, or the motion candidate of the automatic relocation, and the candidate includes at least one of the following: motion vector, CCP model, intra-frame mode information, inter-frame mode information, intra-block copy (IBC) mode information, palette (PLT) mode information, SCC mode information, or transform mode information.

33. The method of claim 32, wherein the motion vector is derived based on the sum of motion vectors and / or block vectors recursively traced from a plurality of time-domain blocks.

34. The method of claim 32, wherein the CCP model is derived based on a plurality of temporal blocks identified by tracked motion vectors or block vectors.

35. The method of claim 32, wherein at least one of the following: the intra-frame mode information, the inter-frame mode information, the IBC mode information, the PLT mode information, the SCC mode information, or the transform mode information is derived based on a plurality of temporal blocks identified by tracked motion vectors or block vectors.

36. The method according to any one of claims 1 to 35, wherein the method of performing the conversion based on determining whether to apply at least one co-location image constraint or motion vector scaling process to the candidate derivation or codec information storage associated with the video unit is used in one of the following ways: Single tree, Twin trees, Inter-frame stripes, or Intra-frame stripes.

37. The method according to any one of claims 1 to 35, wherein the method of performing the transformation based on determining whether to apply at least one co-location image constraint or motion vector scaling process to the candidate derivation or codec information storage associated with the video unit is used in one of the following ways: Chroma encoding and decoding, Luminosity encoding and decoding, Intra-block encoding and decoding, Inter-frame block encoding / decoding, or Intra-Block Copy (IBC) block encoding and decoding.

38. The method according to any one of claims 1 to 37, wherein the method of performing the conversion based on determining whether to apply at least one co-location image constraint or motion vector scaling process to the candidate derivation or codec information storage associated with the video unit is indicated in one of the following places: sequence level, Image group level, Image quality, strip level, or Film series level.

39. The method according to any one of claims 1 to 37, wherein the method of performing the conversion based on determining whether to apply at least one co-location image constraint or motion vector scaling process to the candidate derivation or codec information storage associated with the video unit is indicated in one of the following: Sequence header, Image header, Sequence Parameter Set (SPS) Video Parameter Set (VPS) Dependency Parameter Set (DPS) Decoding Capability Information (DCI) Image Parameter Set (PPS) Adaptive Parameter Set (APS) strip head, or The beginning of the film.

40. The method of any one of claims 1 to 37, wherein the method of performing the conversion based on determining whether to apply at least one co-location image constraint or motion vector scaling process to candidate derivation or codec information storage associated with the video unit is indicated in one of the following: Predicted blocks (PB). Transform block (TB) Code block (CB) Prediction Unit (PU) Transformer Unit (TU) Codec Unit (CU) Virtual Pipeline Data Unit (VPDU). Code-decode tree unit (CTU) CTU line, strip, piece, Sub-images, or A region containing more than one sample point or pixel.

41. The method of any one of claims 1 to 37, wherein the method of performing the conversion based on determining whether to apply at least one co-position image constraint or motion vector scaling process to the candidate derivation or codec information storage associated with the video unit depends on the codec information.

42. The method of claim 41, wherein the encoded / decoded information comprises at least one of the following: Block size, Color format, Single tree segmentation, Two-tree partitioning, Color components, Strip type, or Image type.

43. The method according to any one of claims 1 to 42, wherein the conversion comprises encoding the video unit into the bitstream.

44. The method according to any one of claims 1 to 42, wherein the conversion comprises decoding the video unit from the bitstream.

45. An apparatus for video processing, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 44.

46. ​​A non-transitory computer-readable storage medium storing instructions that cause a processor to execute the method according to any one of claims 1 to 44.

47. A non-transitory computer-readable recording medium storing a bitstream of video generated by a method performed by a video processing apparatus, wherein the method includes: Determine whether to apply at least one isotopic image constraint or motion vector scaling process to the candidate derivation or encoding / decoding information storage associated with the video unit of the video; as well as The bit stream is generated based on the determination.

48. A method for storing a bitstream of video, comprising: Determine whether to apply at least one isotopic image constraint or motion vector scaling process to the candidate derivation or encoding / decoding information storage associated with the video unit of the video; The bit stream is generated based on the determination; as well as The bitstream is stored in a non-transitory computer-readable recording medium.

Images