Interaction between in-loop reshaping and intercoding tools

Abstract

To provide a video processing method related to a correlation between an in-loop reformation and an inter-coding tool.SOLUTION: A method comprises the steps of: executing a motion information elaboration process on the basis of a sample in a first region or a second region for conversion between a current video block of a video and a coded expression of the video; and executing the conversion on the basis of a result of the motion information elaboration process. In the conversion, the sample is acquired in the current video block from a first prediction block in the first region by using motion information which is not elaborated, and at least a second prediction block is generated in the second region by using the elaborated motion information to be used for determining a reconstruction block. A reconstructed sample of the current video block is generated on the basis of at least the second prediction block.SELECTED DRAWING: Figure 31A

Description

[Technical field]

[0001] This patent document relates to video processing techniques, devices and systems. [Background technology]

[0002] Despite advances in video compression, digital video still accounts for the largest bandwidth usage on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth demands for digital video usage are expected to continue to grow. Summary of the Invention

[0003] Devices, systems, and methods are described that relate to digital video processing, and in particular to in-loop reshaping ILR for video processing. The methods described may be applied to video processors, including both existing video processing standards (e.g., High Efficiency Video Coding (HEVC)) and future video processing standards or video codecs.

[0004] In one exemplary aspect, the disclosed technology may be used to provide a method of video processing, the method including: performing a motion information refinement process based on samples in a first domain or a second domain for conversion between a current video block of a video and a coded representation of the video; and performing a conversion based on a result of the motion information refinement process, during the conversion, samples are obtained for the current video block from a first predictive block in the first domain using unrefined motion information, at least a second predictive block is generated in the second domain using refined motion information used to determine a reconstructed block, and the reconstructed samples of the current video block are generated based on at least the second predictive block.

[0005] In another representative aspect, the disclosed techniques may be used to provide a method of video processing, the method including performing a transformation between a current video block of a video and a coded representation of the video, during the transformation, the current video block is reconstructed based on a first region and a second region and / or a chroma residual is scaled depending on the luma, and a coding tool is applied during the transformation using parameters derived based at least on a first set of samples in a video region of the video and a second set of samples in a reference picture of the current video block, and the first region of samples and the second region of samples are aligned.

[0006] In another representative aspect, the disclosed techniques may be used to provide a method of video processing, the method including: determining, for a current video block of a current video region of a video, parameters for a coding mode of the current video block based on one or more parameters for a coding mode of a previous video region; and performing coding on the current video block to generate a coded representation of the video based on the determination, the parameters for the coding mode being included in a parameter set in the coded representation of the video, the performing coding including transforming a representation of the current video block in a first domain to a representation of the current video block in a second domain, and during performing the coding with the coding mode, the current video block is constructed based on the first domain and the second domain, and / or a chroma residual is scaled depending on the luma.

[0007] In another representative aspect, the disclosed techniques may be used to provide a method of video processing, the method including receiving a coded representation of a video including a parameter set including parameter information for a coding mode based on one or more parameters for a coding mode of a previous video region in which the current video block is constructed based on a first region and a second region, and / or a chroma residual is scaled in a luma dependent manner, and performing decoding of the coded representation by using the parameter information to generate from the coded representation a current video block of a current video region of the video, the parameter information for the coding mode being based on one or more parameters for a coding mode of a previous video region in which the current video block is constructed based on a first region and a second region, and / or a chroma residual is scaled in a luma dependent manner.

[0008] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes performing a transformation between a current video block of a video and a coded representation of the video, the transformation including applying a filtering operation to a predictive block in a first domain or in a second domain different from the first domain.

[0009] In another representative aspect, the disclosed techniques may be used to provide a method of video processing, comprising performing a transformation between a current video block of a video and a coded representation of the video, during which a final reconstructed block is determined for the current video block and a temporary reconstructed block is generated using a prediction method and represented in a second domain.

[0010] In another representative aspect, the disclosed techniques may be used to provide a method of video processing, the method including performing a transformation between a current video block of a video region of a video and a coded representation of the video, the transformation using a coding mode in which the current video block is constructed based on a first region and a second region and / or a chroma residual is scaled dependent on luma, and a parameter set in the coded representation has parameter information for the coding mode.

[0011] In another representative aspect, the disclosed techniques may be used to provide a method of video processing, the method including performing a transformation between a current video block of a video, the current video block being a chroma block, and a coded representation of the video, where during the transformation the current video block is constructed based on a first region and a second region, the transformation further including applying a forward reshaping process and / or an inverse reshaping process to one or more chroma components of the current video block.

[0012] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes performing a transformation between a current video chroma block of a video and a coded representation of the video, the performing the transformation including determining, based on a rule, whether luma-dependent chroma residue scaling (LCRS) is enabled or disabled, and reconstructing the current video chroma block based on the determination.

[0013] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes determining whether to disable use of a coding mode for conversion between a current video block of a video and a coded representation of the video based on one or more coefficient values ​​of the current video block, and performing the conversion based on the determination, wherein during conversion using the coding mode, the current video block is constructed based on a first region and a second region, and / or a chroma residual is scaled depending on the luma.

[0014] In another representative aspect, the disclosed techniques may be used to provide a method of video processing, for conversion between a current video block of video over a virtual pipeline data unit (VPDU) and a coded representation of the video, comprising: dividing a current video block into regions and performing the conversion by applying a coding mode to each region separately, wherein during the conversion by applying the coding mode, the current video block is constructed based on a first region and a second region and / or a chroma residual is scaled depending on the luma.

[0015] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes determining whether to disable use of a coding mode for conversion between a current video block of a video and a coded representation of the video based on a size or color format of the current video block, and performing the conversion based on the determination, wherein during the conversion using the coding mode, the current video block is constructed based on a first region and a second region and / or a chroma residual is scaled depending on the luma.

[0016] In another representative aspect, the disclosed techniques may be used to provide a method of video processing, the method including performing a transformation between a current video block of a video and a coded representation of the video, the transformation using a coding mode in which the current video block is constructed based on a first region and a second region and / or a chroma residual is scaled dependent on luma, and at least one syntax element in the coded representation provides an indication of the utilization of the coding mode and an indication of a reshaper model.

[0017] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes determining that a coding mode is disabled for conversion between a current video block of a video and a coded representation of the video, and conditionally skipping forward reshaping and / or backward reshaping based on the determination, in a coding mode in which the current video block is constructed based on a first region and a second region and / or a chroma residual is scaled depending on luma.

[0018] In another representative aspect, the disclosed techniques may be used to provide a method of video processing, the method including performing a transformation between a current video block of a video region of a video and a coded representation of the video, the transformation using a coding mode in which the current video block is constructed based on a first region and a second region and / or in which a chroma residual is scaled dependent on a luma, and a number of forward reshapes and / or a number of backward reshapes are applied to the video region in the reshaping mode.

[0019] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes determining a coding mode to be enabled for conversion between a current video block of video and a coded representation of the video, and performing the conversion using a palette mode in which at least a palette of representative sample values ​​is used for the current video block, where the coding mode is constructed based on samples in a first domain and a second domain, and / or where chroma residual is scaled in a luma dependent manner.

[0020] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes, for conversion between a current video block of a video and a coded representation of the video, determining that a current video block is coded in a palette mode in which at least a palette of representative sample values ​​is used to code the current video block, and, upon the determination, performing the conversion by disabling a coding mode, where if a coding mode is applied to the video block, the video block is constructed based on a luma-dependent scaled chroma residual.

[0021] In another representative aspect, the disclosed techniques may be used to provide a method of video processing, the method including: performing a conversion between a first video block of a video and a coded representation of the video, the conversion using a first coding mode and a palette coding mode in which at least a palette of representative sample values ​​is used to code a current video block; and performing a conversion between a second video block of the video and the coded representation of the video that is coded without using a palette coding mode, the conversion of the second video block using the first coding mode, where if the first coding mode is applied to a video block, the video block is constructed based on a first region and a second region, and / or the chroma residual is scaled depending on the luma, and the first coding mode is applied to the first video block and the second video block in a different manner.

[0022] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes determining a coding mode to be enabled for conversion between a current video block of a video and a coded representation of the video, and performing the conversion using an intra block copy mode to generate a predictive block using at least a block vector that points to a picture that includes the current video block, where the current video block is constructed based on samples in a first domain and a second domain, and / or where a chroma residual is scaled depending on the luma.

[0023] In another representative aspect, the disclosed techniques may be used to provide a method of video processing, the method including: determining that a current video block is coded in an intra block copy (IBC) mode for converting between a current video block of a video and a coded representation of the video, generating a predictive block using at least a block vector pointing to a video frame including the current video block for coding the current video block; and performing the conversion by disabling a coding mode upon the determination, wherein if a coding mode is applied to the video block, the video block is constructed based on a first region and a second region and / or a chroma residual is scaled depending on the luma.

[0024] In another representative aspect, the disclosed techniques may be used to provide a method of video processing, the method including: performing a conversion between a first video block of a video and a coded representation of the video, the conversion using an intra block copy mode that generates a predictive block using at least a block vector that points to a video frame that includes a current video block and a first coding mode; and performing a conversion between a second video block of the video coded without using the intra block copy mode and the coded representation of the video, the conversion of the second video block using the first coding mode, where if the first coding mode is applied to the video block, the video block is constructed based on a first region and a second region, and / or the chroma residual is scaled depending on the luma, and the first coding mode is applied to the first video block and the second video block in a different manner.

[0025] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes determining a coding mode to be enabled for conversion between a current video block of a video and a coded representation of the video, and performing the conversion using a block-based delta pulse code modulation (BDPCM) mode, in which the current video block is constructed based on samples in a first domain and a second domain, and / or a chroma residual is scaled depending on the luma.

[0026] In another representative aspect, the disclosed techniques may be used to provide a method of video processing, the method including, for converting between a current video block of a video and a coded representation of the video, determining that a current video block is coded using a block-based delta pulse code modulation (BDPCM) mode, and performing the conversion by disabling a coding mode upon the determination, where if the coding mode is applied to the video block, the video block is constructed based on a first region and a second region and / or a chroma residual is scaled depending on the luma.

[0027] In another representative aspect, the disclosed techniques may be used to provide a method of video processing, the method including: performing a conversion between a first video block of a video and a coded representation of the video, the conversion of the first video block using a first coding mode and a block-based delta pulse code modulation (BDPCM) mode; and performing a conversion between a second video block of the video and the coded representation of the video, the second video block being coded without using the BDPCM mode, the conversion of the second video block using the first coding mode, and when the first coding mode is applied to the video block, the video block is constructed based on a first region and a second region, and / or the chroma residual is scaled depending on the luma, and the first coding mode is applied to the first video block and the second video block in different manners.

[0028] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes determining a coding mode to be enabled for transforming between a current video block of a video and a coded representation of the video, and performing the transform using a transform skip mode in which a transform on a prediction residual is skipped in coding the current video block, where the current video block is constructed based on samples in a first domain and a second domain, and / or the chroma residual is scaled depending on the luma.

[0029] In another representative aspect, the disclosed techniques may be used to provide a method of video processing, the method including: determining, for conversion between a current video block of a video and a coded representation of the video, that a current video block is coded in a transform skip mode in which a transform on a prediction residual is skipped in coding the current video block; and, upon the determination, performing the transform by disabling a coding mode, in which if a coding mode is applied to the video block, the video block is constructed based on a first region and a second region and / or a chroma residual is scaled depending on the luma.

[0030] In another representative aspect, the disclosed techniques may be used to provide a method of video processing, the method including: performing a conversion between a first video block of a video and a coded representation of the video, the conversion of the first video block using a first coding mode and a transform skip mode in which a transform on a prediction residual is skipped in coding a current video block; and performing a conversion between a second video block of the video and the coded representation of the video, the second video block being coded without using the transform skip mode, the transform of the second video block using the first coding mode, and when the first coding mode is applied to a video block, the video block is constructed based on a first region and a second region, and / or the chroma residual is scaled depending on the luma, and the first coding mode is applied to the first video block and the second video block in different manners.

[0031] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes determining a coding mode to be enabled for converting between a current video block of a video and a coded representation of the video, and performing the conversion using an intrapulse code modulation mode in which the current video block is coded without applying a transform and transform domain quantization, and in which the current video block is constructed based on samples in a first domain and a second domain, and / or a chroma residual is scaled depending on the luma.

[0032] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes, for conversion between a current video block of a video and a coded representation of the video, determining that a current video block is coded in an intrapulse code modulation mode, in which the current video block is coded without applying a transform and transform domain quantization, and, upon the determination, performing the conversion by disabling a coding mode, in which if a coding mode is applied to the video block, the video block is constructed based on a first region and a second region and / or a chroma residual is scaled depending on the luma.

[0033] In another representative aspect, the disclosed techniques may be used to provide a method of video processing, the method including: performing a conversion between a first video block of a video and a coded representation of the video, the conversion of the first video block using a first coding mode and an intrapulse code modulation mode in which the current video block is coded without applying a transform and transform domain quantization; and performing a conversion between a second video block of the video and the coded representation of the video, the second video block being coded without using an intrapulse code modulation mode, the conversion of the second video block using the first coding mode, and when the first coding mode is applied to a video block, the video block is constructed based on a first region and a second region, and / or the chroma residual is scaled depending on the luma, and the first coding mode is applied to the first video block and the second video block in different manners.

[0034] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes determining a coding mode to be enabled for conversion between a current video block of a video and a coded representation of the video, and performing the conversion using a modified transquant-bypass mode in which the current video block is losslessly coded without transform and quantization, and in which the current video block is constructed based on samples in a first domain and a second domain, and / or chroma residuals are scaled depending on luma.

[0035] In another representative aspect, the disclosed techniques may be used to provide a method of video processing that includes determining, for conversion between a current video block of a video and a coded representation of the video, that a current video block is coded in a transquant-bypass mode, in which the current video block is losslessly coded without application of transform and quantization, and, upon the determination, performing the conversion by disabling a coding mode, in which if a coding mode is applied to the video block, the video block is constructed based on a first region and a second region and / or a chroma residual is scaled depending on the luma.

[0036] In another representative aspect, the disclosed techniques may be used to provide a method of video processing, the method including: performing a conversion between a first video block of a video and a coded representation of the video, the conversion of the first video block using a first coding mode and a transquant-bypass mode in which a current video block is losslessly coded without transform and quantization; and performing a conversion between a second video block of the video and the coded representation of the video, the second video block being coded without using the transquant-bypass mode, the conversion of the second video block using the first coding mode, and when the first coding mode is applied to a video block, the video block is constructed based on a first region and a second region, and / or the chroma residual is scaled depending on the luma, and the first coding mode is applied to the first video block and the second video block in different manners.

[0037] In another representative aspect, the disclosed techniques may be used to provide a method of video processing, the method including performing a transformation between a current video block of a video and a coded representation of the video, the transformation using a coding mode in which the current video block is constructed based on a first region and a second region and / or a chroma residual is scaled dependent on luma, and the information used for the coding mode is signaled in a parameter set different from a sequence parameter set (SPS), a video parameter set (VPS), a picture parameter set (PPS), or an adaptation parameter set (APS) used to carry adaptive loop filtering (ALF) parameters.

[0038] In another representative aspect, the disclosed techniques may be used to provide a method of video processing, the method including performing a transformation between a current video block of a video and a coded representation of the video, the transformation using a coding mode in which the current video block is constructed based on a first region and a second region and / or a chroma residual is scaled dependent on luma, the information used for the coding mode is signaled in an adaptation parameter set (APS) together with adaptive loop filtering (ALF) information, and the information used for the coding mode and the ALF information are included in one NAL unit.

[0039] In another representative aspect, the disclosed techniques may be used to provide a method of video processing, the method including performing a transformation between a current video block of a video region of a video and a coded representation of the video, the transformation using a coding mode in which the current video block is constructed based on a first region and a second region and / or a chroma residual is scaled dependent on luma, and information used for the coding mode is signaled with an adaptation parameter set (APS) of a first type that is different from an APS of a second type used to signal adaptive loop filtering (ALF) information.

[0040] In another representative aspect, the disclosed techniques may be used to provide a method of video processing, the method including performing a conversion between a current video block of a video region of a video and a coded representation of the video, the conversion using a coding mode in which the current video block is constructed based on a first region and a second region and / or in which a chroma residual is scaled dependent on a luma, the video region is not allowed to reference a parameter set or an adaptive parameter set that is signaled prior to a particular type of data structure used to process the video, the particular type of data structure being signaled prior to the video region.

[0041] In another representative aspect, the disclosed techniques may be used to provide a method of video processing, the method including performing a transformation between a current video block of a video and a coded representation of the video, the transformation using a coding mode in which the current video block is constructed based on a first region and a second region and / or in which chroma residual is scaled dependent on luma, and wherein parameter set syntax elements including parameters used to process the video have predefined values ​​in a conformance bitstream.

[0042] In another exemplary embodiment, the above method is embodied in a processor executable code and stored on a computer readable program medium.

[0043] In another exemplary aspect, a device configured or operable to perform the above method is disclosed. The device may include a processor programmed to implement the method.

[0044] In another exemplary aspect, a video decoder device may implement the methods described herein.

[0045] These and other aspects and features of the disclosed technology are explained in further detail in the drawings, specification and claims. [Brief description of the drawings]

[0046] [Figure 1] Here is an example of constructing a merge candidate list: [Diagram 2] 4 shows examples of spatial candidate locations. [Diagram 3] 1 shows an example of candidate pairs that are subject to redundancy checking for spatial merge candidates. [Figure 4A] 13 shows an example of a position of a second prediction unit (PU) based on the size and shape of a current block. [Figure 4B] 13 shows an example of a position of a second prediction unit (PU) based on the size and shape of a current block. [Diagram 5] 13 illustrates an example of motion vector scaling for temporal merge candidates. [Figure 6] 13 shows examples of candidate positions for temporal merge candidates. [Figure 7] 13 illustrates an example of generating composite bi-predictive merge candidates. [Figure 8] 13 shows an example of constructing motion vector prediction candidates. [Figure 9] 4 illustrates an example of motion vector scaling for spatial motion vector candidates. [Figure 10]1 shows an example of motion prediction using adaptive temporal motion vector prediction (ATMVP) for a coding unit (CU). [Figure 11] 1 illustrates an example of a coding unit (CU) with sub-blocks and neighboring blocks used by the spatio-temporal motion vector prediction (STMVP) algorithm. [Figure 12] 1 shows an example of adjacent samples for deriving illumination compensation (IC) parameters. [Figure 13A] Here is an example of a simplified four-parameter affine model. [Figure 13B] Here is an example of a simplified six-parameter affine model. [Figure 14] 1 shows an example of an affine motion vector (MVF) for each subblock. [Figure 15A] Here is an example of a four-parameter affine model. [Figure 15B] Here is an example of a six-parameter affine model. [Figure 16] 13 shows an example of motion vector prediction for AF_INTER for genetic affine candidates. [Figure 17] 13 shows an example of motion vector prediction for AF_INTER for constructed affine candidates. [Figure 18A] An example of a candidate block for AF_MERGE mode is shown below. [Figure 18B] 13 shows an example of CPMV predictor derivation for AF_MERGE mode. [Figure 19] 13 shows examples of candidate positions for the affine merge mode. [Figure 20] An example of the UMVE search process is given below. [Figure 21] An example of the UMVE search process is given below. [Figure 22] 1 illustrates an example of decoder-side motion vector refinement (DMVR) based on bilateral template matching. [Figure 23] 1 shows an example of a flowchart of a decoding flow with reshaping. [Figure 24]1 shows an example of adjacent samples used in a bilateral filter. [Diagram 25] Here is an example of a window covering two samples used in the weight calculation. [Figure 26] 1 shows an example of a scan pattern. [Figure 27] 4 shows an example of an inter mode decoding process. [Figure 28] 4 illustrates another example of an inter mode decoding process. [Figure 29] 1 shows an example of an inter-mode decoding process with post-reconstruction filters. [Diagram 30] 4 illustrates another example of an inter-mode decoding process with post-reconstruction filters. [Figure 31A] 1 shows a flowchart of an exemplary video processing method. [Figure 31B] 1 shows a flowchart of an exemplary video processing method. [Figure 32A] 1 shows a flowchart of an exemplary video processing method. [Figure 32B] 1 shows a flowchart of an exemplary video processing method. [Figure 32C] 1 shows a flowchart of an exemplary video processing method. [Fig. 32D] 1 shows a flowchart of an exemplary video processing method. [Diagram 33] 1 shows a flowchart of an exemplary video processing method. [Figure 34A] 1 shows a flowchart of an exemplary video processing method. [Figure 34B] 1 shows a flowchart of an exemplary video processing method. [Figure 35A] 1 shows a flowchart of an exemplary video processing method. [Figure 35B] 1 shows a flowchart of an exemplary video processing method. [Figure 35C] 1 shows a flowchart of an exemplary video processing method. [Figure 35D] 1 shows a flowchart of an exemplary video processing method. [Figure 35E]1 shows a flowchart of an exemplary video processing method. [Fig. 35F] 1 shows a flowchart of an exemplary video processing method. [Figure 36A] 1 shows a flowchart of an exemplary video processing method. [Figure 36B] 1 shows a flowchart of an exemplary video processing method. [Figure 36C] 1 shows a flowchart of an exemplary video processing method. [Figure 37A] 1 shows a flowchart of an exemplary video processing method. [Figure 37B] 1 shows a flowchart of an exemplary video processing method. [Figure 37C] 1 shows a flowchart of an exemplary video processing method. [Figure 38A] 1 shows a flowchart of an exemplary video processing method. [Figure 38B] 1 shows a flowchart of an exemplary video processing method. [Figure 38C] 1 shows a flowchart of an exemplary video processing method. [Figure 38D] 1 shows a flowchart of an exemplary video processing method. [Figure 38E] 1 shows a flowchart of an exemplary video processing method. [Figure 38F] 1 shows a flowchart of an exemplary video processing method. [Figure 38G] 1 shows a flowchart of an exemplary video processing method. [Fig. 38H] 1 shows a flowchart of an exemplary video processing method. [Fig. 38I] 1 shows a flowchart of an exemplary video processing method. [Figure 38J] 1 shows a flowchart of an exemplary video processing method. [Figure 38K] 1 shows a flowchart of an exemplary video processing method. [Figure 38L] 1 shows a flowchart of an exemplary video processing method. [Figure 39A]1 shows a flowchart of an exemplary video processing method. [Figure 39B] 1 shows a flowchart of an exemplary video processing method. [Figure 39C] 1 shows a flowchart of an exemplary video processing method. [Figure 39D] 1 shows a flowchart of an exemplary video processing method. [Figure 39E] 1 shows a flowchart of an exemplary video processing method. [Figure 40A] 1 illustrates an example hardware platform for implementing the visual media decoding or visual media encoding techniques described in this disclosure. [Figure 40B] 1 illustrates an example hardware platform for implementing the visual media decoding or visual media encoding techniques described in this disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0047] Due to the growing demand for higher resolution video, video processing methods and techniques are ubiquitous in current technology. Video codecs usually include electronic circuits or software that compress or decompress digital video and continue to be improved to provide higher coding efficiency. Video codecs convert uncompressed video into compressed formats or vice versa. There is a complex relationship between video quality, the amount of data (determined by bit rate) used to represent the video, the complexity of the encoding and decoding algorithms, the sensitivity to data loss and errors, the ease of editing, random access, and end-to-end delay (latency). Compressed formats usually follow standard video compression specifications, such as the High Efficiency Video Coding (HEVC) standard (also known as H.265 or MPEG-H Part 2), the Versatile Video Coding (VVC) standard to be completed, or other current and / or future video coding standards.

[0048] Embodiments of the disclosed technology may be applied to existing video coding standards (e.g., HEVC, H.265) and future standards to improve compression performance. Section headings are used herein to improve readability of the description and in no way limit the disclosure or embodiments (and / or implementations) to the respective section.

[0049] [1. Example of inter prediction in HEVC / H.265] Video coding standards have improved greatly over the years and now provide high coding efficiency and support higher resolutions. Recent standards such as HEVC and H.265 are based on hybrid video coding structures in which temporal prediction and transform coding are utilized.

[0050] [1.1 Prediction mode example] Each inter-predicted prediction unit (PU) has motion parameters for one or two reference picture lists. In some embodiments, the motion parameters include a motion vector and a reference picture index. In other embodiments, the use of one of the two reference picture lists may be signaled using inter_pred_idc. In yet other embodiments, the motion vector may be explicitly coded as a differential to the predictor.

[0051] When a CU is coded in skip mode, one PU is associated with the CU and there are no significant residual coefficients, coded motion vector differentials, or reference picture indexes. A merge mode is specified, whereby the motion parameters of the current PU are obtained from neighboring PUs, including spatial and temporal candidates. The merge mode is applicable not only to skip mode, but also to any inter-predicted PUE. An alternative to the merge mode is explicit transmission of motion parameters, where the motion vectors (more precisely, the motion vector differences (MVD) compared to the motion vector predictor), the corresponding reference picture index of each reference picture list, and the reference picture list utilization are explicitly signaled for each PU. Such a mode is referred to as advanced motion vector prediction (AMVP) in this specification.

[0052] If the signaling indicates that one of the two reference picture lists should be used, the PU is generated from one block of samples. This is called "uni-prediction". Uni-prediction is available for both P slices and B slices.

[0053] If the signaling indicates that both reference picture lists should be used, the PU is generated from two blocks of samples. This is called "bi-prediction." Bi-prediction is only available for B slices.

[0054] Reference Picture List In HEVC, the term inter prediction is used to denote a prediction derived from data elements (e.g., sample values ​​or motion vectors) of a reference picture other than the currently decoded picture. Similarly, in H.264 / AVC, a picture may be predicted from multiple reference pictures. The reference pictures used for inter prediction are organized into one or more reference picture lists. A reference index identifies which reference picture in the list should be used to generate the prediction signal.

[0055] A single reference picture list List0 is used for P slices, and two reference lists List0 and List1 are used for B slices. It should be noted that the reference pictures included in List0 / 1 may be from past and future pictures with respect to capture / display order.

[0056] 1.1.1 Implementation of Merge Mode Candidate Configuration When a PU is predicted using merge mode, an index pointing to an entry in the merge candidates list is parsed from the bitstream and used to retrieve the motion information. The construction of this list can be briefly described according to the following sequence of steps: Step 1: Derive initial candidates Step 1.1: Derive spatial candidates Step 1.2: Redundancy check for spatial candidates Step 1.3: Deriving time candidates Step 2: Insert additional candidates Step 2.1: Generating bi-prediction candidates Step 2.2: Inserting zero motion candidates

[0057] FIG. 1 shows an example of constructing a merge candidate list based on the sequence of steps described above. For the derivation of the initial merge candidate, up to four merge candidates are selected from among the candidates at five different positions. For the derivation of the temporal merge candidate, up to one merge candidate is selected from among the two candidates. Since a fixed number of candidates per PU are considered at the decoder, additional candidates are generated if the number of candidates does not reach the maximum number of merge candidates (MaxNumMergeCand) signaled in the slice header. Since the number of candidates is fixed, the index of the best merge candidate is coded using truncated unary binarization (TU). When the size of a CU is equal to 8, all PUs of the current CU share a single merge candidate list, which is identical to the merge candidate list of the 2N×2N prediction unit.

[0058] [1.1.2 Construction of spatial merge candidates] In the derivation of spatial merge candidates, up to four merge candidates are selected from the candidates at the positions shown in Fig. 2. The order of derivation is A1, B1, B0, A0, and B2. Position B2 is considered only if any PU at positions A1, B1, B0, A0 is unavailable (because it belongs to another slice or tile) or is intra-coded. After the candidate at position A1 is added, the addition of the remaining candidates is subject to a redundancy check. This ensures that candidates with the same motion information are removed from the list so that the coding efficiency is improved.

[0059] In order to reduce computational complexity, not all possible candidate pairs are considered in the above redundancy check. Instead, pairs connected by arrows in FIG. 3 are considered, and a candidate is added to the list only if the corresponding candidate used for the redundancy check does not have the same motion information. Another source of the exact same motion information is a "second PU" associated with a partition different from 2N×2N. As an example, FIGS. 4A and 4B represent the second PU for the cases of N×2N and 2N×N, respectively. If the current PU is partitioned as N×2N, the candidate at position A1 is not taken into account for list construction. In some embodiments, adding this candidate may result in two prediction units with the same motion information, which is redundant to have only one PU in the coding unit. Similarly, position B1 is not taken into account if the current PU is partitioned as 2N×N.

[0060] [1.1.3 Configuration of Temporal Merge Candidates] In this step, only one candidate is added to the list. In particular, in the derivation of this temporal merge candidate, a scaled motion vector is derived based on the co-located PU that belongs to the picture with the smallest POC difference with the current picture in a given reference picture list. The reference picture list used for the derivation of the co-located PU is explicitly signaled in the slice header.

[0061] FIG. 5 shows an example of derivation of a scaled motion vector of a temporal merge candidate (as a dashed line), where the scaled motion vector is scaled from the motion vector of the co-located PU using POC distances tb and td. tb is defined as the POC distance between the reference picture of the current picture and the current picture, and td is defined as the POC difference between the reference picture of the co-located picture and the co-located picture. The reference picture index of the temporal merge candidate is set equal to zero. For a B slice, two motion vectors (one for reference picture list 0 and the other for reference picture list 1) are obtained and combined to generate a bi-predictive merge candidate.

[0062] For the same position PU(Y) belonging to the reference frame, the position of the temporal candidate is selected between candidates C0 and C1, as shown in Fig. 6. If the PU at position C0 is not available, is intra-coded, or is outside the current CTU, position C1 is used. Otherwise, position C0 is used in the derivation of the temporal merge candidate.

[0063] [1.1.4 Constructing additional types of merge candidates] In addition to spatial and temporal merge candidates, there are two further types of merge candidates: composite bi-predictive merge candidates and zero merge candidates. The composite bi-predictive merge candidates are generated by utilizing spatial and temporal merge candidates. The composite bi-predictive merge candidates are only used for B slices. The composite bi-predictive merge candidates are generated by combining the first reference picture list motion parameters of the first candidate with the other second reference picture list motion parameters. If these two tuples result in different motion hypotheses, they will form a new bi-predictive candidate.

[0064] 7 shows an example of this process, where two candidates in the original list (710 on the left) with mvL0 and refIdxL0 or mvL1 and refIdx1 are used to generate a composite bi-predictive merge candidate that is added to the final list (720 on the right). There are many rules regarding the possible combinations for generating these additional merge candidates.

[0065] Zero motion candidates are inserted to fill the remaining entries in the merge candidate list to reach the MaxNumMergeCand capacity. These candidates have zero spatial displacement and reference picture index that starts at zero and increases each time a new zero motion candidate is added to the list. The number of reference frames used by these candidates is 1 and 2 for unidirectional and bidirectional prediction, respectively. In some embodiments, no redundancy check is performed on these candidates.

[0066] [1.2 Advanced Motion Vector Prediction (AMVP) Implementation] AMVP exploits the spatio-temporal correlation of motion vectors with neighboring PUs, which is used for the explicit transmission of motion parameters. It constructs a motion vector candidate list by first checking the availability of the temporally neighboring PU positions on the left and above, and then removing redundant candidates and adding zero vectors to make the candidate list a certain length. The encoder can then select the best predictor from the candidate list and transmit the corresponding index indicating the selected candidate. Similarly, with merge index signaling, the index of the best motion vector candidate is coded using a truncated unary. The maximum value coded in this case is 2 (see Figure 8). In the following sections, details on the derivation process of motion vector prediction candidates are provided.

[0067] [1.2.1 Example of deriving AMVP candidates] FIG. 8 summarizes the motion vector prediction candidate derivation process, which may be performed for each reference picture list with refidx as input.

[0068] In motion vector prediction, two types of motion vector candidates are considered: spatial motion vector candidates and temporal motion vector candidates. For spatial motion vector candidate derivation, two motion vector candidates are finally derived based on the motion vectors of each PU at five different positions as previously shown in FIG.

[0069] For temporal motion vector candidate derivation, one motion vector candidate is selected from two candidates derived based on two different co-location positions. After the first list of spatio-temporal candidates is generated, the duplicated motion vector candidates in the list are removed. If the number of potential candidates is greater than two, the motion vector candidates whose reference picture index in the associated reference picture list is greater than one are removed from the list. If the number of spatio-temporal motion vector candidates is less than two, an additional zero motion vector candidate is added to the list.

[0070] [1.2.2 Construction of spatial motion vector candidates] In the derivation of spatial motion vector candidates, up to two candidates are considered from five potential candidates derived from PUs in positions as shown in FIG. 2 above (these positions are the same as those of motion merge). The derivation order of the left side of the current PU is defined as A0, A1, scaled A0, scaled A1. The derivation order of the top side of the current PU is defined as B0, B1, B2, scaled B0, scaled B1, scaled B2. For each side, there are therefore four cases that can be used as motion vector candidates, two cases are not required to use spatial scaling, and the other two cases spatial scaling is used. The four different cases are briefly described as follows: No spatial scaling (1) The same reference picture list and the same reference picture index (same POC) (2) Different reference picture lists, but the same reference pictures (same POC) Spatial Scaling (3) Same reference picture list, but different reference pictures (different POC) (4) Different reference picture lists and different reference pictures (different POCs)

[0071] The case without spatial scaling is checked first, followed by spatial scaling. Spatial scaling is considered when the POC is different between the reference pictures of neighboring PUs and that of the current PU regardless of the reference picture list. If all PUs of the left candidate are unavailable or intra-coded, scaling of the above motion vector is enabled to aid in the parallel derivation of left and above MV candidates. Otherwise, spatial scaling is not allowed for the above motion vector.

[0072] As shown in Figure 9, for the spatial scaling case, the motion vectors of neighboring PUs are scaled in the same manner as for the temporal scaling case. The main difference is that the reference picture list and index of the current PU are given as input, and the actual scaling process is the same as that of temporal scaling.

[0073] [1.2.3 Construction of temporal motion vector candidates] Apart from the reference picture index derivation, all the processes for the derivation of temporal merge candidates are the same as for the derivation of spatial motion vector candidates (shown in the example of Figure 6). In some embodiments, the reference picture index is signaled to the decoder.

[0074] [2. Example of new inter-prediction method in JEM] In some embodiments, future video coding techniques are explored using a reference software known as the Joint Exploration Model (JEM), in which sub-block based prediction is introduced in several coding tools such as affine prediction, Alternative Temporal Motion Vector Prediction (ATMVP), Spatial-Temporal Motion Vector Prediction (STMVP), bi-directional optical flow (BIO), Frame-Rate Up Conversion (FRUC), Locally Adaptive Motion Vector Resolution (LAMVR), Overlapped Block Motion Compensation (OBMC), Local Illumination Compensation (LIC), and Decoder side Motion Vector Refinement (DMVR).

[0075] [2.1 Example of sub-CU-based motion vector prediction] In JEM with quadtrees plus binary trees (QTBT), each CU can have at most one set of motion parameters for each prediction direction. In some embodiments, two sub-CU level motion vector prediction methods are considered in the encoder by splitting a large CU into sub-CUs and deriving motion information for all sub-CUs of the large CU. The alternative temporal motion vector prediction (ATMVP) method allows each CU to fetch multiple sets of motion information from multiple blocks smaller than the current CU in the co-located reference picture. In the spatial-temporal motion vector prediction (STMVP) method, the motion vector of a sub-CU is derived recursively by using a temporal motion vector predictor and a spatial neighboring motion vector. In some embodiments, the motion compression for the reference frame may be disabled to keep a more accurate motion field for sub-CU motion prediction.

[0076] [2.1.1 ATMVP Example] In the ATMVP method, the Temporal Motion Vector Prediction (TMVP) method is modified by fetching multiple sets of motion information (including motion vectors and reference indexes) from blocks smaller than the current CU.

[0077] 10 shows an example of ATMVP motion prediction process for CU 1000. The ATMVP method predicts motion vectors of sub-CUs 1001 in CU 1000 in two steps. The first step is to identify corresponding blocks 1051 in reference picture 1050 by a time vector. The reference picture 1050 is also called a motion source picture. The second step is to split the current CU 1000 into sub-CUs 1001 and obtain the motion vectors and reference indexes of each sub-CU from the blocks corresponding to each sub-CU.

[0078] In the first step, the reference picture 1050 and the corresponding blocks are determined by the motion information of the spatial neighboring blocks of the current CU 1000. To avoid the repetitive scanning process of the neighboring blocks, the first merge candidate in the merge candidate list of the current CU 1000 is used. The first available motion vector and its associated reference index are set to be a time vector and an index into the motion source picture. In this way, the corresponding blocks can be more accurately identified compared to TMVP. Here, the corresponding block (sometimes called the co-located block) is always in the bottom-right or center position with respect to the current CU.

[0079] In a second step, the corresponding block of the sub-CU 1051 is identified by adding its time vector to the coordinates of the current CU by the time vector of the motion source picture 1050. For each sub-CU, the motion information of its corresponding block (e.g., the minimum motion grid covering the center sample) is used to derive the motion information of that sub-CU. After the motion information of the corresponding N×N block is identified, it is converted to the motion vector and reference index of the current sub-CU, similar to TMVP of HEVC. At this time, motion scaling and other procedures are applied. For example, the decoder checks whether a low-delay condition (e.g., the POC of all reference pictures of the current picture is smaller than the POC of the current picture) is satisfied, and possibly uses the motion vector MVx (e.g., the motion vector corresponding to the reference picture list X) to predict the motion vector MVy (e.g., X is equal to 0 or 1, and Y is equal to 1-X) for each sub-CU.

[0080] [2.1.2 STMVP example] In the STMVP method, the motion vectors of sub-CUs are derived recursively according to the raster scan order. Figure 11 shows one CU with four sub-blocks, together with the neighboring blocks. Consider an 8x8 CU that contains four 4x4 sub-CUs A (1101), B (1102), C (1103) and D (1104). The neighboring 4x4 blocks of the current frame are denoted as a (1111), b (1112), c (1112) and d (1114).

[0081] Motion derivation of sub-CU A begins by identifying its two spatial neighbors. The first neighbor is an N×N block (block c 1113) above sub-CU A (1101). If this block c (1113) is unavailable or intra-coded, then the other N×N block above sub-CU A (1101) is checked (starting from block c 1113, left to right). The second neighbor is a block (block b 1112) to the left of sub-CU A (1101). If block b (1112) is unavailable or intra-coded, then the other block to the left of sub-CU A (1101) is checked (starting from block b 1112, top to bottom). The motion information obtained from the neighboring blocks for each list is scaled to the first reference frame for a given list. Then, the temporal motion vector predictor (TMVP) of sub-block A (1101) is derived by following the same procedure of TMVP derivation defined in HEVC. The motion information of the co-located block at location D (1104) is fetched and scaled accordingly. Finally, after retrieving and scaling the motion information, all available motion vectors are averaged separately for each reference list. The averaged motion vector is assigned as the motion vector of the current sub-CU.

[0082] [2.1.3 Sub-CU motion prediction mode signaling example] In some embodiments, the sub-CU mode can be used as an additional merge candidate, and no additional syntax elements are required to signal the mode. Two additional merge candidates are added to the merge candidates of each CU to represent ATMVP and STMVP modes. In other embodiments, up to seven merge candidates may be used if the sequence parameter set indicates that ATMVP and STMVP are available. The coding logic of the additional merge candidates is the same as that of the merge candidates in HM. This means that for each CU in a P or B slice, more than one RD check may be required for the two additional merge candidates. In some embodiments, e.g., JEM, all bins of the merge index are context coded by CABAC (Context-based Adaptive Binary Arithmetic Coding). In other embodiments, e.g., HEVC, only the first bin is context coded, and the remaining bins are context bypass coded.

[0083] [2.2 Example of LIC in JEM] Local illumination compensation (LIC) is based on a linear model for illumination changes, with a scaling factor a and an offset b, and it is adaptively enabled or disabled for each inter-mode coded coding unit (CU).

[0084] When LIC is applied to a CU, a least square error method is used to derive parameters a and b by using the neighboring samples of the current CU and their corresponding reference samples. More specifically, as shown in Figure 12, the subsampled (2:1 subsampled) neighboring samples of the CU and the corresponding samples in the reference picture (identified by the motion information of the current CU or sub-CU) are used.

[0085] [2.1.1 Deriving the prediction block] The IC parameters are derived and applied separately for each prediction direction. For each prediction direction, a first prediction block is generated by the combined motion information, and then a temporal prediction block is obtained by applying the LIC model. After that, the two temporal prediction blocks are used to derive the final prediction block.

[0086] If a CU is coded in merge mode, the LIC flag is copied from the neighboring block in the same manner as the motion information copy in merge mode. Otherwise, the LIC flag is signaled for the CU to indicate whether LIC is applied or not.

[0087] If LIC is available for a picture, an additional CU-level RD check is required to determine whether LIC applies to the CU. If LIC is available for a CU, the Mean-Removed Sum of Absolute Difference (MR-SAD) and Mean-Removed Sum of Absolute Hadamard-Transformed Difference (MR-SATD) are used for integer-pel and fractional-pel motion estimation, respectively, instead of SAD and SATD.

[0088] To reduce the coding complexity, the following coding scheme is applied in JEM: LIC is disabled for the entire picture if there is no obvious illumination change between the current picture and its reference pictures. To identify this situation, the histograms of the current picture and every reference picture of the current picture are calculated in the encoder. If the histogram difference between the current picture and every reference picture of the current picture is smaller than a given threshold, LIC is disabled for the current picture; otherwise, LIC is enabled for the current picture.

[0089] [2.3 Example of inter prediction method in VVC] There are several new coding tools for improving inter prediction, such as Adaptive Motion Vector Difference (AMVR) for signaling MVD, Affine Prediction Mode, Triangular Prediction Mode (TPM), ATMVP, Generalized Bi-Prediction (GBI), and Bi-Predictive Optical Flow (BIO).

[0090] [2.3.1 Example of coding block structure in VVC] In VVC, a quadtree / binarytree / multipletree (QT / BT / TT) structure is adopted to divide a picture into square or rectangular blocks. In addition to QT / BT / TT, another tree (also called a dual coding tree) is also adopted in VVC for I-frames. According to the another tree, the coding block structure is signaled separately for the luma and chroma components.

[0091] [2.3.2 Example of adaptive motion vector difference analysis] In some embodiments, the motion vector differential (MVD) (between the PU's motion vector and the predicted motion vector) is signaled in units of quarter luma samples when use_integer_my_flag is equal to 0 in the slice header. In JEM, locally adaptive motion vector decomposition (LAMVR) is introduced. In JEM, MVD can be coded in units of quarter luma samples, integer luma samples, or 4 luma samples. MVD decomposition is controlled at the coding unit (CU) level, and an MVD decomposition flag is conditionally signaled for each CU that has at least one non-zero MVD component.

[0092] For a CU that has at least one non-zero MVD component, a first flag is signaled to indicate whether quarter luma sample MV precision is used for the CU; if the first flag (equal to 1) indicates that quarter luma sample MV precision is not used, another flag is signaled to indicate whether integer luma sample MV precision or 4 luma sample MV precision is used.

[0093] If the first MVD decomposition flag of a CU is zero or is not coded for the CU (i.e., all MVDs in the CU are zero), quarter luma sample MV decomposition is used for that CU. If a CU uses integer luma sample MV precision or a 4 luma sample MVD uses zero precision, the MVPs in the AMVP candidate list for that CU are rounded to the corresponding precision.

[0094] [2.3.3 Example of affine motion compensation prediction] In HEVC, only a translation motion model is applied for Motion Compensation Prediction (MCP). However, the camera and the object may have many kinds of motions, such as zoom in / out, rotation, projective motion, and other irregular motions. In VVC, a simplified affine transformation motion compensation prediction is applied by a four-parameter affine model and a six-parameter affine model. As shown in Figures 13A and 13B, the affine motion field of a block is described by two (in the four-parameter affine model using variables a, b, e, and f) or three (in the six-parameter affine model using variables a, b, c, d, e, and f) control point motion vectors, respectively.

[0095] The motion vector field (MVF) of a block is described by the four-parameter affine model and the six-parameter affine model, respectively, by the following equations:

number

[0096] Here, (mv h 0,mv h 0) is the (CP) motion vector of the upper left corner control point, and (mv h 1,mv h 1) is the motion vector of the control point in the upper right corner, (mv h 2,mv h 2) is the motion vector of the control point of the lower left corner, and (x,y) represent the coordinates of the representative point relative to the top-left sample in the current block. The CP motion vector may be signaled (as in affine AMVP mode) or derived on-the-fly (as in affine mode). w and h are the width and height of the current block. In practice, the division is performed by right-shifting with rounding operation. In VTM, the representative point is defined to be the center position of the subblock, e.g., if the coordinates of the upper-left corner of the subblock relative to the top-left sample in the current block are (xs,ys), then the coordinates of the representative point are defined to be (xs+2,ys+2). For each subblock (e.g., 4x4 in VTM), the representative point is used to derive the motion vector for the entire subblock.

[0097] FIG. 15 shows an example of affine MVF for each subblock of block 1300, and subblock-based affine transformation prediction is applied to further simplify motion compensation prediction. To derive the motion vector of each M×N subblock, the motion vector of the center sample of each subblock is calculated according to equations (1) and (2), and may be rounded to motion vector fractional precision (e.g., 1 / 16 in JEM). Then, a motion compensation interpolation filter is applied to generate a prediction of each subblock according to the derived motion vector. An interpolation filter for 1 / 16 pel is introduced by the affine mode. After MCP, the high-precision motion vector of each subblock is rounded and saved as the same precision as the normal motion vector.

[0098] [2.3.3.1 Example of signaling for affine prediction] Similar to the translational model, there are also two modes for signaling side information with the affine model: AFFINE_INTER and AFFINE_MERGE.

[0099] [2.3.3.2 AF_INTER mode example] For CUs with both width and height greater than 8, AF_INTER mode may be applied. An affine flag at CU level is signaled in the bitstream to indicate whether AF_INTER mode is used. In this mode, for each reference picture list (List0 or List1), an affine AMVP candidate list is constructed with three types of affine motion predictors in the following order, where each candidate contains the estimated CPMV of the current block: 1) the difference between the best CPMV found at the encoder side (e.g., mv0, mv1, mv2 in FIG. 17) and the estimated CPMV is signaled. Furthermore, the index of the affine AMVP candidate from which the estimated CPMV is derived is further signaled.

[0100] 1) Genetic (i.e., inherited) affine motion predictors The test order is similar to that of spatial MVP in HEVC AMVP list construction. First, the left genetic affine motion predictor is derived from the first block in {A1, A0} that has the same reference picture as the current block and is affine coded. Second, the top genetic affine motion predictor is derived from the first block in {B1, B0, B2} that has the same reference picture as the current block and is affine coded. Five blocks A1, A0, B1, B0, B2 are represented in FIG. 16. When it is found that the neighboring blocks are coded in affine mode, the CPMV of the coding unit covering the neighboring blocks is used to derive the predictor of the CPMV of the current block. For example, if A1 is coded in non-affine mode and A0 is coded in 4-parameter affine mode, the left side genetic affine MV predictor will be derived from A0. In this case, for the top left CPMV in FIG. 18B, MV0 N and MV1 for the upper right CPMV N The CPMV of the CU covering A0, represented by C , MV1 C , MV2 C The estimated CPMV of the current block is then derived, denoted by

[0101] 2) Constructed affine motion predictor The constructed affine motion predictor consists of control point motion vectors (CPMVs) derived from neighboring inter-coded blocks that have the same reference picture, as shown in Figure 17. If the current affine motion model is 4-parameter affine, the number of CPMVs is 2, otherwise, if the current affine motion model is 6-parameter affine, the number of CPMVs is 3. Top-left CPMV [Outside 1] TIFF0007597475000002.tif8127 (hereafter, bar mv0) is derived by the MV of the first block in the inter-coded group {A, B, C} that has the same reference picture as the current block. [Outside 2] TIFF0007597475000003.tif8127 (hereafter, bar mv1) is derived by the MV of the first block in the inter-coded group {D,E} that has the same reference picture as the current block. [Outside 3] TIFF0007597475000004.tif8127 (hereafter, mv2) is derived by the MV of the first block in the inter-coded group {F,G} that has the same reference picture as the current block. - If the current affine motion model is four-parameter affine, the constructed affine motion predictor is inserted into the candidate list only if both mv0 and mv1 are determined, i.e., mv0 and mv1 are used as the estimated CPMV for the top-left position (having coordinates (x0, y0)) and top-right position (having coordinates (x1, y1)) of the current block. - If the current affine motion model is a 6-parameter affine, the constructed affine motion predictor is inserted into the candidate list only if mv0, mv1 and mv2 are all determined, i.e., mv0, mv1 and mv2 are used as the estimated CPMVs for the top-left position (having coordinates (x0,y0)), top-right position (having coordinates (x1,y1)) and bottom-right position (having coordinates (x2,y2)) of the current block. If the constructed affine motion predictor inserts into the candidate list, the pruning process is not applied.

[0102] 3) Regular AMVP motion predictor The following is applied until the number of affine motion predictions reaches a maximum value. 1) Derive the affine motion predictors by setting all CPMVs equal to mv2 if available. 2) Derive the affine motion predictor by setting all CPMVs equal to mv1 if available. 3) Derive the affine motion predictors by setting all CPMVs equal to mv0 if available. 4) Derive affine motion predictors by setting all CPMVs equal to HEVC TMVP if available. 5) Derive the affine motion predictor by setting all CPMVs to zero MVs. It should be noted that, [Outside 4] TIFF0007597475000005.tif8127 (hereafter, bar mv i ) is a point that has already been derived in the constructed affine motion.

[0103] In AF_INTER mode, when 4 or 6 parameter affine modes are used, 2 or 3 control points are needed, so 2 or 3 MVDs need to be coded for those control points, as shown in Figures 15A and 15B. In existing implementations, the MVs may be derived as follows, e.g., it predicts mvd1 and mvd2 from mvd0.

number

[0104] Here, bar mv i , mvd i and mv i mvA(xA,yA) and mvB(xB,yB) are the predicted motion vector, motion vector differential, and motion vector of the top-left pixel (i=0), top-right pixel (i=1), or bottom-left pixel (i=2), respectively, as shown in FIG. 15B. In some embodiments, the addition of two motion vectors (e.g., mvA(xA,yA) and mvB(xB,yB)) is equal to the sum of the two components separately. For example, newMV=mvA+mvB implies that the two components of newMV are set to (xA+xB) and (yA+yB), respectively.

[0105] [2.3.3.3 AF_MERGE Mode Example] When a CU is applied in AF_MERGE mode, it obtains the first block coded by affine mode from the valid neighboring reconstructed blocks. And the selection order of the candidate blocks is from left to top, top right, bottom left, top left (represented by A, B, C, D, E, respectively) as shown in Figure 18A. For example, if the neighboring bottom left block is coded in affine mode, as represented by A0 in Figure 18B, the control point (CP) motion vectors mv0 of the top left corner, top right corner, and bottom left corner of the neighboring CU / PU containing block A are N , mv1 N and mv2 N Then, the upper left corner / upper right corner / lower left corner motion vector mv0 on the current CU / PU is fetched. C , mv1 C and mv2 C (Only used for 6-parameter affine models) is mv0 N , mv1 N and mv2 N It should be noted that in VTM-2.0, if the current block is affine coded, the sub-block located in the upper left corner (e.g., in VTM, a 4x4 block) stores mv0, and the sub-block in the upper right corner stores mv1. If the current block is coded with a 6-parameter affine model, the sub-block in the lower left corner stores mv2, otherwise (with a 4-parameter affine model), the LB stores mv2'. The other sub-blocks store the MV used for MC.

[0106] After the CPMVs v0 and v1 of the current CU are calculated according to the affine motion model of equations (1) and (2), the MVF of the current CU may be generated. To identify whether the current CU is coded in AF_MERGE mode, an affine flag may be signaled in the bitstream if there is at least one neighboring block coded in affine mode.

[0107] In some embodiments (e.g., JVET-L0142 and JVET-L0632), the affine merge candidate list may be constructed by the following steps:

[0108] 1) Insertion of affine candidates by inheritance An inherited affine candidate means that the candidate is derived from the affine motion model of its valid neighboring affine coded blocks. Up to two inherited affine candidates are derived from the affine motion models of neighboring blocks and inserted into the candidate list. For the left predictor, the scan order is {A0,A1}, and for the top predictor, the scan order is {B0,B1,B2}.

[0109] 2) Insertion of constructed affine candidates If the number of candidates in the affine merge candidate list is less than MaxNumAffineCand (set to 5 in this configuration), a constructed affine candidate is inserted into the candidate list. A constructed affine candidate means that the candidate is constructed by combining the neighboring motion information of each control point.

[0110] a) The motion information of a control point is first derived from the designated spatial and temporal neighborhoods shown in Figure 19. CPk (k=1,2,3,4) represents the kth control point. A0, A1, A2, B0, B1, B2 and B3 are the spatial positions for predicting CPk (k=1,2,3). T is the temporal position for predicting CP4. The coordinates of CP1, CP2, CP3 and CP4 are (0,0), (W,0), (H,0) and (W,H), respectively, where W and H are the width and height of the current block. The motion information for each control point is obtained according to the following priority: For CP1, the check priority is B2→B3→A2. B2 is used if it is available. Otherwise, if B3 is available, B3 is used. If both B2 and B3 are unavailable, A2 is used. If all three candidates are unavailable, the motion information of CP1 cannot be obtained. For CP2, the check priority is B1→B0. For CP3, the check priority is A1 → A0. For CP4, T is used.

[0111] b) Second, a combination of the control points is used to construct affine merge candidates. I. Motion information of three control points is required to construct a six-parameter affine candidate. The three control points can be selected from one of the following four combinations: {CP1,CP2,CP4}, {CP1,CP2,CP3}, {CP2,CP3,CP4}, {CP1,CP3,CP4}. The combinations {CP1,CP2,CP3}, {CP2,CP3,CP4}, {CP1,CP3,CP4} will be transformed into a six-parameter motion model represented by the top-left, top-right, and bottom-left control points. II. Motion vectors of two control points are needed to construct a four-parameter affine candidate. The two control points can be selected from one of the following six combinations: {CP1,CP4}, {CP2,CP3}, {CP1,CP2}, {CP2,CP4}, {CP1,CP3}, {CP3,CP4}. The combinations {CP1,CP4}, {CP2,CP3}, {CP2,CP4}, {CP1,CP3}, {CP3,CP4} will be transformed into a four-parameter motion model represented by the top-left and top-right control points. III. The constructed combinations of affine candidates are inserted into the candidate list in the following order: {CP1,CP2,CP3}, {CP1,CP2,CP4}, {CP1,CP3,CP4}, {CP2,CP3,CP4}, {CP1,CP2}, {CP1,CP3}, {CP2,CP3}, {CP1,CP4}, {CP2,CP4}, {CP3,CP4}. i. For a combined reference list X (X is 0 or 1), the reference index with the maximum utilization among the control points is selected as the reference index of list X, and the motion vector pointing to the differential reference picture will be scaled. After the candidates are derived, a full pruning process is performed to check whether the same candidate is inserted into the list. If the same candidate exists, the derived candidate is discarded.

[0112] 3) Padding with zero motion vectors If the number of candidates in the affine merge candidate list is less than five, then a zero motion vector with a zero reference index is inserted into the candidate list until the list is full. More specifically, for the sub-block merging candidate list, the 4-parameter merging candidates have MV set to (0,0) and prediction direction set to uni-prediction (for P slices) and bi-prediction (for B slices) from list 0.

[0113] [2.3.4 Example of Merging by Motion Vector Difference (MMVD)] In JVET-L0054, the Ultimate Motion Vector Expression (UMVE) (also known as Merge mode with Motion Vector Difference (MMVD)) is presented. UMVE is a proposed motion vector expression method used in either skip or merge mode.

[0114] UMVE reuses the same merge candidates as those included in the regular merge candidate list in VVC. Among the merge candidates, a base candidate can be selected and is further extended by the proposed motion vector representation method.

[0115] UMVE provides a new motion vector difference (MVD) representation, in which a starting point, a motion magnitude, and a motion direction are used to represent the MVD.

[0116] The proposed technique uses the merge candidate list as is. However, only candidates with the default merge type (MRG_TYPE_DEFAULT_N) are considered for extending UMVE.

[0117] The base candidate index defines a starting point. It points to the best candidate among the candidates in the list, as follows: [Table 1]

[0118] If the number of base candidates is equal to 1, the base candidate IDX is not signaled.

[0119] The distance index is the information of the magnitude of the movement. The distance index indicates a predefined distance from the starting point information. The predefined distances are as follows: [Table 2]

[0120] The direction index represents the direction of the MVD relative to the starting point. The direction index can represent four directions, as shown below. [Table 3]

[0121] In some embodiments, the UMVE flag is signaled immediately after sending the skip and merge flags. If the skip and merge flags are true, the UMVE flag is parsed. If the UMVE flag is equal to 1, the UMVE syntax is parsed. But if it is not 1, the AFFINE flag is parsed. If the AFFINE flag is equal to 1, it is affine mode. But if it is not 1, the skip / merge index is parsed for the VTM's skip / merge mode.

[0122] No additional line buffers for UMVE candidates are needed, because the software skip / merge candidates are directly used as base candidates. With the input UMVE index, the completion of the MV is determined just before motion compensation. Therefore, there is no need to maintain a long line buffer.

[0123] In the current general test conditions, either the first or second merge candidate in the merge candidate list can be selected as the base candidate.

[0124] [2.3.5 Decoder-side Motion Vector Refinement (DMVR) Example] In bi-predictive operation, for prediction of one block region, two prediction blocks formed using the motion vectors (MVs) of list 0 and MVs of list 1, respectively, are combined to form a single prediction signal. In the decoder-side motion vector refinement (DMVR) method, the two motion vectors of bi-prediction are further refined.

[0125] In the JEM design, the motion vectors are refined by a bilateral template matching process. Bilateral template matching is applied at the decoder to perform a distortion-based search between the bilateral template and the reconstructed samples in the reference picture to obtain the refined MV without transmitting additional motion information. An example is illustrated in FIG. 22. The bilateral template is generated as a weighted combination (i.e., average) of two prediction blocks from the initial MV0 of list 0 and MV1 of list 1, respectively, as shown in FIG. 22. The template matching operation consists of calculating a cost metric between the generated template and a sample region (around the initial prediction block) in the reference picture. For each of the two reference pictures, the MV that results in the minimum template cost is considered as the updated MV of that list to replace the original MV. In JEM, nine MV candidates are searched for each list. The nine MVs include the original MV and eight surrounding MVs offset by one luma sample with respect to the original MV in either the horizontal or vertical direction or both directions. Finally, two new MVs, namely MV0' and MV1' shown in Figure 22, are used to generate the final bi-prediction result. The sum of absolute differences (SAD) is used as the cost index. Note that when calculating the cost of a prediction block generated by one surrounding MV, the rounded MV (to integer pels) is actually used to obtain the prediction block instead of the actual MV.

[0126] To further simplify the DMVR process, JVET-M0147 proposed some changes to the design in JEM. More specifically, the adopted DMVR for VTM-4.0 (soon to be released) has the following main features: ○Early termination at (0.0) position SAD between list 0 and list 1 ○DMVR block size W×H>=64&&H>=8 ○For DMVR with CU size > 16x16, divide the CU into multiple 16x16 sub-blocks ○ Reference block size (W+7)×(H+7) (for luma) ○ 25-point SAD-based integer pel search (i.e., (+-)2 narrow search range, single stage) ○DMVR based on bilinear interpolation ○ MVD mirroring between list 0 and list 1 to enable bilateral matching Subpel refinement based on the "Parametric error surface equation" ○ Luma / chroma MC with reference block padding (if necessary) ○ Refined MV used only for MC and TMVP

[0127] [2.3.6 Combined Intra-Inter Prediction (CIIP) Example] In JVET-L0100, multi-hypothesis prediction is proposed, and combined intra and inter prediction is one method for generating multiple hypotheses.

[0128] When multiple hypothesis prediction is applied to improve intra mode, the multiple hypothesis prediction combines one intra prediction and one merge indexed prediction. In merge CU, one flag is signaled for merge mode to select intra mode from intra candidate list if the flag is true. For luma component, intra candidate list is derived from four intra prediction modes including DC, planar, horizontal, and vertical modes, and the size of intra candidate list can be 3 or 4 depending on block shape. If CU width is larger than twice CU height, horizontal mode is excluded from intra mode list, and if CU height is larger than twice CU width, vertical mode is excluded from intra list mode. One intra prediction mode selected by intra mode index and one merge indexed prediction selected by merge index are combined using weighted average. For chroma component, DM is always applied without extra signaling. The weight for combining prediction is described as follows: Equal weights are applied when DC or planar mode is selected or the CB width or height is less than 4. For CBs whose CB width and height are 4 or more, when horizontal / vertical mode is selected, one CB is first divided vertically / horizontally into 4 equal area regions. (w_intra1,w_inter1)=(6,2), (w_intra2,w_inter2)=(5,3), (w_intra3,w_inter3)=(3,5), and (w_intra4,w_inter4)=(2,6), where i is 1 to 4 and (w_intra1,w_inter1)=(6,2), (w_intra2,w_inter2)=(5,3), (w_intra3,w_inter3)=(3,5), and (w_intra4,w_inter4)=(2,6), i ,w_inter iEach weight set, denoted as w_intra1, w_inter1, will be applied to a corresponding region. (w_intra1, w_inter1) is for the region closest to the reference sample, and (w_intra4, w_inter4) is for the region farthest from the reference sample. A combined prediction may then be calculated by summing the two weighted predictions and right shifting them by 3 bits. Furthermore, the intra prediction mode for the intra hypothesis of the predictor may be saved for reference of the next neighboring CU.

[0129] [2.4 In-loop reshaping (ILR) in JVET-M0427] The basic idea of ​​in-loop reshaping (ILR) is to transform the original (first domain) signal (the predicted / reconstructed signal) into the second domain (the reshaped domain).

[0130] The in-loop luma reshaper is implemented as a pair of look-up tables (LUTs), but only one of the two LUTs needs to be signaled; the other can be calculated from the signaled LUT. Each LUT is a one-dimensional, 10-bit, 1024-entry mapping table (1D-LUT). One LUT is the forward LUT (FwdLUT), which is used to calculate the input luma code value Y i The value after changing Y r :Y r =FwdLUT[Y i The other LUT is the inverse LTU (InvLUT), which maps the modified code values ​​Y r of [Outside 5] Map to TIFF0007597475000010.tif9127 ( [Outside 6] TIFF0007597475000011.tif9127 is Y i represents the reconstructed value of .

[0131] [2.4.1 Piecewise Linear (PWL) Model] In some embodiments, piece-wise linear (PWL) is implemented as follows:

[0132] Let x1, x2 be two input pivot points and y1, y2 be their corresponding output pivot points of a piece. The output value y for any input value x between x1 and x2 can be interpolated by the following formula: y=((y2-y1) / (x2-x1))×(x-x1)+y1

[0133] In a fixed-point implementation, the formula is: y=((m×x+2FP_PREC-1)>>FP_PREC)+c where m is a scalar, c is an offset, and FP_PREC is a constant value that specifies the precision.

[0134] In the CE-12 software, the PWL model is used to recalculate the 1024 entry FwdLUT and InvLUT mapping tables, but it should be noted that the PWL model also allows for implementations that calculate the same mapping values ​​on-the-fly without recalculating the LUTs.

[0135] [2.4.2 Test CE12-2] [2.4.2.1 Luma Reshaping] Test 2 of in-loop luma reshaping (i.e., proposed CE12-2) provides a lower complexity pipeline that also removes the decoding latency of block-by-block intra prediction in inter slice reconstruction. Intra prediction is performed in the reshaped domain for both inter and intra slices.

[0136] Intra prediction is always performed in the reshaped domain, regardless of slice type. With such an arrangement, intra prediction can start immediately after the previous UT reconstruction is performed. Such an arrangement can also provide a unified process for intra modes instead of being slice-dependent. Figure 23 shows a block diagram of the CE12-2 combined process based on modes.

[0137] CE12-2 also tests a 16-piece piecewise linear (PWL) model for luma and chroma residual scaling instead of the 32-piece PWL model of CE12-1.

[0138] Interslice reconstruction with in-loop luma reshaper in CE12-2 (light green shaded blocks indicate the signal in the reshaped regions: luma residual, predicted intra-luma, and reconstructed intra-luma).

[0139] [2.4.2.2 Luma-Dependent Chroma Residual Scaling (LCRS)] Luma-dependent chroma residue scaling (LCRS) is a multiplicative process implemented with fixed-point integer arithmetic. Chroma residual scaling compensates for the interaction between the luma and chroma signals. Chroma residual scaling is applied at the TU level. More specifically, the following is applied: For intra, the reconstructed luma is averaged. For inter, the predicted luma is averaged.

[0140] The average is used to identify an index in the PWL model that identifies the scaling factor cScaleInv by which the chroma residual is multiplied.

[0141] The chroma scaling factors are calculated from the forward-mapped predicted luma values ​​rather than the reconstructed luma values.

[0142] [2.4.2.3 Signaling of ILR side information] The parameters are (currently) transmitted in tile group headers (similar to ALF). They reportedly take 40-100 bits. Tile groups can be other ways of representing pictures. The following table is based on version 9 of JVET-L1001. Added syntax is highlighted in italics. [Table 4] [Table 5] [Table 6] [Table 7] TIFF0007597475000016.tif245161

[0143] [2.4.2.4 Use of ILR] At the encoder side, each picture (or tile group) is first transformed into the reshaped domain. Then, all coding processes are performed in the reshaped domain. For intra prediction, the neighboring blocks are in the reshaped domain. For inter prediction, the reference block (generated from the original domain, from the decoded picture buffer) is first transformed into the reshaped domain. Then, the residual is generated and coded into the bitstream.

[0144] After the entire picture (or tile group) has finished encoding / decoding, the samples in the reshaped region are transformed back to the original region and then the deblocking filter and other filters are applied.

[0145] Forward reshaping on the prediction signal is disabled if: The current block is intra-coded The current block is coded as current picture referencing (CPR), also known as intra-block copy (IBC). The current block is coded as a combined inter-intra mode (CIIP) and forward reshaping is disabled for intra predicted blocks.

[0146] [JVET-N0805] To avoid signaling ILR side information in the tile group header, JVET-N0805 proposes to signal them in the APS. It includes the following main ideas: - Optionally, send LMCS parameters in the SPS. LMCS refers to the luma mapping with chroma scaling (LMCS) technique defined in the relevant video coding standard. - Define the APS type for the ALF and LMCS parameters. Each APS has only one type. - If LMCS tools are enabled, have a flag in TGH to indicate the presence or absence of LMCS aps_id. If not signaled, the SPS parameters are used. *Semantic constraints need to be added to always have something valid referenced when the tool is enabled.

[0147] [2.5.2.5.1 Implementation of the proposed design on JVET-M1001 (VVC Working Draft 4)] Below, proposed changes are shown in italics. [Table 8] TIFF0007597475000018.tif253166TIFF0007597475000019.tif206157

[0148] [2.4.2.6 JVET-N0138] This contribution proposes an extended use of adaptation parameter sets (APS) for the carriage of reshaper model parameters in addition to ALF parameters. At the last meeting, it was decided that ALF parameters are carried by APS instead of tile group headers in order to improve coding efficiency by avoiding unnecessary redundant signaling of parameters in multiple tile groups. Based on the same reason, it is proposed to carry reshaper model parameters by APS instead of tile group headers. In order to identify the type of parameters in APS (at least whether ALS or reshaper model), APS type information is required in the APS syntax along with APS ID.

[0149] Below, proposed changes are shown in italics. [Table 9]

[0150] [2.5 Virtual Pipeline Data Unit (VPDU)] A virtual pipeline data unit (VPDU) is defined as a non-overlapping M×M-Luma (L) / N×N-Chroma (C) unit in a picture. In a hardware decoder, consecutive VPDUs are processed by multiple pipeline stages at the same time, with different stages processing different VPDUs simultaneously. Since the VPDU size is roughly proportional to the buffer size in most pipeline stages, keeping the VPDU size small is very important. In a HEVC hardware decoder, the VPDU size is set to the maximum transform block (TB) size. Increasing the maximum TB size from 32×32-L / 16×16-C (similar to HEVC) to 64×64-L / 32×32-C (similar to VVC today) can result in coding gains that predictably result in a 4x VPDU size (64×64-L / 32×32-C) compared to HEVC. However, in addition to quad-tree (QT) coding unit (CU) partitioning, ternary tree (TT) and binary tree (BT) are adopted in VVC to achieve further coding gain, and the TT and BT partitioning can be recursively applied to a 128x128-L / 64x64-C coding tree block (CTU), which is said to result in a 16x VPDU size (128x128-L / 64x64-C) compared to HEVC.

[0151] In the current design of VVC, the VPDU is defined as 64x64-L / 32x32-C.

[0152] [2.6 Adaptive Parameter Set] An adaptation parameter set (APS) is adopted in VVC to carry ALF parameters. The tile group header contains an aps_id that is conditionally present if ALF is enabled. The APS contains the aps_id and the ALF parameters. A new NUT (NAL unit type as found in AVC and HEVC) value is assigned for APS (from JVET-M0132). For typical test conditions in VTM-4.0 (as shown), it is proposed to send APS with each picture using aps_id=0. For the time being, APS IDs range from 0 to 31 and APS are sharable across pictures (they may be different for different tile groups in a picture). ID values ​​should be fixed-length coded if present. ID values ​​cannot be reused for different content in the same picture.

[0153] [2.7 Related Tools] [2.7.1 Diffusion Filter (DF)] In JVET-L0157, a diffusion filter is proposed, and the intra / inter prediction signal of a CU may be further modified by the diffusion filter.

[0154] Uniform diffusion filter. The uniform diffusion filter is defined as follows: I as or h IV This is achieved by convolving the prediction signal with a fixed mask given as: In addition to the prediction signal itself, one line of reconstructed samples to the left and top of the block are used as input for the filtered signal; the use of those reconstructed signals may be avoided in inter blocks.

[0155] Let pred be the prediction signal for a given block obtained by intra or motion compensated prediction. To handle the filter boundary values, the prediction signal is ext This extended prediction can be formed in two ways.

[0156] As an intermediate step, one line of reconstructed samples from the left and top of the block is added to the prediction signal and the resulting signal is then mirrored in all directions. Alternatively, only the prediction signal itself is mirrored in all directions. The latter extension is used for inter blocks. In this case, only the prediction signal itself is mirrored in all directions. ext The scalar does not have input for

[0157] Filter h I If h is to be used, the predicted signal pred can be expanded using the boundary extension described above. I × pred, where the filter mask h I teeth,

number

[0158] Filter h IV If the prediction signal is to be used, IV × pred, where the filter h IV teeth, h IV =h I ×h I ×h I ×h I is given as:

[0159] Directional diffusion filter. Instead of using a signal-adaptive diffusion filter, we use a directional filter, i.e., a horizontal filter h hor and vertical filter h ver are used, which still have a fixed mask. More precisely, the mask h IThe uniform diffusion filtering corresponding to is simply restricted to be applied only along the vertical or horizontal direction. The vertical filter is realized by applying a fixed filter mask to the prediction signal:

number

[0160] [2.7.2 Bilateral Filter (BF)] A bilateral filter is proposed in JVET-L0406, which is always applied to luma blocks with non-zero transform coefficients and slice quantization parameters greater than 17. Therefore, there is no need to signal the use of a bilateral filter. When applied, the bilateral filter is performed on the decoded samples immediately after the inverse transform. Moreover, the filter parameters, i.e., weights, are explicitly derived from the coded information.

[0161] The filtering process is:

number

[0162] More specifically, the weight W associated with the kth neighboring sample k (x) is defined as follows:

number

number

[0163] In order to better capture the statistical characteristics of the video signal and improve the performance of the filter, the weighting function obtained from equation (2) is adjusted to σ depending on the parameters of block partitioning (minimum size) and coding mode, as listed in Table 4. d is adjusted by parameters. [Table 10]

[0164] To further improve the coding performance, for an inter-coded block when the TU is not split, the intensity difference between the current sample and one of its neighboring samples is replaced by a representative intensity difference between two windows covering the current sample and the neighboring samples. Thus, the equation for the filtering process is:

number

[0165] [2.7.3 Hadamard transform domain filter (HF)] In JVET-K0068, an in-loop filter in the 1D Hadamard transform domain is applied at the CU level after reconstruction and has a multiplication-free implementation. The proposed filter is applied to all CU blocks that satisfy predefined conditions, and the filter parameters are derived from the coded information.

[0166] The proposed filtering is always applied to luma reconstructed blocks with non-zero transform coefficients, except for 4x4 blocks, when the slice quantization parameter is greater than 17. The filter parameters are explicitly derived from the coded information. The proposed filter, when applied, is performed on the decoded samples immediately after the inverse transform.

[0167] For each pixel from the reconstructed block, the pixel processing has the following steps: Scan the four adjacent pixels around the processed pixel including the current one according to a scan pattern 4-point Hadamard transform of the read pixels The following formula

number

[0168] where (i) is the index of the spectral component in the Hadamard spectrum, R(i) is the spectral component of the reconstructed pixel corresponding to the index, and σ is expressed as:

number

[0169] An example of a scan pattern is shown in Figure 26, where A is the current pixel and {B,C,D} are within the current CU.

[0170] [Shortcomings of existing implementations] Existing ILR implementations may have the following shortcomings:

[0171] 1) Signaling ILR side information in the tile group header is not feasible since it requires too many bits. Moreover, prediction between different pictures / tile groups is not allowed. Therefore, ILR side information needs to be transmitted for each tile group, which may cause coding loss at low bitrates, especially for low resolutions.

[0172] 2) The interaction between ILR and DMVR (or other newly introduced coding tools) is unclear. For example, ILR is applied to the inter-prediction signal to transform the original signal into the reshaped domain, and the decoded residual is in the reshaped domain. Meanwhile, DMVR also relies on the prediction signal to refine the motion vector of one block. It is unclear whether DMVR should be applied in the original domain or the reshaped domain.

[0173] 3) The interaction between the ILR and screen content coding tools, e.g., palette, B-DPCM, IBC, transform-skip, transform-quantization-bypass, I-PCM mode, is not clear.

[0174] 4) Luma-dependent chroma residual scaling is used in the ILR, thus introducing additional delay (due to the dependency between luma and chroma) that is not beneficial for hardware design.

[0175] 5) The goal of the VPDU is to guarantee the completion of processing one 64x64 square before starting processing another 64x64 square. However, due to the design of the ILR, there are no restrictions on the use of the ILR that could cause a violation of the VPDU, as chroma depends on the luma prediction signal.

[0176] 6) When all zero coefficients occur in one CU, the prediction block and the reconstruction block still perform the forward and backward reshaping process, which wastes computational complexity.

[0177] 7) In JVET-N0138, it is proposed to signal ILR information in APS. Some new problems may be caused by this solution. For example, two types of APS are designed, but the adaptation_parameter_set_id signaled for ILR may refer to an APS that does not contain ILR information. Similarly, the adaptation_parameter_set_id signaled for adaptive loop filtering (ALF) may refer to an APS that does not contain ALF information.

[0178] 4. Exemplary Methods of In-Loop Reshaping for Video Coding Embodiments of the presently disclosed technology overcome the shortcomings of existing implementations, thereby bringing higher coding efficiency to video coding. The in-loop reshaping method based on the disclosed technology can enhance both existing and future video coding standards, as illustrated in the following examples that describe various implementations. The examples of the disclosed technology provided below illustrate general concepts and are not intended to be construed as limiting. Various features described in the examples may be combined, unless expressly indicated to the contrary in the examples. It should be noted that some of the proposed technologies may be applied to existing candidate list construction processes.

[0179] In this specification, decoder-side motion vector derivation (DMVD) includes methods such as DMVR and FRUC that perform motion estimation to derive or refine block / sub-block motion information, and BIO that performs sample-by-sample motion refinement. Various examples (Examples 1 to 42) are given in the numbered list below.

[0180] 1. The motion information refinement process in DMVD techniques such as DMVR may rely on information in the reshaped domain. In one example, a prediction block generated from a reference picture in the original domain may first be transformed to the reshaped domain before being used for motion information refinement. i. Alternatively, and in addition, cost calculations (eg, SAD, MR-SAD) / gradient calculations are performed in the reshaped domain. ii. Alternatively, further, after the motion information is refined, the reshaping process is disabled for prediction blocks generated with the refined motion information. b. Alternatively, the motion information refinement process in DMVD techniques such as DMVR may rely on information in the original domain. The i.DMVD process may be invoked with the predicted block in the original domain. ii. In one example, after motion information refinement, the prediction block obtained by the refined motion information, or the final prediction block (e.g., a weighted average of two prediction blocks) may be further transformed into a reshaped domain to generate a final reconstructed block. iii. Alternatively, further, after the motion information is refined, the reshaping process is disabled for prediction blocks generated with the refined motion information.

[0181] 2. It is proposed to align the regions (either the original regions or the reshaped regions) of samples in the current tile / tile group / picture and those derived from the reference picture used to derive the Local Illumination Compensation (LIC) parameters. In one example, the reshaped regions are utilized to derive LIC parameters. i. Alternatively or additionally, samples (e.g., reference samples (with or without interpolation) in a reference picture and adjacent / non-adjacent samples (with or without interpolation) of the reference sample) may be first transformed into a reshaped domain before being used to derive the LIC parameters. b. In one example, the original domain is utilized to derive LIC parameters. i. Alternatively or additionally, spatially adjacent / non-adjacent samples of the current block (e.g., within the current tile group / picture / tile) may first be transformed back to the original domain before being used to derive the LIC parameters. c. If the LIC parameters are derived in one region, then the same region of the predicted block should be used when applying the LIC parameters to that predicted block. i. In one example, when a. above is invoked, the reference block may be transformed into the reshaped domain and the LIC model is applied to the reshaped reference block. ii. In one example, when b. above is invoked, the reference block remains in the original domain and the LIC model is applied to the reference block in the original domain. d. In one example, the LIC model is applied to the prediction block in the reshaped domain (eg, the prediction block is first transformed to the reshaped domain by forward reshaping). e. In one example, the LIC model may first be applied to a prediction block in the original domain, after which the final prediction block dependent on the LIC-applied prediction block may then be transformed (e.g., by forward reshaping) to the reshaped domain and utilized to derive a reconstructed block. f. The above method may be extended to other coding tools that rely on both spatially adjacent / non-adjacent samples and reference samples within reference pictures.

[0182] 3. For a filter applied to a predicted block (eg, a diffusion filter (DF)), the filter is applied to the predicted block in the original domain. a. Alternatively or additionally thereafter, a reshaping is applied to the filtered prediction signal to generate the reconstructed block. b. An example of a process for inter-coding is shown in FIG. c. Alternatively, the filter is applied to the predicted signal in the reshaped domain. i. Alternatively or additionally, the reshaping may be applied to the prediction block first, after which a filtering method may be further applied to the reshaped prediction block to generate a reconstructed block. ii. An example of a process for inter-coding is shown in FIG. d. The filter parameters may depend on whether the ILR is enabled or not.

[0183] 4. For filters applied to the reconstructed block (eg, bilateral filters (BF), Hadamard transform domain filters (HF)), the filters are applied to the reconstructed block in the original domain, not in the reshaped domain. a. Alternatively or additionally, the reconstructed block in the reshaped domain may first be transformed back to the original domain and then a filter may be applied and utilized to generate the reconstructed block. b. An example of a process for inter-coding is shown in FIG. c. Alternatively, the filter may be applied to the reconstructed block in the reshaped domain. Alternatively or even before applying the inverse reshaping, the filter may be applied first, after which the filtered reconstructed block may then be transformed back to the original domain. ii. An example of a process for inter-coding is shown in FIG. d. The filter parameters may depend on whether the ILR is enabled or not.

[0184] 5. It is proposed to apply a filtering process that can be applied to the reconstructed blocks in the reshaped domain (eg after intra / inter or other types of prediction methods). In one example, a deblocking filter (DBF) process is performed in the reshaped domain. In this case, no backward reshaping is applied before the DBF. i. In this case, the DBF parameters may differ depending on whether reshaping is applied or not. ii. In one example, the DBF process may depend on whether reshaping is enabled or not. 1. In one example, this method is applied when a DBF is called in the original domain. 2. Alternatively, this method is applied when DBF is invoked in the reshape region. b. In one example, a sample adaptive offset (SAO) filtering process is performed in the reshaped domain. In this case, no backward reshaping is applied before the SAO. c. In one example, an adaptive loop filter (ALF) filtering process is performed in the reshape domain. In this case, no backward reshaping is applied before the ALF. d. Alternatively or additionally, inverse reshaping may be applied to the blocks after DBF. e. Alternatively or additionally, reverse reshaping may be applied to the blocks after SAO. f. Alternatively or additionally, inverse reshaping may be applied to the blocks after ALF. g. The above filtering methods may be replaced with other types of filtering methods.

[0185] 6. It is proposed to signal the ILR parameters in a new parameter set (eg, ILR APS) instead of the tile group header. In one example, the tile group header may include an aps_id. Alternatively, or in addition, the aps_id is conditionally present if ILR is enabled. b. In one example, the ILR APS includes the aps_id and ILR parameters. c. In one example, a new NUT (NAL unit type as found in AVC and HEVC) value is assigned to the ILR APS. d. In one example, the ILR APS ID value ranges from 0 to M (eg, M=2K-1). e. In one example, the ILR APS may be shared across a picture (and may be different for different tile groups within a picture). In one example, the ID value, if present, may be fixed-length coded. Alternatively, it may be coded using Exponential-Golomb (EG) coding, truncated unary, or other binarization methods. g. In one example, ID values ​​cannot be reused by different content within the same picture. h. In one example, the ILR APS and the APS for ALF parameters may share the same NUT. Alternatively, the ILR parameters may be carried in the current APS for the ALF parameters. In this case, the above method describing the ILR APS may be replaced with the current APS. j. Alternatively, the ILR parameters may be carried in the SPS / VPS / PPS / Sequence Header / Picture Header. k. In one example, the ILR parameters may include reshaper model information, ILR method usage, and chroma residual scaling factor. l. Alternatively, further, the ILR parameters may be signaled in one instance (e.g., in the APS) and / or the utilization of the ILR may be further signaled at a second level (e.g., in the tile group header). m. Alternatively or additionally, predictive coding may be applied to code the ILR parameters at different APS indexes.

[0186] 7. Instead of applying Luma-Dependent Chroma Residual Scaling (LCRS) to chroma blocks, it is proposed to apply a forward / backward reshaping process to chroma blocks to remove the dependency between luma and chroma. In one example, one M-piece piecewise linear (PWL) model and / or forward / backward look-up table may be used for one chroma component. Alternatively, two PWL models and / or forward / backward look-up tables may be used to code two chroma components, respectively. b. In one example, the chroma PWL model and / or forward / backward references may be derived from the luma PWL model and / or forward / backward lookup tables. i. In one example, there is no need to further signal the chroma PWL model / lookup table. c. In one example, the chroma PWL model / forward / reverse lookup table may be signaled in the SPS / VPS / APS / PPS / Sequence Header / Picture Header / Tile Group Header / Tile Header / CTU Row / Group of CTU / Region.

[0187] 8. In one example, the method of signaling the ILR parameters of one picture / tile group may depend on the ILR parameters of a previously coded picture / tile group. For example, the ILR parameters of one picture / tile group may be predicted by the ILR parameters of one or more previously coded pictures / tile groups.

[0188] 9. It is proposed to disable Luma-Dependent Chroma Residual Scaling (LCRS) for certain block dimensions / temporal layers / tile group types / picture types / coding modes / specific types of motion information. a. In one example, even if a forward / backward reshaping process is applied to a luma block, the LCRS may not be applied to the corresponding chroma block. b. Alternatively, even if the forward / backward reshaping process is not applied to a luma block, the LCRS may still be applied to the corresponding chroma block. In one example, LCRS is not used when a cross-component linear model (CCLM) mode is applied. CCLM modes include LM, LM-A, and LM-L. In one example, LCRS is not used when a cross-component linear model (CCLM) mode does not apply. CCLM modes include LM, LM-A, and LM-L. e. In one example, when the coded lumablock exceeds one VPDU (e.g., 64×64). i. In one example, when including samples where the lumablock size is smaller than M×H, e.g., 16 or 32 or 64 lumasamples, LCRS is not allowed. ii. Alternatively, when the minimum size of the width and / or height of the lumablock is not smaller than or not larger than X, LCRS is not allowed. In one example, X is set to 8. iii. Alternatively, when the minimum size of the width and / or height of the lumablock is greater than or equal to X, LCRS is not allowed. In one example, X is set to 8. iv. Alternatively, when the width of the lumablock > th1 or >= th1 and / or the height of the lumablock > th2 or >= 2, LCRS is not allowed. In one example, th1 and / or th2 are set to 8. 1. In one example, th1 and / or th2 are set to 128. 2. In one example, th1 and / or th2 are set to 64. v. Alternatively, when the width of the lumablock < th1 or <= th1 and / or the height of the lumablock th2 < or <= th2, LCRS is not allowed. In one example, th1 and / or th2 are set to 8.

[0189] 10. Whether to disable ILR (forward reshaping process and / or reverse reshaping process) may depend on the coefficient. a. In one example, when one block is coded with all zero coefficients, the forward reshaping process applied to the prediction block is skipped. b. In one example, when one block is coded with all zero coefficients, the reverse reshaping process applied to the reconstruction block is skipped. c. In one example, when one block is coded with only one non-zero coefficient at a specific position (e.g., the DC coefficient at the upper left position of one block, the coefficient in the upper left coding group within one block), the process of forward reshaping applied to the prediction block and / or the reverse reshaping applied to the reconstruction block is skipped. d. In one example, when one block is coded with only M (e.g., M = 1) non-zero coefficients, the process of forward reshaping applied to the prediction block and / or the reverse reshaping applied to the reconstruction block is skipped.

[0190] 11. It is proposed to divide the ILR application region into virtual pipeline data units (VPDUs) when the coded block exceeds one VPDU. Each application region (e.g., having a maximum size of 64×64) is regarded as an individual CU for the ILR operation. a. In one example, when the width of the block > th1 or >= th1 and / or the height of the block > th2 or >= th2, it may be divided into sub-blocks having a width < th1 or <= th1 and / or a height < th2 or <= th2, and the ILR may be executed for each sub-block. i. In one example, the sub-blocks may have the same width and / or height. ii. In one example, the sub-blocks except those at the right boundary and / or the lower boundary may have the same width and / or height. iii. In one example, the sub-blocks except those at the left boundary and / or the upper boundary may have the same width and / or height. b. In one example, when the size of the block (i.e., width × height) > th3 or >= th3, it may be divided into sub-blocks having a size < th3 or <= th3, and the ILR may be executed for each sub-block. i. In one example, the sub-blocks may have the same size. ii. In one example, the sub-blocks except those at the right boundary and / or the lower boundary may have the same size. iii. In one example, sub - blocks excluding those at the left boundary and / or the upper boundary may have the same size. c. Alternatively, the use of ILR is restricted only to specific block dimensions. i. In one example, when the coded block exceeds one VPDU (e.g., 64×64), ILR is not allowed. ii. In one example, when the sample with a block size smaller than M×H, e.g., including 16 or 32 or 64 luma samples, ILR is not allowed. iii. Alternatively, when the minimum size of the width and / or height of the block is not smaller than or not larger than X, ILR is not allowed. In one example, X is set to 8. iv. Alternatively, when the minimum size of the width and / or height of the block is X or more, ILR is not allowed. In one example, X is set to 8. v. Alternatively, when the width of the block > th1 or >= th1 and / or the height of the block > th2 or >= th2, ILR is not allowed. In one example, th1 and / or th2 are set to 8. 1. In one example, th1 and / or th2 are set to 128. 2. In one example, th1 and / or th2 are set to 64. vi. Alternatively, when the width of the block < th1 or <= th1 and / or the height of the block < th2 or <= th2, ILR is not allowed. In one example, th1 and / or th2 are set to 8.

[0191] 12. The above - mentioned method (e.g., whether ILR should be disabled and / or whether LCRS should be disabled and / or whether to signal the PWL / reference table for chroma coding) may depend on color formats such as 4:4:4 / 4:2:0.

[0192] 13. An indication to enable the ILR (eg, tile_group_reshaper_enable_flag) may be coded under the condition of an indication of the reshaper model being presented (eg, tile_group_reshaper_model_present_flag). a. Alternatively, tile_group_reshaper_model_present_flag may be coded under the condition of tile_group_reshaper_enable_flag. b. Alternatively, only one of the two syntax elements including tile_group_reshaper_model_present_flag and tile_group_reshaper_enable_flag may be coded, with the value of the other syntax element being set equal to the one that may be signaled.

[0193] 14. Different clipping methods may be applied to the prediction signal and the reconstruction process. In one example, an adaptive clipping method may be applied, where maximum and minimum values ​​to be clipped may be defined in the reshape region. b. In one example, an adaptive clipping method may be applied to the predicted signal in the reshaped domain. c. Alternatively, also a fixed clipping (eg according to bit depth) may be applied to the reconstructed block.

[0194] 15. Filter parameters (eg, those used in DF, BF, HF) may depend on whether ILR is enabled or not.

[0195] 16. It is proposed that for blocks coded in palette mode, the ILR be disabled or applied differently. In one example, if the block is coded in palette mode, reshaping and backward reshaping are skipped. b. Alternatively, if the block is coded in palette mode, a different reshaping and a backward reshaping may be applied.

[0196] 17. Alternatively, the palette mode may be coded differently when ILR is applied. In one example, when ILR is applied, the palette mode may be coded in the original domain. b. Alternatively, if ILR is applied, the palette mode may be coded in the reshaped region. c. In one example, when ILR is applied, the palette predictor may be signaled in the original domain. d. Alternatively, the palette predictor may be signaled in the reshape domain.

[0197] 18. It is proposed that for blocks coded in IBC mode, the ILR be disabled or applied differently. In one example, if the block is coded in IBC mode, reshaping and backward reshaping are skipped. b. Alternatively, if the block is coded in IBC mode, different reshaping and backward reshaping may be applied.

[0198] 19. Alternatively, the IBC may be coded differently when the ILR applies. In one example, when ILR is applied, IBC mode may be performed in the original region. b. Alternatively, if ILR is applied, the IBC mode may be performed in the reshaping region.

[0199] 20. It is proposed that for blocks coded in B-DPCM mode, the ILR is either disabled or applied differently. In one example, if the block is coded in B-DPCM mode, reshaping and backward reshaping are skipped. b. Alternatively, if the block is coded in B-DPCM mode, different reshaping and backward reshaping are applied.

[0200] 21. Alternatively, the B-DPCM modes may be coded differently when ILR is applied. In one example, when ILR is applied, B-DPCM may be performed in the original domain. b. Alternatively, when ILR is applied, B-DPCM may be performed in reshaping mode.

[0201] 22. It is proposed that for blocks coded in transform skip mode, the ILR be disabled or applied differently. In one example, if the block is coded in transform skip mode, the reshaping and backward reshaping are skipped. b. Alternatively, different reshaping and backward reshaping may be applied if the block is coded in transform skip mode.

[0202] 23. Alternatively, the transform skip mode may be coded differently when ILR is applied. In one example, when ILR is applied, transform skipping may be performed in the original domain. b. Alternatively, when ILR is applied, transform skipping may be performed in the reshaped domain.

[0203] 24. It is proposed that for blocks coded in I-PCM mode, the ILR is either disabled or applied differently. In one example, if the block is coded in palette mode, reshaping and backward reshaping are skipped. b. Alternatively, if the block is coded in palette mode, a different reshaping and a backward reshaping may be applied.

[0204] 25. Alternatively, the I-PCM modes may be coded differently when ILR is applied. In one example, when ILR is applied, the I-PCM mode may be coded in the original domain. b. Alternatively, if ILR is applied, the I-PCM mode may be coded in the reshaped region.

[0205] 26. It is proposed that for blocks coded in transform quantization bypass mode, the ILR be disabled or applied differently. In one example, if the block is coded in transform quantization bypass mode, reshaping and backward reshaping are skipped.

[0206] 27. Alternatively, different reshaping and backward reshaping are applied when the block is coded in transform quantization bypass mode.

[0207] 28. Regarding the above bullet point, if the ILR is disabled, the forward reshaping and / or reverse reshaping process may be skipped. a. Alternatively, the prediction and / or the reconstruction and / or the residual signal is in the original domain. b. Alternatively, the prediction and / or the reconstructed and / or the residual signal is in the reshaped domain.

[0208] 29. Multiple reshaping / reverse reshaping functions (e.g., multiple PWL models) may be enabled for coding one picture / one tile group / one VPDU / one region / one CTU row / multiple CUs. a. The method of selecting from multiple features may depend on block dimensions / coding mode / picture type / low latency check flag / motion information / reference pictures / video content, etc. b. In one example, multiple sets of ILR side information (eg, reshaping / reverse reshaping capabilities) may be signaled per SPS / VPS / PPS / sequence header / picture header / tile group header / tile header / region / VPDU / etc. i. Alternatively or additionally, predictive coding of the ILR side information may be utilized. c. In one example, more than one aps_id may be signaled in the PPS / picture header / tile group header / tile header / region / VPDU / etc.

[0209] 30. In one example, the reshaping information is signaled in a new syntax set other than VPS, SPS, PPS, or APS. For example, the reshaping information is signaled in a set denoted as inloop_reshaping u_parameter_set() (IRPS or other name). An example syntax design is below, with added syntax highlighted in italics. [Table 11] b. An example syntax design is below, with added syntax highlighted in italics. [Table 12]

[0210] 31. In one example, IRL information is signaled together with ALF information in the APS. An example syntax design is below, with added syntax highlighted in italics. [Table 13] b. In one example, one tile_group_aps_id is signaled in the tile group header to indicate the adaptation_paramter_set_id of the APS that the tile group refers to. Both ALF and ILR information for the current tile group are signaled in the specified APS. i. An example syntax design is below, where added syntax is highlighted in italics. [Table 14]

[0211] 32. In one example, ILR information and ALF information are signaled in different APS. A first ID (which may be called tile_group_aps_id_alf) is signaled in the tile group header to indicate the first adaptation_paramter_set_id of the first APS that the tile group references. The ALF information for the current tile group is signaled in the designated first APS. b. A second ID (which may be called tile_group_aps_id_irps) is signaled in the tile group header to indicate the second adaptation_paramter_set_id of the second APS that the tile group references. The ILR information for the current tile group is signaled in the designated second APS. c. In one example, the first APS should have ALF information in the conformance bitstream. d. In one example, the second APS should have ILR information in the conformance bitstream. e. An example syntax design is below, with added syntax highlighted in italics. [Table 15]

[0212] 33. In one example, some APSs that are specified with adaptation_paramter_set_id should have ALF information. As another example, some APSs that are specified with adaptation_paramter_set_id should have ILR information. For example, an APS with adaptation_paramter_set_id equal to 2N should have ALF information, where N is any integer. b. For example, an APS with adaptation_paramter_set_id equal to 2N+1 should have ILR information, where N is any integer. c. An example syntax design is below, with added syntax highlighted in italics. [Table 16] For example, 2×tile_group_aps_id_alf indicates the first adaptation_paramter_set_id of the first APS referenced by the tile group. The ALF information for the current tile group is signaled in the specified first APS. ii. For example, 2×tile_group_aps_id_irps+1 indicates the second adaptation_paramter_set_id of the second APS that the tile group refers to. The ILR information for the current tile group is signaled in the designated second APS.

[0213] 34. In one example, a tile group cannot reference an APS (or IRPS) signaled before a Network Abstraction Layer (NAL) unit of a specified type that is signaled before the current tile group. In one example, a tile group cannot reference an APS (or IRPS) signaled before a tile group of a specified type that is signaled before the current tile group. b. For example, a tile group cannot reference an APS (or IRPS) that is signaled before an SPS that is signaled before the current tile group. c. For example, a tile group cannot reference an APS (or IRPS) that is signaled before a PPS that is signaled before the current tile group. d. For example, a tile group cannot reference an APS (or IRPS) signaled before an access unit delimiter (AUD) NAL that is signaled before the current tile group. e. For example, a tile group cannot reference an APS (or IRPS) that is signaled before an end of bitstream (EoB) NAL that is signaled before the current tile group. f. For example, a tile group cannot reference an APS (or IRPS) that is signaled before an end of sequence (EoS) NAL that is signaled before the current tile group. g. For example, a tile group cannot reference an APS (or IRPS) signaled before an instantaneous decoding refresh (IDR) NAL that is signaled before the current tile group. h. For example, a tile group cannot reference an APS (or IRPS) that is signaled before a clean random access (CRA) NAL that is signaled before the current tile group. i. For example, a tile group cannot reference an APS (or IRPS) that was signaled before an intra random access point (IRAP), which is signaled before the current tile group. j. For example, a tile group cannot reference an APS (or IRPS) that was signaled before an I tile group (or picture, or slice), which is signaled before the current tile group. The methods disclosed in k. IDF-P1903237401H and IDF-P1903234501H are also applicable when the ILR information is carried in the APS or IRPS.

[0214] 35. A conformance bitstream should satisfy that if the in-loop reshaping method is enabled for one video data unit (eg, a sequence), default ILR parameters, such as a default model, should be defined. a.sps_lmcs_default_model_present_flag should be set to 1 if sps_lmcs_enabled_flag is set to 1. b. A default parameter may be set to false under the ILR enable flag instead of the default model present flag (e.g., sps_lmcs_default_model_present_flag). c. For each tile group, a default model use flag (e.g., tile_group_lmcs_use_default_model_flag) may be signaled without reference to the SPS default model use flag. d. The conformance bitstream should satisfy that if there is no ILR information in the corresponding APS type of ILR and one video data unit (e.g., a tile group) is forced to use the ILR technique, the default model should be utilized. e. Alternatively, in the conformance bitstream, if there is no ILR information in the corresponding APS type of the ILR and one video data unit (e.g., a tile group) is forced to use the ILR technique (e.g., tile_group_lmcs_enable_flag is equal to 1), the instruction to use the default model should be true, e.g., tile_group_lmcs_use_default_model_flag should be 1. f. It is constrained that the default ILR parameters (eg, default model) should be transmitted in the video data unit (eg, SPS). i. Alternatively or additionally, the default ILR parameters should be sent if the SPS flag indicating the use of the ILR is true. g. It is constrained that there is at least one ILR APS transmitted in the video data unit (eg, SPS). i. In one example, at least one ILR APS includes default ILR parameters (eg, a default model).

[0215] 36. Default ILR parameters may be indicated by a flag. If this flag indicates that the default ILR parameters are to be used, no further ILR data needs to be sent.

[0216] 37. A default ILR parameter may be predefined if it is not signaled. For example, the default ILR parameter may correspond to an identity mapping.

[0217] 38. Temporal layer information may be signaled together with the ILR parameters, for example in the ILR APS. In one example, the temporal layer index may be signaled in lmcs_data(). b. In one example, a temporal layer index -1 may be signaled in lmcs_data(). c. Alternatively, or in addition, when encoding / decoding one tile group / tile, referencing an ILR APS associated with a smaller or equal temporal layer index is restricted. d. Alternatively, when encoding / decoding one tile group / tile, referencing an ILR APS associated with a smaller temporal layer index is restricted. e. Alternatively, when encoding / decoding one tile group / tile, referencing an ILR APS associated with a higher temporal layer index is restricted. f. Alternatively, when encoding / decoding one tile group / tile, referencing ILR APSs associated with a greater or equal temporal layer index is restricted. g. Alternatively, when encoding / decoding one tile group / tile, referencing ILR APSs associated with the same temporal layer index is restricted. h. In one example, whether the above restrictions apply may depend on a piece of information that may be signaled to or inferred by the decoder.

[0218] 39. The temporal layer information may be signaled together with the ALF information, for example in the ALF APS. In one example, the temporal layer index may be signaled in alf_data(). b. In one example, a temporal layer index -1 may be signaled in alf_data(). c. Alternatively, or in addition, when encoding / decoding one tile group / tile or one CTU within one tile group / tile, referencing an ALF APS associated with a smaller or equal temporal layer index is restricted. d. Alternatively, when encoding / decoding one tile group / tile, referencing ALF APSs associated with a smaller temporal layer index is restricted. e. Alternatively, when encoding / decoding one tile group / tile, referencing ALF APS associated with a higher temporal layer index is restricted. f. Alternatively, when encoding / decoding one tile group / tile, referencing ALF APSs associated with a greater or equal temporal layer index is restricted. g. Alternatively, when encoding / decoding one tile group / tile, referencing ALF APSs associated with the same temporal layer index is restricted. h. In one example, whether the above restrictions apply may depend on a piece of information that may be signaled to or inferred by the decoder.

[0219] 40. In one example, the reshaping mapping between the original samples and the reshaped samples may not be a positive relationship, ie, a larger value is not allowed to be mapped to a smaller value. a. For example, the reshaped mapping between the original and reshaped samples may be a negative relationship, where in the case of two values, a larger value in the original domain may be mapped to a smaller value in the reshaped domain.

[0220] 41. In a conformance bitstream, the syntax element aps_params_type is only allowed to have some predefined values, such as 0 and 1. In another example, only 0 and 7 are allowed.

[0221] 42. In one example, default ILR information should be signaled if ILR applies (eg, sps_lmcs_enabled_flag is true).

[0222] [5. Examples of the disclosed technology] In some embodiments, tile_group_reshaper_enable_flag is conditionally present if tile_group_reshaper_model_present_flag is enabled. Added syntax is highlighted in italics. [Table 17] TIFF0007597475000037.tif63159

[0223] Embodiment #2 on JVET-N0805. Added syntax is highlighted in italics. [Table 18] TIFF0007597475000039.tif253166TIFF0007597475000040.tif197156

[0224] Luma Mapping with Chroma Scaling (LMCS) Data Syntax [Table 19]

[0225] The above examples may be incorporated in connection with the methods described below, for example the methods illustrated in Figures 31A to 39E, which may be implemented in a video decoder or video encoder.

[0226] FIG. 31A shows a flowchart of an example method for video processing, where method 3110 includes, at step 3112, performing a motion information refinement process based on samples in a first domain or a second domain for conversion between a current video block of the video and a coded representation of the video. Method 3120 includes, at step 3114, performing a conversion based on results of the motion information refinement process. In some implementations, during the conversion, samples are obtained for the current video block from a first predictive block in the first domain using unrefined motion information, and at least a second predictive block is generated in the second domain using refined motion information used to determine a reconstructed block, and reconstructed samples of the current video block are generated based on at least the second predictive block.

[0227] FIG. 31B shows a flowchart of an example method for video processing. The method 3120 includes, at step 3122, reconstructing a current video block based on at least one predictive block in a second region. In some implementations, during the transform, the current video block is constructed based on the first region and the second region, and / or the chroma residual is scaled depending on the luma. In some implementations, a coding tool is applied during the transform using parameters derived based at least on a first set of samples in a video region of the video and a second set of samples in a reference picture of the current video block. In some implementations, the first region of samples and the second region of samples are aligned.

[0228] FIG. 32A shows a flowchart of an example method for video processing. The method 3210 includes, at step 3212, determining, for a current video block of a current video region of a video, parameters for a coding mode of the current video block based on one or more parameters for a coding mode of a previous video region. The method 3210 further includes, at step 3214, performing coding on the current video block to generate a coded representation of the video based on the determination. In some implementations, the parameters for the coding mode are included in a parameter set in the coded representation of the video. In some implementations, performing the coding includes converting a representation of the current video block in a first domain to a representation of the current video block in a second domain. In some implementations, while performing the coding using the coding mode, the current video block is constructed based on the first domain and the second domain, and / or the chroma residual is scaled depending on the luma.

[0229] 32B shows a flowchart of an example method for video processing. The method 3220 includes, at step 3222, receiving a coded representation of the video including a parameter set including parameter information for a coding mode. The method 3220 further includes, at step 3224, performing decoding of the coded representation by using the parameter information to generate a current video block of a current video region of the video from the coded representation. In some implementations, the parameter information for the coding mode is based on one or more parameters for a coding mode of a previous video region. In some embodiments, in the coding mode, the current video block is constructed based on a first region and a second region, and / or the chroma residual is scaled depending on the luma.

[0230] 32C shows a flowchart of an example method for video processing. The method 3230 includes, at step 3232, performing a transformation between a current video block of the video and a coded representation of the video. In some implementations, the transformation includes applying a filtering operation to the predictive block in a first domain or in a second domain different from the first domain.

[0231] 32D shows a flowchart of an example method for video processing. The method 3240 includes, at step 3242, performing a transformation between a current video block of the video and a coded representation of the video. In some implementations, during the transformation, a final reconstructed block is determined for the current video block. In some implementations, a temporary reconstructed block is generated using a prediction method and represented in a second domain.

[0232] 33 shows a flowchart of an example method for video processing. The method 3300 includes, at step 3302, performing a conversion between a current video block of a video region of a video and a coded representation of the video, where the conversion uses a coding mode in which the current video block is constructed based on a first region and a second region and / or the chroma residual is scaled depending on the luma, and a parameter set in the coded representation has parameter information for the coding mode.

[0233] 34A shows a flowchart of an example method for video processing. The method 3410 includes, at step 3412, performing a transformation between a current video block of the video, the current video block being a chroma block, and a coded representation of the video, where during the transformation the current video block is constructed based on a first region and a second region, and the transformation further includes applying a forward reshape process and / or a backward reshape process to one or more chroma components of the current video block.

[0234] 34B shows a flowchart of an example method for video processing. The method 3420 includes, at step 3422, performing a conversion between a current video chroma block of the video and a coded representation of the video, the conversion including determining, based on the rule, whether luma-dependent chroma residual scaling (LCRS) is enabled or disabled, and reconstructing the current video chroma block based on the determination.

[0235] 35A shows a flowchart of an example method for video processing. The method 3510 includes, at step 3512, determining whether to disable use of a coding mode for transforming between a current video block of a video and a coded representation of the video based on one or more coefficient values ​​of the current video block. The method 3510 further includes, at step 3514, performing the transform based on the determination. In some implementations, during the transform using the coding mode, the current video block is constructed based on a first region and a second region, and / or a chroma residual is scaled dependent on the luma.

[0236] 35B shows a flowchart of an example method for video processing. The method 3520 includes, at step 3522, dividing a current video block into regions for conversion between the current video block of the video over a virtual pipeline data unit (VPDU) of the video and a coded representation of the video. The method 3520 further includes, at step 3524, performing the conversion by applying a coding mode to each region separately. In some implementations, during the conversion by applying the coding mode, the current video block is constructed based on the first region and the second region, and / or the chroma residual is scaled depending on the luma.

[0237] 35C shows a flowchart of an example method for video processing. The method 3530 includes, at step 3532, determining whether to disable use of a coding mode for conversion between a current video block of a video and a coded representation of the video based on a size or color format of the current video block. The method 3530 further includes, at step 3534, performing the conversion based on the determination. In some implementations, during conversion using the coding mode, the current video block is constructed based on the first region and the second region, and / or the chroma residual is scaled depending on the luma.

[0238] 35D shows a flowchart of an example method for video processing. The method 3540 includes, at step 3542, performing a conversion between a current video block of a video and a coded representation of the video, where the conversion uses a coding mode in which the current video block is constructed based on a first region and a second region and / or the chroma residual is scaled depending on the luma, and where at least one syntax element in the coded representation provides an indication of the use of the coding mode and an indication of the reshaper model.

[0239] 35E shows a flowchart of an example method for video processing. The method 3550 includes, at step 3552, determining that a coding mode is disabled for conversion between a current video block of a video and a coded representation of the video. The method 3550 further includes, at step 3554, conditionally skipping the forward reshaping and / or the backward reshaping based on the determination. In some implementations, in the coding mode, the current video block is constructed based on a first region and a second region, and / or the chroma residual is scaled dependent on the luma.

[0240] 35F shows a flowchart of an example method for video processing. The method 3560 includes, at step 3562, performing a conversion between a current video block of a video region of the video and a coded representation of the video, where the conversion uses a coding mode in which the current video block is constructed based on a first region and a second region and / or in which chroma residual is scaled dependent on luma, and where multiple forward reshapes and / or multiple backward reshapes are applied to the video region in the reshape mode.

[0241] 36A shows a flowchart of an example method for video processing. The method 3610 includes, at step 3612, determining that a coding mode is enabled for converting between a current video block of a video and a coded representation of the video. The method 3610 further includes, at step 3614, performing the conversion using a palette mode in which at least a palette of representative sample values ​​is used for the current video block. In some implementations, in the coding mode, the current video block is constructed based on samples in a first domain and a second domain, and / or the chroma residual is scaled dependent on the luma.

[0242] 36B shows a flowchart of an example method for video processing. The method 3620 includes, at step 3622, determining that a current video block is coded in a palette mode in which at least a palette of representative sample values ​​is used to code the current video block for conversion between the current video block and a coded representation of the video. The method 3620 further includes, at step 3624, performing the conversion by disabling the coding mode upon the determination. In some implementations, when a coding mode is applied to a video block, the video block is constructed based on a luma-dependent scaled chroma residual.

[0243] FIG. 36C shows a flowchart of an example method for video processing. The method 3630 includes, at step 3632, performing a conversion between a first video block of a video and a coded representation of the video, the conversion using a first coding mode and a palette coding mode in which at least a palette of representative sample values ​​is used to code the current video block. The method 3630 further includes, at step 3634, performing a conversion between a second video block of a video coded without using a palette coding mode and the coded representation of the video, the conversion of the second video block using the first coding mode. When the first coding mode is applied to a video block, the video block is constructed based on a first region and a second region and / or the chroma residual is scaled depending on the luma. In some implementations, the first coding mode is applied to the first video block and the second video block in different manners.

[0244] 37A shows a flowchart of an example method for video processing. The method 3710 includes, at step 3712, determining that a coding mode is enabled for converting between a current video block of a video and a coded representation of the video. The method 3710 further includes, at step 3714, performing the conversion using an intra block copy mode that generates a predictive block using at least a block vector that points to a picture that includes the current video block. In the coding mode, the current video block is constructed based on samples in a first domain and a second domain, and / or the chroma residual is scaled depending on the luma.

[0245] 37B shows a flowchart of an example method for video processing. The method 3720 includes, at step 3722, determining that a current video block is coded in an intra block copy (IBC) mode that generates a predictive block using at least a block vector that points to a video frame that includes the current video block for coding the current video block for conversion between the current video block and a coded representation of the video. The method 3720 further includes, at step 3724, performing the conversion by disabling the coding mode upon the determination. When the coding mode is applied to the video block, the video block is constructed based on the first region and the second region and / or the chroma residual is scaled depending on the luma.

[0246] FIG. 37C shows a flowchart of an example method for video processing. The method 3730 includes, at step 3732, performing a conversion between a first video block of a video and a coded representation of the video, the conversion using an intra block copy mode that generates a predictive block using at least a block vector that points to a video frame that includes the current video block, and a first coding mode. The method 3730 further includes, at step 3734, performing a conversion between a second video block of a video coded without using the intra block copy mode and the coded representation of the video, the conversion of the second video block using the first coding mode. If the first coding mode is applied to a video block, the video block is constructed based on a first region and a second region, and / or the chroma residual is scaled depending on the luma, and the first coding mode is applied to the first video block and the second video block in a different manner.

[0247] 38A shows a flowchart of an example method for video processing. The method 3810 includes, at step 3812, determining that a coding mode is enabled for converting between a current video block of the video and a coded representation of the video. The method 3810 further includes, at step 3814, performing the conversion using a block-based delta pulse code modulation (BDPCM) mode. In the coding mode, the current video block is constructed based on samples in a first domain and a second domain, and / or the chroma residual is scaled dependent on the luma.

[0248] 38B shows a flowchart of an example method for video processing. The method 3820 includes, at step 3822, determining that a current video block is coded using a block-based delta pulse code modulation (BDPCM) mode for converting between a current video block of the video and a coded representation of the video. The method 3820 further includes, at step 3824, performing the conversion by disabling the coding mode upon the determination. When the coding mode is applied to the video block, the video block is constructed based on the first region and the second region and / or the chroma residual is scaled dependent on the luma.

[0249] FIG. 38C shows a flowchart of an example method for video processing. The method 3830 includes, at step 3832, performing a conversion between a first video block of the video and a coded representation of the video, where the conversion of the first video block uses a first coding mode and a block-based delta pulse code modulation (BDPCM) mode. The method 3830 further includes, at step 3834, performing a conversion between a second video block of the video and the coded representation of the video, where the second video block is coded without using a BDPCM mode and the conversion of the second video block uses the first coding mode. When the first coding mode is applied to a video block, the video block is constructed based on a first region and a second region, and / or the chroma residual is scaled depending on the luma, and the first coding mode is applied to the first video block and the second video block in a different manner.

[0250] 38D shows a flowchart of an example method for video processing. The method 3840 includes, at step 3842, determining that a coding mode is enabled for converting between a current video block of the video and a coded representation of the video. The method 3840 further includes, at step 3844, performing the transform using a transform skip mode in which a transform on a prediction residual is skipped in coding the current video block. In the coding mode, the current video block is constructed based on samples in a first domain and a second domain and / or the chroma residual is scaled dependent on the luma.

[0251] 38E shows a flowchart of an example method for video processing. The method 3850 includes, at step 3852, determining that the current video block is coded in a transform skip mode in which a transform on a prediction residual is skipped in coding the current video block for transforming between the current video block and a coded representation of the video. The method 3850 further includes, at step 3854, performing the transform by disabling the coding mode upon the determination. When the coding mode is applied to the video block, the video block is constructed based on the first region and the second region and / or the chroma residual is scaled depending on the luma.

[0252] FIG. 38F shows a flowchart of an example method for video processing. The method 3860 includes, at step 3862, performing a conversion between a first video block of the video and a coded representation of the video, where the conversion of the first video block uses a first coding mode and a transform skip mode in which a transform on a prediction residual is skipped in coding the current video block. The method 3860 further includes, at step 3864, performing a conversion between a second video block of the video and the coded representation of the video, where the second video block is coded without using the transform skip mode and the transform of the second video block uses the first coding mode. If the first coding mode is applied to a video block, the video block is configured based on a first region and a second region and / or the chroma residual is scaled depending on the luma, and the first coding mode is applied to the first video block and the second video block in a different manner.

[0253] 38G shows a flowchart of an example method for video processing. The method 3870 includes, at step 3872, determining that a coding mode is enabled for converting between a current video block of a video and a coded representation of the video. The method 3870 further includes, at step 3874, performing the conversion using an intrapulse code modulation mode in which the current video block is coded without applying a transform and transform domain quantization. In the coding mode, the current video block is constructed based on samples in a first domain and a second domain and / or chroma residual is scaled dependent on luma.

[0254] 38H shows a flowchart of an example method for video processing. The method 3880 includes, at step 3882, determining that the current video block is coded in an intrapulse code modulation mode in which the current video block is coded without applying a transform and transform domain quantization for conversion between the current video block of the video and the coded representation of the video. The method 3880 further includes, at step 3884, performing the conversion by disabling the coding mode upon the determination. When the coding mode is applied to the video block, the video block is constructed based on the first region and the second region and / or the chroma residual is scaled depending on the luma.

[0255] FIG. 38I shows a flowchart of an example method for video processing. The method 3890 includes, at step 3892, performing a conversion between a first video block of the video and a coded representation of the video, where the conversion of the first video block uses a first coding mode and an intrapulse code modulation mode in which the current video block is coded without applying a transform and transform domain quantization. The method 3890 further includes, at step 3894, performing a conversion between a second video block of the video and a coded representation of the video, where the second video block is coded without using an intrapulse code modulation mode and the conversion of the second video block uses the first coding mode. When the first coding mode is applied to a video block, the video block is constructed based on a first region and a second region and / or the chroma residual is scaled depending on the luma, and the first coding mode is applied to the first video block and the second video block in different manners.

[0256] FIG. 38J shows a flowchart of an example method for video processing. The method 3910 includes, at step 3912, determining that a coding mode is enabled for converting between a current video block of the video and a coded representation of the video. The method 3910 further includes, at step 3914, performing the conversion using a modified transquant-bypass mode in which the current video block is losslessly coded without transform and quantization. In the coding mode, the current video block is constructed based on samples in a first domain and a second domain and / or chroma residual is scaled dependent on luma.

[0257] 38K shows a flowchart of an example method for video processing. The method 3920 includes, at step 3922, determining that the current video block is to be coded in a transquant-bypass mode in which the current video block is losslessly coded without applying transform and quantization for conversion between the current video block of the video and a coded representation of the video. The method 3920 further includes, at step 3924, performing the conversion by disabling a coding mode upon the determination. When a coding mode is applied to the video block, the video block is constructed based on the first region and the second region and / or the chroma residual is scaled dependent on the luma.

[0258] FIG. 38L shows a flowchart of an example method for video processing. The method 3930 includes, at step 3932, performing a conversion between a first video block of the video and a coded representation of the video, where the conversion of the first video block uses a first coding mode and a transquant-bypass mode in which the current video block is losslessly coded without transform and quantization. The method 3930 further includes, at step 3934, performing a conversion between a second video block of the video and a coded representation of the video, where the second video block is coded without using the transquant-bypass mode and the conversion of the second video block uses the first coding mode. When the first coding mode is applied to a video block, the video block is configured based on a first region and a second region and / or the chroma residual is scaled depending on the luma, and the first coding mode is applied to the first video block and the second video block in different manners.

[0259] 39A shows a flowchart of an example method for video processing. The method 3940 includes, at step 3942, performing a conversion between a current video block of a video and a coded representation of the video, where the conversion uses a coding mode in which the current video block is constructed based on a first region and a second region and / or the chroma residual is scaled dependent on the luma, and the information used for the coding mode is signaled in a parameter set different from a sequence parameter set (SPS), a video parameter set (VPS), a picture parameter set (PPS), or an adaptation parameter set (APS) used to carry adaptive loop filtering (ALF) parameters.

[0260] 39B shows a flowchart of an example method for video processing. The method 3950 includes, at step 3952, performing a conversion between a current video block of a video and a coded representation of the video, where the conversion uses a coding mode in which the current video block is constructed based on a first region and a second region and / or a chroma residual is scaled dependent on luma, where the information used for the coding mode is signaled in an adaptation parameter set (APS) together with adaptive loop filtering (ALF) information, and where the information used for the coding mode and the ALF information are included in one NAL unit.

[0261] 39C shows a flowchart of an example method for video processing. The method 3960 includes, at step 3962, performing a conversion between a current video block of a video region of the video and a coded representation of the video, where the conversion uses a coding mode in which the current video block is constructed based on a first region and a second region and / or a chroma residual is scaled dependent on luma, and the information used for the coding mode is signaled with a first type of adaptation parameter set (APS) that is different from a second type of APS used to signal adaptive loop filtering (ALF) information.

[0262] 39D shows a flowchart of an example method for video processing. The method 3970 includes, at step 3972, performing a conversion between a current video block of a video region of the video and a coded representation of the video, where the conversion uses a coding mode in which the current video block is constructed based on a first region and a second region and / or in which chroma residual is scaled dependent on luma, where the video region is not allowed to reference a parameter set or an adaptive parameter set that is signaled prior to a particular type of data structure used to process the video, where the particular type of data structure is signaled prior to the video region.

[0263] 39E shows a flowchart of an example method for video processing. The method 3980 includes, at step 3982, performing a conversion between a current video block of a video and a coded representation of the video, where the conversion uses a coding mode in which the current video block is constructed based on a first region and a second region and / or in which chroma residual is scaled dependent on luma, and where a parameter set syntax element containing parameters used to process the video has a predefined value in a conformance bitstream.

[0264] FIG. 40A is a block diagram of a video processing device 4000. The device 4000 may be used to implement one or more of the methods described herein. The device 4000 may be embodied in a smartphone, a tablet, a computer, an Internet of Things (IoT) receiver, etc. The device 4000 may include one or more processors 4002, one or more memories 4004, and video processing hardware 4006. The processor 4002 may be configured to implement one or more of the methods described herein, including but not limited to the methods illustrated in FIGS. 31A-39E. The memory(s) 4004 may be used to store data and code used to implement the methods and techniques described herein. The video processing hardware 4006 is a hardware circuit that may be used to implement some of the techniques described herein.

[0265] FIG. 40B is another example of a block diagram of a video processing system in which the disclosed techniques may be implemented. FIG. 40B is a block diagram illustrating an example video processing system 4100 in which various techniques described herein may be implemented. Various implementations may include some or all of the components of the system 4100. The system 4100 may include an input 4102 that receives video content. The video content may be received in a raw or uncompressed format, for example, 8 or 10-bit multi-component pixel values, or may be in a compressed or encoded format. The input 4102 may correspond to a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, Passive Optical Network (PON), etc., and wireless interfaces such as Wi-Fi or cellular interfaces.

[0266] The system 4100 may include a coding component 4104 that may implement various coding or encoding methods described herein. The coding component 4104 may reduce the average bit rate of the video from the input 4102 to an output of the coding component 4104 to generate a coded representation of the video. Coding techniques are therefore sometimes referred to as video compression or video transcoding techniques. The output of the coding component 4104 may be either stored or transmitted over a communication connection, as represented by component 4106. The stored or communicated bitstream (or coded) representation of the video received at the input 4102 may be used by component 4108 to generate pixel values ​​or displayable video that are sent to the display interface 4110. The process of generating a user-viewable video from the bitstream representation is sometimes referred to as video decompression. Furthermore, although certain video processing operations are referred to as "coding" operations or tools, the coding tools or operations are used in an encoder, and the corresponding decoding tools or operations that cause the results of the coding to be determined will be performed in a decoder.

[0267] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB) or High Definition Multimedia Interface (HDMI), etc. Examples of storage interfaces include Serial Advanced Technology Attachment (SATA), PCI, IDE interfaces, etc. The techniques described herein may be embodied in various electronic devices such as mobile phones, laptops, smartphones, or other devices capable of performing digital data processing and / or video display.

[0268] In some embodiments, the video coding method may be implemented using an apparatus implemented on a hardware platform as described in connection with FIG. 40A or FIG. 40B.

[0269] Various techniques and embodiments may be described using the following bullet points.

[0270] The first set of clauses describes particular features and aspects of the disclosed techniques listed in the previous section, including, for example, Examples 1 and 2.

[0271] 1. A method of video processing, comprising: performing a motion information refinement process based on samples in a first domain or a second domain for a transformation between a current video block of a video and a coded representation of said video; performing said transformation based on the results of said motion information refinement process; having During the transformation, the samples are obtained for the current video block from a first predictive block in the first domain using unrefined motion information, and at least a second predictive block is generated in the second domain using refined motion information used to determine a reconstructed block, and reconstructed samples of the current video block are generated based on the at least second predictive block. method.

[0272] 2. The method of clause 1, the at least second predictive block is generated from samples in a reference picture in the first region using the refined motion information; a reconstruction process is further applied to the at least second predicted block, the reconstruction process transforming the first region into the second region. method.

[0273] 3. The method of clause 2, after the reconstruction process, the second prediction block is transformed into the representation in the second domain before being used to generate the reconstructed samples of the current video block. method.

[0274] 4. The method of clause 1, the step of performing the motion information refinement process is based on a decoder-side motion vector derivation (DMVD) method; method.

[0275] 5. The method of clause 4, The DMVD method has DMVR (decoder-side motion vector refinement) or FRUC (frame-rate up conversion) or BIO (Bi-Directional Optical Flow), method.

[0276] 6. The method of clause 4, The cost calculation or gradient calculation in the DMVD method is performed based on samples in the first region. method.

[0277] 7. The method of clause 6, The cost calculation comprises sum of absolute differences (SAD) or mean removed sum of absolute differences (MR-SAD). method.

[0278] 8. The method of clause 1, The motion information refinement process is performed based on samples transformed from at least the samples in the first prediction block in the first domain to the samples in the second domain, After the refined motion information is obtained, the current video block is constructed based on a first region and a second region, and / or a coding mode in which a chroma residual is scaled depending on luma is disabled for at least the second predictive block. method.

[0279] 9. The method of clause 4, The motion information refinement process is performed based on at least the first predicted block in the first region; the motion information refinement process is invoked by the first prediction block in the first region; method.

[0280] 10. The method of clause 1, The final prediction block is generated as a weighted sum of the two second prediction blocks, the reconstructed samples of the current video block are generated based on the final prediction block. method.

[0281] 11. The method of clause 1, The motion information refinement process is performed based on the prediction block in the first region; After performing the motion information refinement process, the current video block is reconstructed based on the first region and the second region, and / or a coding mode in which chroma residual is scaled depending on luma is disabled for the at least second predictive block. method.

[0282] 12. A method of video processing comprising: performing a conversion between a current video block of a video and a coded representation of the video; During the transformation, the current video block is reconstructed based on a first region and a second region, and / or a chroma residual is scaled depending on a luma; a coding tool is applied during the transform using parameters derived based at least on a first set of samples in a video region of the video and a second set of samples in a reference picture of the current video block; the first sample area and the second sample area are aligned; method.

[0283] 13. A method according to clause 12, the coding tool includes a local illumination compensation (LIC) model that uses a linear model of illumination changes within the current video block during the transform; The LIC model is applied based on the parameters. method.

[0284] 14. The method of clause 12, the video region includes a current tile, tile group, or picture; method.

[0285] 15. The method of clause 13, the LIC model is applied to a prediction block in the second region; the first set of samples and the second set of samples are in the second region; method.

[0286] 16. The method of clause 13, The reference block is transformed into the second domain; the LIC model is applied to the prediction block in the second region; method.

[0287] 17. The method of clause 15, the first set of samples and the second set of samples are transformed into the second domain before being used to derive the parameters. method.

[0288] 18. The method of clause 17, the second set of samples includes reference samples in a reference picture and adjacent and / or non-adjacent samples of the reference samples; method.

[0289] 19. The method of clause 13, the LIC model is applied to a prediction block in the first region; the first set of samples and the second set of samples are in the first region; method.

[0290] 20. A method according to clause 13, A reference block is held in the first region; the LIC model is applied to a prediction block in the first region; method.

[0291] 21. The method of clause 19, the first set of samples is transformed into the first domain before being used to derive the parameters. method.

[0292] 22. The method of clause 21, the first set of samples includes spatially adjacent and / or non-adjacent samples of a current video block. method.

[0293] 23. A method according to clause 12, the region used to derive the parameters is used to apply the parameters to the predictive block; method.

[0294] 24. A method according to clause 13, the LIC model is applied to the prediction block in the second region; method.

[0295] 25. The method of clause 20 or 21, After the LIC model is applied to the prediction block in the first domain, a final prediction block that depends on the prediction block is transformed to the second domain. method.

[0296] 26. Any of the methods of clauses 1 to 25, the first region is an original region, The second region is a reshaped region that uses a luma mapping with chroma scaling (LMCS) method to map luma samples to specific values. method.

[0297] 27. A method according to clause 26, The LMCS method uses a piecewise linear model to map the luma samples to the specific values. method.

[0298] 28. Any of the methods of clauses 1 to 27, performing the transformation includes generating the coded representation from the current block. method

[0299] 29. Any of the methods of clauses 1 to 27, performing the transformation includes generating the current block from the coded representation. method.

[0300] 30. An apparatus in a video system having a processor and non-transitory memory storing instructions, comprising: The instructions, when executed by the processor, cause the processor to perform any one of the methods of clauses 1 to 29. Device.

[0301] 31. A computer program product stored on a non-transitory computer-readable medium, comprising: 29. A computer program product comprising a program code for performing the method of any one of clauses 1 to 29.

[0302] The second set of clauses describes particular features and aspects of the disclosed techniques listed in the previous section, including, for example, Examples 3-5, 8, and 15.

[0303] 1. A method of video processing, comprising: determining, for a current video block of a current video region of a video, parameters for a coding mode of the current video block based on one or more parameters for a coding mode of a previous video region; performing coding on the current video block to generate a coded representation of the video based on the determination; having the parameters for the coding mode are included in a parameter set in the coded representation of the video; The step of performing the coding includes transforming a representation of the current video block in a first domain to a representation of the current video block in a second domain; During the coding using the coding mode, the current video block is constructed based on the first region and the second region, and / or a chroma residual is scaled depending on a luma. method.

[0304] 2. A method of video processing comprising: receiving a coded representation of the video including a parameter set including parameter information for a coding mode; performing decoding of the coded representation by using the parameter information to generate a current video block of a current video region of the video from the coded representation; having the parameter information for the coding mode is based on one or more parameters for the coding mode of a previous video region; In the coding mode, the current video block is constructed based on a first region and a second region, and / or a chroma residual is scaled depending on a luma. method.

[0305] 3. The method according to clause 1 or 2, the parameter set is different from the tile group header, method.

[0306] 4. The method according to clause 1 or 2, The parameter set is an adaptive parameter set (APS). method.

[0307] 5. The method according to clause 1 or 2, the current video region comprises one tile of a video picture of the video or a video picture of the video; method.

[0308] 6. The method according to clause 1 or 2, the previous video region comprises one or more tiles of a picture; method.

[0309] 7. The method according to clause 1 or 2, the previous video region comprises one or more video pictures of the video; method.

[0310] 8. A method of video processing comprising: performing a conversion between a current video block of a video and a coded representation of the video; the transforming includes applying a filtering operation to the prediction block in a first domain or in a second domain different from the first domain; method.

[0311] 9. The method of clause 8, the filtering operation is performed on the prediction block in the first domain to generate a filtered prediction signal; A coding mode is applied to the filtered prediction signal to generate a reshaped prediction signal in the second domain; the current video block is constructed using the reshaped prediction signal. method.

[0312] 10. A method according to clause 8, a coding mode is applied to the prediction block prior to application of the filtering operation to generate a reshaped prediction signal in the second domain; the filtering operation is performed using the reshaped prediction signal to generate a filtered prediction signal; the current video block is constructed using the filtered prediction signal. method.

[0313] 11. The method according to clause 9 or 10, In the coding mode, the current video block is constructed based on the first region and the second region, and / or a chroma residual is scaled depending on luma. method.

[0314] 12. Any of the methods of clauses 8 to 11, The filtering operation comprises a diffusion filter. method.

[0315] 13. Any of the methods of clauses 8 to 11, parameters associated with the filtering operation depend on whether the filtering operation is applied to blocks in the first region or the second region. method.

[0316] 14. A method according to clause 8, The conversion is applying a motion compensated prediction to the current video block to obtain a prediction signal before applying the filtering operation; applying a coding mode to a filtered prediction signal generated by applying the filtering operation to the prediction signal to generate a reshaped prediction signal after application of the filtering operation; constructing the current video block using the reshaped prediction signal; Further comprising: method.

[0317] 15. A method according to clause 8, The conversion is applying a motion compensated prediction to the current video block to obtain a prediction signal before applying the filtering operation; applying a coding mode to the prediction signal to generate a reshaped prediction signal; constructing the current video block using a filtered reshaped prediction signal after applying the filtering operation, the filtered reshaped prediction signal being generated by applying the filtering operation to the reshaped prediction signal; Further comprising: method.

[0318] 16. A method of video processing comprising the steps of: performing a conversion between a current video block of a video and a coded representation of the video; During the transformation, a final reconstruction block is determined for the current video block; A temporary reconstruction block is generated using a prediction method and represented in a second domain: method.

[0319] 17. The method of clause 16, The conversion is applying motion compensated prediction to the current video block to obtain a prediction signal; applying a forward reshaping to said prediction signal to generate a reshaped prediction signal which is used to generate said temporary reconstructed block; applying an inverse reshaping to said temporary reconstructed block to obtain an inverse reconstructed block; Further comprising: filtering is applied to the inverse reconstruction block to generate the final reconstruction block. method.

[0320] 18. The method of clause 16, The conversion is applying motion compensated prediction to the current video block to obtain a prediction signal; applying a forward reshaping to said prediction signal to generate a reshaped prediction signal which is used to generate said temporary reconstructed block; applying an inverse reshaping to the filtered reconstructed block to obtain the final reconstructed block; Further comprising: the filtered reconstructed block is generated by applying filtering to the temporary reconstructed block. method.

[0321] 19. Any of the methods of clauses 16 to 18, The conversion further includes applying a luma-dependent chroma residual scaling (LMCS) process that maps luma samples to specific values. method.

[0322] 20. A method according to clause 16, A filter is applied to the temporary reconstruction block in a first region; the temporary reconstruction block in the second domain is first transformed to the first domain using an inverse reshaping process before application of the filter; the final reconstruction block depends on the filtered temporary reconstruction block. method.

[0323] 21. A method according to clause 16, a filter is applied directly to the reconstructed block in the second domain, after which a reverse reconstruction operation is applied to generate the final reconstructed block. method.

[0324] 22. The method of clause 20 or 21, The filter comprises a bilateral filter (BF) or a Hadamard transform domain filter (HF); method.

[0325] 23. The method of clause 20 or 21, The filter comprises a deblocking filter (DBF) process, a sample adaptive offset (SAO) filtering process, or an adaptive loop filter (ALF) filtering process. method.

[0326] 24. Any one of the methods of clauses 1 to 23, filter parameters used for the filtering operation or the filter depend on whether a coding mode is enabled for the current video block, In the coding mode, the current video block is constructed based on a first region and a second region, and / or a chroma residual is scaled depending on a luma. method.

[0327] 25. Any of the methods of clauses 1 to 24, the first region is an original region, The second region is a reshaped region that uses a luma mapping with chroma scaling (LMCS) method to map luma samples to specific values. method.

[0328] 26. A method according to clause 25, The LMCS method uses a piecewise linear model to map the luma samples to the specific values. method.

[0329] 27. Any of the methods of clauses 8 to 26, performing the transformation includes generating the coded representation from the current block. method

[0330] 28. Any of the methods of clauses 8 to 26, performing the transformation includes generating the current block from the coded representation. method.

[0331] 29. An apparatus in a video system having a processor and non-transitory memory storing instructions, comprising: The instructions, when executed by the processor, cause the processor to perform any one of the methods of clauses 1 to 28. Device.

[0332] 30. A computer program product stored on a non-transitory computer-readable medium, comprising: A computer program product comprising a program code for performing the method of any one of clauses 1 to 28.

[0333] The third set of clauses describes particular features and aspects of the disclosed techniques listed in the previous section, including, for example, Example 6.

[0334] 1. A method of video processing, comprising: performing a conversion between a current video block of a video region of a video and a coded representation of the video; the transform uses a coding mode in which the current video block is constructed based on a first region and a second region and / or a chroma residual is scaled depending on luma; the parameter set in the coded representation comprises parameter information for a coding mode, method.

[0335] 2. The method of clause 1, the parameter set is different from the tile group header, method.

[0336] 3. The method of clause 2 The parameter set is an adaptive parameter set (APS). method.

[0337] 4. The method of clause 3, The APS for coding mode information is named LMCS (luma mapping with chroma scaling) APS; method.

[0338] 5. The method of clause 3, an identifier for the APS to be used for the current video block is included in the coded representation of the video. method.

[0339] 6. The method of clause 5, Whether the identifier is present in the coded representation of the video depends on whether the coding mode is enabled for the video region. method.

[0340] 7. The method of clause 3, The parameter set includes an identifier of the APS. method.

[0341] 8. The method of clause 1, A NAL unit type value is assigned to the parameter set. method.

[0342] 9. The method of clause 1, the parameter set identifier has a range between 0 and M; M is 2 K -1, method.

[0343] 10. The method of clause 1, the parameter set is shared across pictures of the video; method.

[0344] 11. The method of clause 1, The parameter set identifier has a fixed-length coded value. method.

[0345] 12. The method of clause 1, The parameter set identifier is coded using an exponential-golomb (EG) code, a truncated unary code, or a binary code. method.

[0346] 13. The method of clause 1, For two sub-regions in the same picture, the parameter sets have two different value identifiers. method.

[0347] 14. A method according to clause 3, The APS for the parameter set and adaptive loop filter (ALF) information share the same network abstraction layer (NAL) unit type (NUT). method.

[0348] 15. The method of clause 1, The parameter information is carried by a current APS for adaptive loop filter (ALF) information; method.

[0349] 16. The method of clause 1, the parameter information is carried in a sequence parameter set (SPS), a video parameter set (VPS), a picture parameter set (PPS), a sequence, header or a picture header; method.

[0350] 17. The method of clause 1, the parameter information includes at least one of an indication of reshaper model information, a use of the coding mode, or a chroma residual scaling factor; method.

[0351] 18. The method of clause 1, The parameter information is signaled at one level. method.

[0352] 19. The method of clause 1, the parameter information includes the use of the coding mode signaled at a second level. method.

[0353] 20. The method of clause 18 or 19, The parameter information is signaled in an APS; the use of the coding mode is signaled at the level of the video region. method.

[0354] 21. A method according to clause 1, The parameter information is parsed at one level. method.

[0355] 22. A method according to clause 1, the parameter information includes the use of the coding mode parsed at a second level; method.

[0356] 23. The method of clause 21 or 22, The parameter information is parsed by the APS; The coding mode usage is parsed at the level of the video region. method.

[0357] 24. A method according to clause 1, predictive coding is applied to code the parameter information at different APS indexes; method.

[0358] 25. Any of the methods of clauses 1 to 24, the first region is an original region, The second region is a reshaped region that uses a luma mapping with chroma scaling (LMCS) method to map luma samples to specific values. method.

[0359] 26. A method according to clause 25, The LMCS method uses a piecewise linear model to map the luma samples to the specific values. method.

[0360] 27. Any of the methods of clauses 1 to 26, the video region is a picture or a tile group; method.

[0361] 28. Any of the methods of clauses 1 to 26, The level of the video region is a picture header or a tile group header. method.

[0362] 29. Any of the methods of clauses 1 to 28, the first region is an original region, The second region is a reshaped region that uses a luma mapping with chroma scaling (LMCS) method to map luma samples to specific values ​​according to a reshaping model. method.

[0363] 30. The method of clause 29, The LMCS method uses a piecewise linear model to map the luma samples to the specific values. method.

[0364] 31. Any of the methods of clauses 1 to 30, performing the transformation includes generating the coded representation from the current block. method.

[0365] 32. Any of the methods of clauses 1 to 30, performing the transformation includes generating the current block from the coded representation. method.

[0366] 33. An apparatus in a video system having a processor and non-transitory memory storing instructions, comprising: The instructions, when executed by the processor, cause the processor to perform any one of the methods of clauses 1 to 32. Device.

[0367] 34. A computer program product stored on a non-transitory computer-readable medium, comprising: A computer program product comprising program code for performing the method of any one of clauses 1 to 32.

[0368] The fourth set of clauses describes particular features and aspects of the disclosed techniques listed in the previous section, including, for example, Examples 7 and 9.

[0369] 1. A method of video processing, comprising: performing a transformation between a current video block of a video, the current video block being a chroma block, and a coded representation of the video; During the transformation, the current video block is constructed based on a first region and a second region; and wherein the transforming further includes applying a forward reshape process and / or a backward reshape process to one or more chroma components of the current video block. method.

[0370] 2. The method of clause 1, The method further includes avoiding applying luma-dependent chroma residual scaling (LCRS) to the one or more chroma components of the current video block.

[0371] 3. The method of clause 1, At least one of a piecewise linear (PWL) model, a forward look-up table, or a backward look-up table is utilized for the chroma components; method.

[0372] 4. The method of clause 3, the PWL model, the forward lookup table, and the backward lookup table for the chroma components are derived from the PWL model, the forward lookup table, and the backward lookup table of the corresponding luma component, respectively; method.

[0373] 5. The method of clause 3, The PWL model is signaled in a sequence parameter set (SPS), a video parameter set (VPS), an adaptation parameter set (APS), a picture parameter set (PPS), a sequence header, a picture header, a tile group header, a tile header, a coding tree unit (CTU) row, a group of CTUs, or a region. method.

[0374] 6. The method of clause 3, the forward reference table and the backward reference table are signaled in a sequence parameter set (SPS), a video parameter set (VPS), an adaptation parameter set (APS), a picture parameter set (PPS), a sequence header, a picture header, a tile group header, a tile header, a coding tree unit (CTU) row, a group of CTUs, or a region; method.

[0375] 7. A method of video processing comprising: performing a conversion between a current video chroma block of a video and a coded representation of the video; performing the transform includes determining, based on a rule, whether luma-dependent chroma residual scaling (LCRS) is enabled or disabled, and reconstructing the current video chroma block based on the determination. method.

[0376] 8. The method of clause 7, The rules specify that the LCRS is disabled for certain block dimensions, temporal layers, tile group types, picture types, coding modes, and certain types of motion information. method.

[0377] 9. The method of clause 7, the rule specifies that the LCRS is disabled for a chroma block; A forward and / or backward reshaping process is applied to the corresponding luma block; method.

[0378] 10. The method of clause 7, the rules provide that the LCRS is applied to a chroma block; No forward and / or backward reshaping process is applied to the corresponding luma block; method.

[0379] 11. The method of clause 7, The rule specifies that the LCRS is disabled for the current video chroma block coded using a cross-component linear model (CCLM). method.

[0380] 12. A method according to clause 7, The rule specifies that the LCRS is disabled for the current video chroma blocks that are not coded using a cross-component linear model (CCLM). method.

[0381] 13. A method according to clause 7, The rule provides that disabling the LCRS is based on a size of a video block that exceeds a virtual pipeline data unit (VPDU). method.

[0382] 14. The method of clause 13, The LCRS is not permitted if the video block contains fewer than M×H video samples. method.

[0383] 15. The method of clause 13, The LCRS is not allowed if the minimum size of the width and / or height of the video block is below a certain value; method.

[0384] 16. The method of clause 13, The LCRS is not allowed if the minimum size of the width and / or height of the video block is greater than or equal to a certain value; method.

[0385] 17. The method of clause 15 or 16, The specific value is 8. method.

[0386] 18. The method of clause 13, the LCRS is not allowed if the width of the video block is greater than or equal to a first value and / or if the height of the video block is greater than or equal to a second value; method.

[0387] 19. The method of clause 13, the LCRS is not allowed if the width of the video block is less than or equal to a first value and / or if the height of the video block is less than or equal to a second value; method.

[0388] 20. The method of clause 18 or 19, At least one of the first value or the second value is 8, 64, or 128; method.

[0389] 21. Any of the methods of clauses 13 to 20, the video block is a luma block or a chroma block; method.

[0390] 22. Any of the methods of clauses 1 to 21, the first region is an original region, The second region is a reshaped region that uses a luma mapping with chroma scaling (LMCS) method to map luma samples to specific values ​​according to a reshaping model. method.

[0391] 23. The method of clause 22, The LMCS method uses a piecewise linear model to map the luma samples to the specific values. method.

[0392] 24. Any of the methods of clauses 1 to 23, the chroma residual is luma-dependently scaled by performing a luma-dependent chroma residual scaling operation that includes scaling the chroma residual before it is used to derive the reconstruction of the video chroma block, the parameters of the scaling being derived from the luma samples; method.

[0393] 25. Any of the methods of clauses 1 to 24, performing the transformation includes generating the coded representation from the current block. method.

[0394] 26. Any of the methods of clauses 1 to 24, performing the transformation includes generating the current block from the coded representation. method.

[0395] 27. An apparatus in a video system having a processor and non-transitory memory storing instructions, comprising: The instructions, when executed by the processor, cause the processor to perform any one of the methods of clauses 1 to 26. Device.

[0396] 28. A computer program product stored on a non-transitory computer-readable medium, comprising: A computer program product comprising a program code for performing the method of any one of clauses 1 to 26.

[0397] The fifth set of clauses describes particular features and aspects of the disclosed techniques listed in the previous section, including, for example, Examples 10-14, 28, 29, and 40.

[0398] 1. A method of video processing, comprising: determining whether to disable use of a coding mode for transforming between a current video block of a video and a coded representation of the video based on one or more coefficient values ​​of the current video block; performing the conversion based on the determination; having During the transform using the coding mode, the current video block is constructed based on a first region and a second region, and / or a chroma residual is scaled depending on luma. method.

[0399] 2. The method of clause 1, The remolding process selectively applying a forward reshaping process to samples in the first domain, whereby the samples in the first domain are transformed into samples in the second domain; selectively applying an inverse reshaping process to samples in the second domain, whereby the samples in the second domain are then transformed into a representation in the first domain; having method.

[0400] 3. The method according to clause 1 or 2, the reshaping process further comprising selectively applying a luma-dependent chroma residual scaling process. method.

[0401] 4. Any of the methods according to clauses 1 to 3, the determining step is based on whether the current video block is coded with all zero coefficients. method.

[0402] 5. The method of clause 2, the forward reshaping process is skipped based on whether the current video block is coded with all zero coefficients. method.

[0403] 6. The method of clause 2, the current video block is coded with all zero coefficients; The reverse reshaping process is skipped. method.

[0404] 7. The method of clause 3, the current video block is coded with all zero coefficients; The luma-dependent chroma residual scaling process is skipped. method.

[0405] 8. Any of the methods according to clauses 1 to 3, the determining step is based on whether the current video block is coded with only one non-zero coefficient at a particular position. method.

[0406] 9. The method according to clause 2 or 3, the current video block is coded with only one non-zero coefficient located at a particular position; At least one of the forward reshape process, the backward reshape process, or the luma-dependent chroma residual scaling process is skipped. method.

[0407] 10. Any of the methods according to clauses 1 to 3, the determining step is based on whether the current video block is coded with M non-zero coefficients. method.

[0408] 11. The method according to clause 2 or 3, the current video block is coded with M non-zero coefficients; At least one of the forward reshape process, the backward reshape process, or the luma-dependent chroma residual scaling process is skipped. method.

[0409] 12. The method of clause 11, M is 1, method.

[0410] 13. A method of video processing comprising: partitioning a current video block into regions for conversion between a current video block of video and a coded representation of the video over a virtual pipeline data unit (VPDU); performing said transformation by applying a coding mode to each region separately; having During the transform by applying the coding mode, the current video block is constructed based on a first region and a second region, and / or a chroma residual is scaled depending on a luma. method.

[0411] 14. The method of clause 13, Each region corresponds to an individual coding unit (CU) of the coding mode. method.

[0412] 15. The method of clause 13, the width of the current video block is greater than or equal to a first value; the current video block is divided into one or more sub-blocks having widths less than or equal to the first value; the coding mode is enabled for each sub-block; method.

[0413] 16. The method of clause 13, the height of the current video block is greater than or equal to a second value; the current video block is divided into one or more sub-blocks having heights less than or equal to the second value; the coding mode is enabled for each sub-block; method.

[0414] 17. A method according to clause 13, the size of the current video block is greater than or equal to a third value; the current video block is divided into one or more sub-blocks having a size less than or equal to the third value; the coding mode is enabled for each sub-block; method.

[0415] 18. Any of the methods of clauses 15 to 17, The sub-blocks have the same width or height. method.

[0416] 19. A method of video processing comprising: determining whether to disable use of a coding mode for conversion between a current video block of a video and a coded representation of the video based on a size or color format of the current video block; performing the conversion based on the determination; having During the transform using the coding mode, the current video block is constructed based on a first region and a second region, and / or a chroma residual is scaled depending on luma. method.

[0417] 20. The method of clause 19, determining, in said determining step, to disable the coding mode for the current video block that exceeds a virtual pipeline data unit (VPDU); method.

[0418] 21. The method of clause 19, and determining to disable the coding mode for the current video block having a size that includes fewer than M×H samples. method.

[0419] 22. A method according to clause 19, The determining step determines to disable the coding mode for the current video block if a minimum size of a width and / or height of the current video block is less than or equal to X, where X is an integer. method.

[0420] 23. A method according to clause 19, The determining step determines to disable the coding mode for the current video block if a minimum size of a width and / or height of the current video block is greater than or equal to an integer number X. method.

[0421] 24. The method of clause 22 or 23, X is 8, method.

[0422] 25. A method according to clause 19, the determining step determines to disable the coding mode for the current video block if the current video block has a width that is greater than or equal to a first value and / or a height that is greater than or equal to a second value. method.

[0423] 26. A method according to clause 19, the determining step determines to disable the coding mode for the current video block if the current video block has a width that is less than or equal to a first value and / or a height that is less than or equal to a second value. method.

[0424] 27. The method of clause 25 or 26, At least one of the first value or the second value is 8; method.

[0425] 28. Any of the methods of clauses 19 to 27, disabling the coding mode includes disabling at least one of: 1) forward reshaping to transform samples in the first domain to the second domain; 2) backward reshaping to transform samples in the second domain to the first domain; and 3) luma-dependent chroma residual scaling. method.

[0426] 29. A method of video processing comprising the steps of: performing a conversion between a current video block of a video and a coded representation of the video; the transform uses a coding mode in which the current video block is constructed based on a first region and a second region and / or a chroma residual is scaled depending on luma; At least one syntax element in the coded representation provides an indication of the use of the coding mode and an indication of a reshaper model. method.

[0427] 30. The method of clause 29, The indication of the use of the coding mode is coded based on the indication of the reshaper model. method.

[0428] 31. The method of clause 29, The indication of the reshaper model is coded based on the indication of the coding mode. method.

[0429] 32. The method of clause 29, Only one syntax element is coded, method.

[0430] 33. Any of the methods of clauses 1 to 32, Different clipping methods are applied to the predicted signal and the reconstructed signal. method.

[0431] 34. The method of clause 33, adaptive clipping is applied to the predicted signal, allowing different clipping parameters within the video; method.

[0432] 35. A method according to clause 34, maximum and minimum values ​​for the adaptive clipping are defined in the second domain; method.

[0433] 36. The method of clause 33, A fixed clipping is applied to the reconstructed signal. method.

[0434] 37. A method of video processing comprising the steps of: determining that a coding mode is disabled for a conversion between a current video block of a video and a coded representation of the video; conditionally skipping forward reshaping and / or reverse reshaping based on said determining; having In the coding mode, the current video block is constructed based on a first region and a second region, and / or a chroma residual is scaled depending on a luma. method.

[0435] 38. The method of clause 37, At least one of the prediction signal, the reconstructed signal, or the residual signal is in the first domain. method.

[0436] 39. The method of clause 37, At least one of the prediction signal, the reconstructed signal, or the residual signal is in the second domain. method.

[0437] 40. A method of video processing comprising: performing a conversion between a current video block of a video region of a video and a coded representation of the video; the transform uses a coding mode in which the current video block is constructed based on a first region and a second region and / or a chroma residual is scaled depending on luma; a plurality of forward reshapes and / or a plurality of backward reshapes are applied to said video region in a reshape mode; method.

[0438] 41. A method according to clause 40, the video region includes a picture, a tile group, a virtual pipeline data unit (VPDU), a coding tree unit (CTU), a row, or multiple coding units; method.

[0439] 42. The method of clause 40 or 41, how the multiple forward reshapes and / or the multiple backward reshapes should be selected depends on at least one of: i) block dimensions or video region dimensions; ii) a coding mode of the current video block or of the video region; iii) a picture type of the current video block or of the video region; iv) a low latency check flag of the current video block or of the video region; v) motion information of the current video block or of the video region; vi) reference pictures of the current video block or of the video region; or vii) video content of the current video block or of the video region. method.

[0440] 43. During the transformation, samples in the first domain are mapped to samples in a second domain that are smaller in value than the samples in the first domain. method.

[0441] 44. Any of the methods of clauses 1 to 43, the first region is an original region, The second region is a reshaped region that uses a luma mapping with chroma scaling (LMCS) method to map luma samples to specific values. method.

[0442] 45. A method according to clause 44, The LMCS method uses a piecewise linear model to map the luma samples to the specific values. method.

[0443] 46. ​​Any of the methods of clauses 1 to 45, performing the transformation includes generating the coded representation from the current block. method

[0444] 47. Any of the methods of clauses 1 to 45, performing the transformation includes generating the current block from the coded representation. method.

[0445] 48. An apparatus in a video system having a processor and non-transitory memory storing instructions, comprising: The instructions, when executed by the processor, cause the processor to perform any one of the methods of clauses 1 to 47. Device.

[0446] 49. A computer program product stored on a non-transitory computer-readable medium, comprising: A computer program product comprising a program code for performing the method of any one of clauses 1 to 47.

[0447] The sixth set of clauses describes particular features and aspects of the disclosed techniques listed in the previous section, including, for example, Examples 16 and 17.

[0448] 1. A method of video processing, comprising: determining a coding mode to be enabled for conversion between a current video block of a video and a coded representation of the video; performing said transformation using a palette mode in which at least the palette of representative sample values ​​is used for said current video block; having In the coding mode, the current video block is constructed based on samples in a first domain and a second domain, and / or a chroma residual is scaled depending on the luma. method.

[0449] 2. The method of clause 1, the palette of representative sample values ​​comprises at least one of: 1) a palette predictor; or 2) an escape sample. method.

[0450] 3. The method of clause 1, The representative sample value represents a value in the first region. method.

[0451] 4. The method of clause 1, the representative sample value represents a value in the second region; method.

[0452] 5. The method according to clause 1 or 2, a palette predictor used in said palette mode and included in said coded representation is in said first domain or in said second domain; method.

[0453] 6. The method according to clause 1 or 2, an escape sample used in said palette mode and included in said coded representation is in said first region or in said second region; method.

[0454] 7. The method according to clause 1 or 2, palette predictors and / or escape samples used in said palette mode and included in said coded representation are in said second region; a first reconstruction block in the second domain is generated first and is used to code a subsequent block; method.

[0455] 8. The method of clause 7, a palette predictor and / or escape sample used in an improved palette mode and included in the coded representation is in the second region; a final reconstructed block in the first region is generated using the first reconstructed block and a reverse reshaping process. method.

[0456] 9. The method of clause 8, The backward reshaping process is invoked immediately before the deblocking filter process. method.

[0457] 10. Any one of the methods of clauses 1 to 9, the transformation is performed based on color components of the current video block. method.

[0458] 11. The method of clause 10, the color component is a luma component; method.

[0459] 12. A method of video processing comprising: determining that the current video block is coded in a palette mode, where at least a palette of representative sample values ​​is used to code the current video block, for conversion between a current video block of a video and a coded representation of the video; performing said conversion by disabling a coding mode upon said determination; having and when the coding mode is applied to a video block, the video block is constructed based on a chroma residual that is scaled depending on the luma. method.

[0460] 13. The method of clause 12, If the current video block is coded in the palette mode, the coding mode is disabled. method.

[0461] 14. A method of video processing comprising: performing a conversion between a first video block of a video and a coded representation of the video, the conversion using a first coding mode and a palette coding mode in which at least a palette of representative sample values ​​is used to code the current video block; performing a conversion between a second video block of the video coded without using the palette coding mode and a coded representation of the video; having the transforming of the second video block uses the first coding mode; when the first coding mode is applied to a video block, the video block is constructed based on a first region and a second region and / or a chroma residual is scaled depending on luma; the first coding mode is applied differently to the first video block and the second video block. method.

[0462] 15. The method of clause 14, the first coding mode applied to the first video block differs from the first coding mode applied to the second video block by disabling use of forward reshaping and backward reshaping used to transform samples between the first domain and the second domain; method.

[0463] 16. The method of clause 14, the first coding mode applied to the first video block differs from the first coding mode applied to the second video block by using different reshaping and / or different backward reshaping functions used to convert samples between the first domain and the second domain; method.

[0464] 17. Any of the methods of clauses 1 to 11 and 14 to 16, the first region is an original region, The second region is a reshaped region that uses a luma mapping with chroma scaling (LMCS) method to map luma samples to specific values. method.

[0465] 18. The method of clause 17, The LMCS method uses a piecewise linear model to map the luma samples to the specific values. method.

[0466] 19. Any of the methods of clauses 1 to 18, performing the transformation includes generating the coded representation from the current block. method.

[0467] 20. Any of the methods of clauses 1 to 18, performing the transformation includes generating the current block from the coded representation. method.

[0468] 21. An apparatus in a video system having a processor and non-transitory memory storing instructions, comprising: The instructions, when executed by the processor, cause the processor to perform any one of the methods of clauses 1 to 20. Device.

[0469] 22. A computer program product stored on a non-transitory computer-readable medium, comprising: A computer program product comprising program code for carrying out the method of any one of clauses 1 to 20.

[0470] The seventh set of clauses describes particular features and aspects of the disclosed techniques listed in the previous section, including, for example, Examples 18 and 19.

[0471] 1. A method of video processing, comprising: determining a coding mode to be enabled for conversion between a current video block of a video and a coded representation of the video; performing the transformation using an intra block copy mode that generates a prediction block using at least a block vector that points to a picture containing the current video block; having In the coding mode, the current video block is constructed based on samples in a first domain and a second domain, and / or a chroma residual is scaled depending on the luma. method.

[0472] 2. The method of clause 1, The prediction block is generated in the first region. method.

[0473] 3. The method of clause 1, a residual block is represented in the coded representation in the first domain; method.

[0474] 4. The method of clause 1, The prediction block is generated in the second region. method.

[0475] 5. The method of clause 1, a residual block is represented in the coded representation in the second domain; method.

[0476] 6. The method according to clause 4 or 5, a first constituent block of the current video block is obtained based on a sum of the residual block and the predictive block in the second region; the first configuration block is utilized to convert between a subsequent video block and the coded representation of the video. method.

[0477] 7. The method according to clause 4 or 5, a final constituent block of the current video block is obtained based on a backward reshaping applied to the first constituent block to transform the first constituent block from the second domain to the first domain; method.

[0478] 8. Any one of the methods of clauses 1 to 7, the transformation is performed based on color components of the current video block. method.

[0479] 9. The method of clause 8, the color component is a luma component; method.

[0480] 10. A method of video processing comprising: determining that a current video block is coded in an intra block copy (IBC) mode for conversion between a current video block of a video and a coded representation of the video, the IBC mode comprising: generating a predictive block using at least a block vector pointing to a video frame including the current video block for coding the current video block; performing said conversion by disabling a coding mode upon said determination; having when the coding mode is applied to a video block, the video block is constructed based on a first region and a second region, and / or a chroma residual is scaled depending on the luma. method.

[0481] 11. The method of clause 10, the current video block is coded in the IBC mode; The coding mode is disabled. method.

[0482] 12. A method of video processing comprising: performing a conversion between a first video block of a video and a coded representation of the video, the conversion using an intra block copy mode that generates a predictive block using at least a block vector that points to a video frame that includes the current video block, and a first coding mode; performing a conversion between a second video block of the video coded without using the intra block copy mode and a coded representation of the video; having the transforming of the second video block uses the first coding mode; when the first coding mode is applied to a video block, the video block is constructed based on a first region and a second region and / or a chroma residual is scaled depending on luma; the first coding mode is applied differently to the first video block and the second video block. method.

[0483] 13. A method according to clause 12, the first coding mode applied to the first video block differs from the first coding mode applied to the second video block by disabling use of forward reshaping and backward reshaping used to transform samples between the first domain and the second domain; method.

[0484] 14. The method of clause 12, the first coding mode applied to the first video block differs from the first coding mode applied to the second video block by using different forward reshaping and / or different backward reshaping used to transform samples between the first domain and the second domain; method.

[0485] 15. Any of the methods of clauses 1 to 14, the first region is an original region, The second region is a reshaped region that uses a luma mapping with chroma scaling (LMCS) method to map luma samples to specific values. method.

[0486] 16. The method of clause 15, The LMCS method uses a piecewise linear model to map the luma samples to the specific values. method.

[0487] 17. Any of the methods of clauses 1 to 16, performing the transformation includes generating the coded representation from the current block. method.

[0488] 18. Any of the methods of clauses 1 to 16, performing the transformation includes generating the current block from the coded representation. method.

[0489] 19. An apparatus in a video system having a processor and non-transitory memory storing instructions, comprising: The instructions, when executed by the processor, cause the processor to perform any one of the methods of clauses 1 to 18. Device.

[0490] 20. A computer program product stored on a non-transitory computer-readable medium, comprising: A computer program product comprising a program code for performing the method of any one of clauses 1 to 18.

[0491] The eighth set of clauses describes particular features and aspects of the disclosed techniques listed in the previous section, including, for example, Examples 20-27.

[0492] 1. A method of video processing, comprising: determining a coding mode to be enabled for conversion between a current video block of a video and a coded representation of the video; performing the conversion using a block-based delta pulse code modulation (BDPCM) mode; having In the coding mode, the current video block is constructed based on samples in a first domain and a second domain, and / or a chroma residual is scaled depending on the luma. method.

[0493] 2. The method of clause 1, a prediction block of the current video block is generated in the first region. method.

[0494] 3. The method of clause 1, a residual block of the current video block represented in the coded representation in a first domain; method.

[0495] 4. The method of clause 1, a prediction block of the current video block is generated in the second domain. method.

[0496] 5. The method of clause 1, a residual block of the current video block represented in the coded representation in the second domain. method.

[0497] 6. The method according to clause 4 or 5, a first constituent block of the current video block is obtained based on a sum of the residual block and the predictive block in the second region; the first configuration block is utilized to convert between a subsequent video block and the coded representation of the video. method.

[0498] 7. The method according to clause 4 or 5, a final constituent block of the current video block is obtained based on a backward reshaping applied to a first constituent block to transform the first constituent block from the second domain to the first domain; method.

[0499] 8. A method of video processing comprising: For conversion between a current video block of a video and a coded representation of the video, determining that the current video block is coded using a block-based delta pulse code modulation (BDPCM) mode; performing said conversion by disabling a coding mode upon said determination; having when the coding mode is applied to a video block, the video block is constructed based on a first region and a second region, and / or a chroma residual is scaled depending on the luma. method.

[0500] 9. The method of clause 8, If the current video block is coded in the BDPCM mode, the coding mode is disabled. method.

[0501] 10. A method of video processing comprising: performing a conversion between a first video block of a video and a coded representation of the video, the conversion of the first video block using a first coding mode and a block-based delta pulse code modulation (BDPCM) mode; performing a conversion between a second video block of the video and a coded representation of the video; having the second video block is coded without using the BDPCM mode, and the transform of the second video block uses the first coding mode; when the first coding mode is applied to a video block, the video block is constructed based on a first region and a second region and / or a chroma residual is scaled depending on luma; the first coding mode is applied differently to the first video block and the second video block. method.

[0502] 11. The method of clause 10, the first coding mode applied to the first video block differs from the first coding mode applied to the second video block by disabling use of forward reshaping and backward reshaping used to transform samples between the first domain and the second domain; method.

[0503] 12. The method of clause 10, the first coding mode applied to the first video block differs from the first coding mode applied to the second video block by using different forward reshaping and / or different backward reshaping used to transform samples between the first domain and the second domain; method.

[0504] 13. A method of video processing comprising: determining a coding mode to be enabled for conversion between a current video block of a video and a coded representation of the video; performing the transform using a transform skip mode in which a transform for a prediction residual is skipped in coding the current video block; having In the coding mode, the current video block is constructed based on samples in a first domain and a second domain, and / or a chroma residual is scaled depending on the luma. method.

[0505] 14. The method of clause 13, a prediction block of the current video block is generated in the first region. method.

[0506] 15. The method of clause 13, a residual block of the current video block represented in the coded representation in a first domain; method.

[0507] 16. The method of clause 13, a prediction block of the current video block is generated in the second domain. method.

[0508] 17. A method according to clause 13, a residual block of the current video block represented in the coded representation in the second domain. method.

[0509] 18. The method of clause 16 or 17, a first constituent block of the current video block is obtained based on a sum of the residual block and the predictive block in the second region; the first configuration block is utilized to convert between a subsequent video block and the coded representation of the video. method.

[0510] 19. The method of clause 16 or 17, a final constituent block of the current video block is obtained based on a backward reshaping applied to a first constituent block to transform the first constituent block from the second domain to the first domain; method.

[0511] 20. A method of video processing comprising the steps of: determining, for a transformation between a current video block of a video and a coded representation of the video, that the current video block is coded in a transform skip mode in which a transform on a prediction residual is skipped in coding the current video block; performing said conversion by disabling a coding mode upon said determination; having when the coding mode is applied to a video block, the video block is constructed based on a first region and a second region, and / or a chroma residual is scaled depending on the luma. method.

[0512] 21. The method of clause 20, If the current video block is coded in the transform skip mode, the coding mode is disabled. method.

[0513] 22. A method of video processing comprising the steps of: performing a transform between a first video block of a video and a coded representation of the video, the transform of the first video block using a first coding mode and a transform skip mode in which a transform on a prediction residual is skipped in coding the current video block; performing a conversion between a second video block of the video and a coded representation of the video; having the second video block is coded without using the transform skip mode, and the transform of the second video block uses the first coding mode; when the first coding mode is applied to a video block, the video block is constructed based on a first region and a second region and / or a chroma residual is scaled depending on luma; the first coding mode is applied differently to the first video block and the second video block. method.

[0514] 23. The method of clause 22, the first coding mode applied to the first video block differs from the first coding mode applied to the second video block by disabling use of forward reshaping and backward reshaping used to transform samples between the first domain and the second domain; method.

[0515] 24. The method of clause 22, the first coding mode applied to the first video block differs from the first coding mode applied to the second video block by using different forward reshaping and / or different backward reshaping used to transform samples between the first domain and the second domain; method.

[0516] 25. A method of video processing comprising the steps of: determining a coding mode to be enabled for conversion between a current video block of a video and a coded representation of the video; performing the transform using an intrapulse code modulation mode in which the current video block is coded without applying a transform and transform domain quantization; having In the coding mode, the current video block is constructed based on samples in a first domain and a second domain, and / or a chroma residual is scaled depending on the luma. method.

[0517] 26. A method according to clause 25, a prediction block of the current video block is generated in the first region. method.

[0518] 27. A method according to clause 25, a residual block of the current video block represented in the coded representation in a first domain; method.

[0519] 28. A method according to clause 25, a prediction block of the current video block is generated in the second domain. method.

[0520] 29. A method according to clause 25, a residual block of the current video block represented in the coded representation in the second domain. method.

[0521] 30. The method of clause 28 or 29, a first constituent block of the current video block is obtained based on a sum of the residual block and the predictive block in the second region; the first configuration block is utilized to convert between a subsequent video block and the coded representation of the video. method.

[0522] 31. The method of clause 28 or 29, a final constituent block of the current video block is obtained based on a backward reshaping applied to a first constituent block to transform the first constituent block from the second domain to the first domain; method.

[0523] 32. A method of video processing comprising: determining that the current video block is coded in an intrapulse code modulation mode, in which the current video block is coded without applying a transform and transform domain quantization for a conversion between a current video block of a video and a coded representation of the video; performing said conversion by disabling a coding mode upon said determination; having when the coding mode is applied to a video block, the video block is constructed based on a first region and a second region, and / or a chroma residual is scaled depending on the luma. method.

[0524] 33. The method of clause 32, If the current video block is coded in the intrapulse code modulation mode, the coding mode is disabled. method.

[0525] 34. A method of video processing comprising the steps of: performing a conversion between a first video block of a video and a coded representation of the video, the conversion of the first video block using a first coding mode and an intrapulse code modulation mode in which the current video block is coded without applying a transform and transform domain quantization; performing a conversion between a second video block of the video and a coded representation of the video; having the second video block is coded without using the intrapulse code modulation mode, and the transforming of the second video block uses the first coding mode; when the first coding mode is applied to a video block, the video block is constructed based on a first region and a second region and / or a chroma residual is scaled depending on luma; the first coding mode is applied differently to the first video block and the second video block. method.

[0526] 35. A method according to clause 34, the first coding mode applied to the first video block differs from the first coding mode applied to the second video block by disabling use of forward reshaping and backward reshaping used to transform samples between the first domain and the second domain; method.

[0527] 36. The method of clause 34, the first coding mode applied to the first video block differs from the first coding mode applied to the second video block by using different forward reshaping and / or different backward reshaping used to transform samples between the first domain and the second domain; method.

[0528] 37. A method of video processing comprising the steps of: determining a coding mode to be enabled for conversion between a current video block of a video and a coded representation of the video; performing the transform using an improved transquant-bypass mode in which the current video block is losslessly coded without transform and quantization; having In the coding mode, the current video block is constructed based on samples in a first domain and a second domain, and / or a chroma residual is scaled depending on the luma. method.

[0529] 38. The method of clause 37, a prediction block of the current video block is generated in the first region. method.

[0530] 39. The method of clause 37, a residual block of the current video block represented in the coded representation in a first domain; method.

[0531] 40. A method according to clause 37, a prediction block of the current video block is generated in the second domain. method.

[0532] 41. A method according to clause 37, a residual block of the current video block represented in the coded representation in the second domain. method.

[0533] 42. The method of clause 40 or 41, a first constituent block of the current video block is obtained based on a sum of the residual block and the predictive block in the second region; the first configuration block is utilized to convert between a subsequent video block and the coded representation of the video. method.

[0534] 43. The method of clause 40 or 41, a final constituent block of the current video block is obtained based on a backward reshaping applied to a first constituent block to transform the first constituent block from the second domain to the first domain; method.

[0535] 44. A method of video processing comprising the steps of: determining that the current video block is to be coded in a transquant-bypass mode in which the current video block is losslessly coded without applying transform and quantization for a conversion between a current video block of a video and a coded representation of the video; performing said conversion by disabling a coding mode upon said determination; having when the coding mode is applied to a video block, the video block is constructed based on a first region and a second region, and / or a chroma residual is scaled depending on the luma. method.

[0536] 45. A method according to clause 44, If the current video block is coded in the transquant-bypass mode, the coding mode is disabled. method.

[0537] 46. ​​A method of video processing comprising the steps of: performing a transformation between a first video block of a video and a coded representation of the video, the transformation of the first video block using a first coding mode and a transquant-bypass mode in which the current video block is losslessly coded without transform and quantization; performing a conversion between a second video block of the video and a coded representation of the video; having the second video block is coded without using the transquant-bypass mode, and the transform of the second video block uses the first coding mode; when the first coding mode is applied to a video block, the video block is constructed based on a first region and a second region and / or a chroma residual is scaled depending on luma; the first coding mode is applied differently to the first video block and the second video block. method.

[0538] 47. A method according to clause 46, the first coding mode applied to the first video block differs from the first coding mode applied to the second video block by disabling use of forward reshaping and backward reshaping used to transform samples between the first domain and the second domain; method.

[0539] 48. A method according to clause 46, the first coding mode applied to the first video block differs from the first coding mode applied to the second video block by using different forward reshaping and / or different backward reshaping used to transform samples between the first domain and the second domain; method.

[0540] 49. Any one of the methods of clauses 1 to 48, the transformation is performed based on color components of the current video block. method.

[0541] 50. A method according to clause 49, the color component is a luma component; method.

[0542] 51. Any of the methods of clauses 1 to 50, the first region is an original region, The second region is a reshaped region that uses a luma mapping with chroma scaling (LMCS) method to map luma samples to specific values. method.

[0543] 52. The method of clause 51, The LMCS method uses a piecewise linear model to map the luma samples to the specific values. method.

[0544] 53. Any of the methods of clauses 1 to 52, performing the transformation includes generating the coded representation from the current block. method.

[0545] 54. Any of the methods of clauses 1 to 52, performing the transformation includes generating the current block from the coded representation. method.

[0546] 55. An apparatus in a video system having a processor and non-transitory memory storing instructions, comprising: The instructions, when executed by the processor, cause the processor to perform any one of the methods of clauses 1 to 54. Device.

[0547] 56. A computer program product stored on a non-transitory computer-readable medium, comprising: A computer program product comprising a program code for performing the method of any one of clauses 1 to 54.

[0548] The ninth set of clauses describes particular features and aspects of the disclosed techniques listed in the previous section, including, for example, Examples 30-34 and 41.

[0549] 1. A method of video processing, comprising: performing a conversion between a current video block of a video and a coded representation of the video; the transform uses a coding mode in which the current video block is constructed based on a first region and a second region and / or a chroma residual is scaled depending on luma; The information used for the coding mode is signaled in a parameter set that is different from a sequence parameter set (SPS), a video parameter set (VPS), a picture parameter set (PPS), or an adaptation parameter set (APS) that is used to carry adaptive loop filtering (ALF) parameters. method.

[0550] 2. The method of clause 1, The parameter set is shared across pictures. method.

[0551] 3. The method of clause 1, the parameter set includes one or more syntax elements including at least one of an identifier of the parameter set or a flag indicating the presence of extension data for the parameter set; method.

[0552] 4. The method of clause 1, the parameter set is specific to a tile group within a picture; method.

[0553] 5. A method of video processing comprising: performing a conversion between a current video block of a video and a coded representation of the video; the transform uses a coding mode in which the current video block is constructed based on a first region and a second region and / or a chroma residual is scaled depending on luma; The information used for the coding mode is signaled in an adaptation parameter set (APS) together with adaptive loop filtering (ALF) information; The information used for the coding mode and the ALF information are included in one NAL unit. method.

[0554] 6. The method of clause 5, The APS identifier is signaled in a tile group header. method.

[0555] 7. A method of video processing comprising: performing a conversion between a current video block of a video region of a video and a coded representation of the video; the transform uses a coding mode in which the current video block is constructed based on a first region and a second region and / or a chroma residual is scaled depending on luma; The information used for the coding mode is signaled in a first type of adaptation parameter set (APS) that is different from a second type of APS used to signal adaptive loop filtering (ALF) information. method.

[0556] 8. The method of clause 7, The identifier of the second type APS is signaled at the level of the video region. method.

[0557] 9. The method of clause 7, an identifier of the first type APS is signaled at the level of the video region; method.

[0558] 10. The method of clause 7, The first type of APS included in the coded representation includes the second type of APS including the ALF information in a conformance bitstream. method.

[0559] 11. The method of clause 7, the second type APS included in the coded representation includes the first type APS including the information used for the coding mode in a conformance bitstream; method.

[0560] 12. A method according to clause 7, The first type APS and the second type APS are associated with different identifiers. method.

[0561] 13. A method according to clause 12, The second type of APS has an identifier equal to 2N, where N is an integer. method.

[0562] 14. The method of clause 13, said first type APS having an identifier equal to 2N+1, where N is an integer; method.

[0563] 1. A method of video processing, comprising: performing a conversion between a current video block of a video region of a video and a coded representation of the video; the transform uses a coding mode in which the current video block is constructed based on a first region and a second region and / or a chroma residual is scaled depending on luma; the video region is not allowed to reference a parameter set or an adaptation parameter set that is signaled prior to a particular type of data structure used to process the video; the particular type of data structure is signaled before the video region; method.

[0564] 16. The method of clause 15, The data structure includes at least one of a network abstraction layer (NAL) unit, a tile group, a sequence parameter set (SPS), a picture parameter set (PPS), an access unit delimiter NAL (AUD), an end of bitstream NAL (EoB), an end of sequence NAL (NAL), an instantaneous decoding refresh (IDR) NAL, a clean random access (CRA) NAL, an intra random access point (IRAP) access unit, an I-tile group, a picture, or a slice; method.

[0565] 17. A method of video processing comprising: performing a conversion between a current video block of a video and a coded representation of the video; the transform uses a coding mode in which the current video block is constructed based on a first region and a second region and / or a chroma residual is scaled depending on luma; a parameter set syntax element containing parameters used to process the video has a predefined value in a conformance bitstream; method.

[0566] 18. The method of clause 17, The predefined values ​​are 0 and 1. method.

[0567] 19. The method of clause 17, The predefined values ​​are 0 and 7. method.

[0568] 20. Any of the methods of clauses 1 to 19, the video region comprises at least one of a tile group, a picture, a slice, or a tile; method.

[0569] 21. Any of the methods of clauses 1 to 20, the first region is an original region, The second region is a reshaped region that uses a luma mapping with chroma scaling (LMCS) method to map luma samples to specific values. method.

[0570] 22. The method of clause 21, The LMCS method uses a piecewise linear model to map the luma samples to the specific values. method.

[0571] 23. Any of the methods of clauses 1 to 22, performing the transformation includes generating the coded representation from the current block. method.

[0572] 24. Any of the methods of clauses 1 to 22, performing the transformation includes generating the current block from the coded representation. method.

[0573] 25. An apparatus in a video system having a processor and non-transitory memory storing instructions, comprising: The instructions, when executed by the processor, cause the processor to perform any one of the methods of clauses 1 to 24. Device.

[0574] 26. A computer program product stored on a non-transitory computer-readable medium, comprising: A computer program product comprising a program code for performing the method of any one of clauses 1 to 24.

[0575] From the foregoing, it will be appreciated that, although specific embodiments of the presently disclosed technology have been described herein for purposes of illustration, various modifications may be made therein without departing from the scope of the invention. Accordingly, the presently disclosed technology is not to be limited, except as by the appended claims.

[0576] Implementations of the subject matter and functional operations described herein can be implemented in various systems or digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed herein and their structural equivalents, or in one or more combinations thereof. Implementations of the subject matter described herein can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded in a tangible and non-transitory computer-readable medium for execution by or for controlling the operation of a data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage carrier, a memory device, a composition that provides a machine-readable propagated signal, or one or more combinations thereof. The term "data processing unit" or "data processing apparatus" encompasses all apparatus, devices, and machines that process data, including, by way of example, a programmable processor, a computer, or multiple processors or computers. An apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, such as code that constitutes a processor firmware, a protocol stack, a database management system, an operating system, or one or more combinations thereof.

[0577] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a single file dedicated to the program in question, or in multiple coordinated files (e.g., a file storing one or more modules, subprograms, or portions of code), in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document). A computer program can be deployed to be executed on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communication network.

[0578] The processes and logic flows described herein may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and an apparatus may be implemented as, special purpose logic circuitry, such as a Field Programmable Gate Array (FPGA) or an Application-Specific Integrated Circuit (ASIC).

[0579] Processors suitable for executing computer programs include, by way of example, both general purpose and special purpose microprocessors, as well as any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory, or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include one or more mass storage devices, e.g., magnetic, optical magnetic disks, or optical disks, for storing data, or be operatively coupled to receive data from and / or transfer data to such one or more mass storage devices. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all types of non-volatile memory, media, and memory devices, including, by way of example, semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices. The processor and memory may be augmented by, or incorporated in, dedicated logic circuitry.

[0580] This specification, together with the drawings, are intended to be considered merely illustrative, where illustrative means exemplary. As used herein, the use of "or" is intended to include "and / or" unless the context clearly indicates otherwise.

[0581] Although the specification contains numerous details, they should not be construed as limitations on the scope of any invention or what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of a particular invention. Certain features described in this specification in the context of separate embodiments may also be implemented in combination with a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as operating in a particular combination, and may even be initially claimed as such, one or more features from a claimed combination may in some cases be deleted from the combination, and the claimed combination may be directed to a subcombination or a variation of the subcombination.

[0582] Similarly, although operations may be depicted in the figures in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown, or in a sequential order, or that all of the depicted operations be performed, to achieve desired results. Further, the separation of various system components in the embodiments described herein should not be understood as requiring such separation in all embodiments.

[0583] Only a few implementations and examples have been described; other implementations, enhancements and variations may be made based on what is described and illustrated herein.

[0584] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application is based on International Patent Application No. PCT / CN2020 / 074136, filed on February 1, 2020, which claims priority to and the benefit of International Patent Application No. PCT / CN2019 / 074437, filed on February 1, 2019, and Japanese Patent Application No. 2021-544569, which entered the national phase on July 30, 2021. The entire disclosures of the above applications are incorporated by reference as part of the disclosure of this application.

Claims

1. 1. A method for processing video data, comprising the steps of: performing a conversion between a current video block of a video region of a video and a bitstream of said video; The transformation may be performed such that the current video block is 1) a forward mapping process for the luma component, in which samples of the luma component of the current video block are transformed from an original domain to a reshaped domain; or 2) a backward mapping process that is the inverse of the forward mapping process to transform the samples of the luma component in the reshaped domain back to the original domain; or 3) a scaling process in which residual samples of chroma components of the current video block are scaled before being used to reconstruct the chroma components of the current video block. Using a coding tool configured based on at least one of an adaptation parameter set is included in the bitstream, the adaptation parameter set including parameter information of the coding tool and a first identifier of the adaptation parameter set; The first identifier of the adaptation parameter set is fixed length coded and has a range between 0 and M, where M is 2. Ｋ −1, the parameter information includes at least one of: reshaper model information used in the forward mapping process to map the samples of the luma component to specific values; or chroma residual scaling coefficients used in the scaling process to derive scaled residual samples of the chroma components. method.

2. the adaptation parameter sets may be shared between pictures of the video; The method of claim 1.

3. The reshaper model is a piecewise linear model, a scale coefficient of the piecewise linear model is determined based on a first variable determined based on the parameter information included in the adaptive parameter set and a second variable determined based on a bit depth; scaling parameters of the scaling process are derived based on reconstructed luma samples within the reshaped region of a picture to which the current video block belongs and the chroma residual scaling factors. The method according to claim 1 or 2.

4. the adaptation parameter set for the coding tool and the other adaptation parameter set for adaptive loop filter information share the same network abstraction layer unit type.

4. The method according to any one of claims 1 to 3.

5. The conversion is determining a first prediction block in the original domain from at least one reference picture using unrefined motion information for the luma component of the current video block; applying a motion information refinement process to the unrefined motion information based on at least the first prediction block to obtain motion offset information; generating a second predictive block from samples in the at least one reference picture using the motion offset information; transforming the second predicted block from the original domain to the reshaped domain during the forward mapping process, in which samples of the second predicted block are mapped to the specific values; generating reconstructed samples of the luma component of the current video block based on the second predictive block in the reshaped domain; transforming the reconstructed samples of the luma component of the current video block from the reshaped domain to the original domain during the backward mapping process; applying a filtering process to the transformed reconstructed samples in the original domain; having 5. The method according to any one of claims 1 to 4.

6. by applying the scaling process, the residual samples of the chroma components of the current video block are derived based on reconstructed samples within the reshaped region of a luma component of a picture to which the current video block belongs. The method according to claim 5.

7. whether to apply the scaling process to the residual samples of the chroma components of the current video block is based on a block dimension of the current video block.

7. The method according to any one of claims 1 to 6.

8. and in response to the chroma components of the current video block being coded with all-zero coefficients, the scaling process is skipped. The method according to claim 7.

9. The conversion is determining that a particular non-inter coding mode is to be used for the luma component of the current video block; performing the transformation by using the particular non-inter coding mode on the reshaped region for the luma component to generate intermediate luma reconstructed samples of the luma component; having the backward mapping process is applied to the intermediate luma reconstructed samples to generate final luma reconstructed samples in the original domain, and a determination is made whether to apply the scaling process to the residual samples of the chroma components of the current video block.

9. The method according to any one of claims 1 to 8.

10. The specific non-inter-coding mode comprises a palette mode, an intra block copy mode, or a block-based delta pulse code modulation mode.

10. The method of claim 9.

11. the transforming includes encoding the current video block into the bitstream.

11. The method according to any one of claims 1 to 10.

12. the converting includes decoding the current video block from the bitstream. A method according to any one of claims 1 to 10.

13. a processor and a non-transitory memory having instructions; The instructions, when executed by the processor, cause the processor to: performing a conversion between a current video block of a video region of a video and a bitstream of said video; The transformation may be performed such that the current video block is 1) a forward mapping process for the luma component, in which samples of the luma component of the current video block are transformed from an original domain to a reshaped domain; or 2) a backward mapping process that is the inverse of the forward mapping process to transform the samples of the luma component in the reshaped domain back to the original domain; or 3) a scaling process in which residual samples of chroma components of the current video block are scaled before being used to reconstruct the chroma components of the current video block. Using a coding tool configured based on at least one of an adaptation parameter set is included in the bitstream, the adaptation parameter set including parameter information of the coding tool and a first identifier of the adaptation parameter set; The first identifier of the adaptation parameter set is fixed length coded and has a range between 0 and M, where M is 2. Ｋ −1, the parameter information includes at least one of: reshaper model information used in the forward mapping process to map the samples of the luma component to specific values; or chroma residual scaling coefficients used in the scaling process to derive scaled residual samples of the chroma components. Device.

14. storing instructions for causing a processor to perform a conversion between a current video block of a video region of a video and a bitstream of said video; The transformation may be performed such that the current video block is 1) a forward mapping process for the luma component, in which samples of the luma component of the current video block are transformed from an original domain to a reshaped domain; or 2) a backward mapping process that is the inverse of the forward mapping process to transform the samples of the luma component in the reshaped domain back to the original domain; or 3) a scaling process in which residual samples of chroma components of the current video block are scaled before being used to reconstruct the chroma components of the current video block. Using a coding tool configured based on at least one of an adaptation parameter set is included in the bitstream, the adaptation parameter set including parameter information of the coding tool and a first identifier of the adaptation parameter set; The first identifier of the adaptation parameter set is fixed length coded and has a range between 0 and M, where M is 2. Ｋ −1, the parameter information includes at least one of: reshaper model information used in the forward mapping process to map the samples of the luma component to specific values; or chroma residual scaling coefficients used in the scaling process to derive scaled residual samples of the chroma components. A non-transitory computer-readable storage medium.

15. A method for storing a bitstream of video, comprising the steps of: generating the bitstream based on a current video block of a video region of the video; storing the bitstream on a non-transitory computer-readable recording medium; A coding tool is used during the generating step, and in the coding tool, the current video block is 1) a forward mapping process for the luma component, in which samples of the luma component of the current video block are transformed from an original domain to a reshaped domain; or 2) a backward mapping process that is the inverse of the forward mapping process to transform the samples of the luma component in the reshaped domain back to the original domain; or 3) a scaling process in which residual samples of chroma components of the current video block are scaled before being used to reconstruct the chroma components of the current video block. Based on at least one of the following: an adaptation parameter set is included in the bitstream, the adaptation parameter set including parameter information of the coding tool and a first identifier of the adaptation parameter set; The first identifier of the adaptation parameter set is fixed length coded and has a range between 0 and M, where M is 2. Ｋ −1, the parameter information includes at least one of: reshaper model information used in the forward mapping process to map the samples of the luma component to specific values; or chroma residual scaling coefficients used in the scaling process to derive scaled residual samples of the chroma components. method.

Images