Method for Video Decoding, Computing Device, Non-Transitory Computer-Readable Storage Medium, Computer Program, and Method for Storing a Bitstream

Abstract

To provide methods and devices for rectifying a forward mapping coding bit length issue introduced by luma mapping with chroma scaling (LMCS).SOLUTION: In a method, a plurality of prediction samples, in a mapped domain, of luma component of a coding unit (CU) that is coded by an inter mode or combined inter and intra prediction mode under LMCS framework are obtained; a plurality of residual samples, in the mapped domain, of the luma component of the CU are received from a bitstream; the plurality of prediction samples in the mapped domain are added to the plurality of residual samples in the mapped domain, so that a plurality of reconstructed samples of the luma component are obtained in the mapped domain; and the plurality of reconstructed samples of the luma component are converted from the mapped domain into an original domain based on a pre-defined plurality of inverse mapping scaling factors.SELECTED DRAWING: Figure 23

Description

Technical Field

[0001] Cross - reference to Related Applications This application claims the benefit of U.S. Provisional Patent Application No. 63 / 043,569, filed Jun. 24, 2020 The entire disclosure of the foregoing application is incorporated herein by reference in its entirety for all purposes

[0002] The present invention generally relates to video encoding and compression. More particularly, the present invention relates to systems and methods for performing video encoding using prediction dependent residual scaling (PDRS) on an encoding unit nt residual scaling, PDRS) to perform video encoding

Background Art

[0003] This section provides background information related to the present disclosure. The information included in this section is not necessarily to be construed as prior art

[0004] To compress video data, any of a variety of video encoding techniques can be used Video encoding can be performed according to one or more video encoding standards Some exemplary video encoding standards include Versatile Video Coding (VVC), joint exploration model (JEM) encoding, High Efficiency Video Coding (H.26 5 / HEVC), Advanced Video Coding (H.264 / AVC), and Moving Picture Experts Group (MPEG) encoding 5 / HEVC), Advanced Video Coding (H.264 / AVC), and Moving Picture Experts Group (MPEG) encoding

[0005] ​​​​Video encoding generally utilizes prediction methods (e.g., inter prediction, intra prediction, etc.) that exploit the redundancy unique to the video image or sequence. One of the objectives of video encoding techniques is to compress the video data into a form that uses a lower bit rate while avoiding or minimizing degradation of video quality. SUMMARY OF THE INVENTION

[0006] This section provides a general overview of the present disclosure and is not an exhaustive disclosure of its full scope or all of its features. MEANS FOR SOLVING THE PROBLEM

[0007] According to a first aspect of the present application, under the framework of luma mapping with chroma scaling (LMCS), , knot a plurality of prediction samples in the mapped domain of the luminance component of an encoded unit (CU) encoded by a combined inter and intra prediction (CIIP) mode are obtained, and a plurality of residual samples is obtained in the mapped domain of the luminance component of the CU are obtained. The plurality of prediction samples in the mapped domain are added to the plurality of residual samples in the mapped domain to obtain a plurality of reconstructed samples in the mapped domain of the luminance component. The plurality of reconstructed samples of the luminance component are converted from the mapped domain to the original domain based on a plurality of predefined inverse mapping scaling coefficients. According to a second aspect of the present application, a method for video decoding is provided. This method involves obtaining a plurality of prediction samples in the mapped domain of the luminance component of an encoded unit (CU) encoded in a combined inter-intra prediction (CIIP) mode under the framework of luminance mapping with chroma scaling (LMCS), obtaining a plurality of residual samples in the mapped domain of the luminance component of the CU, adding the plurality of prediction samples in the mapped domain to the plurality of residual samples in the mapped domain to obtain a plurality of reconstructed samples in the mapped domain of the luminance component, and converting the plurality of reconstructed samples of the luminance component from the mapped domain to the original domain based on a plurality of predefined inverse mapping scaling coefficients. In one embodiment, obtaining a plurality of prediction samples in the mapped domain of the luminance component of the CU includes deriving a plurality of inter-prediction samples in the original domain of the luminance component of the CU from a temporal reference picture of the CU, and converting the plurality of inter-prediction samples of the luminance component from the original domain to the mapped domain based on a predefined encoding bit depth and a plurality of predefined forward mapping scaling coefficients within a predefined forward mapping accuracy. In one embodiment, converting the plurality of inter-prediction samples of the luminance component from the original domain to the mapped domain based on a predefined encoding bit depth and a plurality of predefined forward mapping scaling coefficients within a predefined forward mapping accuracy includes converting the plurality of inter-prediction samples of the luminance component from the original domain to the mapped domain without a clipping operation using the plurality of predefined forward mapping scaling coefficients. In one embodiment, the predefined forward mapping accuracy is 11 bits.

[0008] According to the 3 aspect of the present application, a computing device includes one or more processors, a memory, and a plurality of programs stored in the memory. When the programs are executed by one or more processors, the computing device is caused to perform the operations as described in the first aspect or the second aspect of the present application as described above.

[0009] According to the 4 aspect of the present application, a non-transitory computer-readable storage medium stores a plurality of programs that are executed by a computing device having one or more processors. When the programs are executed by one or more processors, the computing device is caused to perform the operations as described in the first aspect or the second aspect of the present application as described above.

[0010] Hereinafter, a set of exemplary and non-limiting embodiments of the present disclosure will be described in conjunction with the accompanying drawings . Modifications of structure, method, or function may be implemented by those skilled in the art based on the examples presented herein, and all such modifications are included within the scope of the present disclosure. If there are no contradictions , the teachings of different embodiments can be combined with each other, but it is not necessary to combine them.

Brief Description of the Drawings

[0011] [Figure 1] FIG. is a block diagram showing an exemplary block-based hybrid video encoder that can be used with many video coding standards. [Figure 2] FIG. is a block diagram showing an exemplary video decoder that can be used with many video coding standards. [Figure 3] FIG. is a diagram of block partitioning in a multi-type tree structure that can be used in conjunction with many video coding standards. [Figure 4] It is a flowchart showing the decoding process to which LMCS is applied. [Figure 5] It is a diagram of bi - directional optical flow (BDOF) processing. [Figure 6] It is a flowchart showing the workflow of chroma residual scaling in LMCS when all of decoder - side motion vector refinement (DMVR), BDOF, and CIIP are enabled. [Figure 7] It is a flowchart showing the steps of the prediction - dependent residual scaling (PDRS) procedure. [Figure 8] It is a flowchart showing the workflow of the decoding process when the PDRS procedure is applied in the LMCS process. [Figure 9] It is a diagram showing the residual mapping error caused by using only prediction samples to derive the scaling factor. [Figure 10] It is a flowchart showing the steps of the chroma sample reconstruction procedure. [Figure 11] It is a flowchart showing the steps of the second chroma sample reconstruction procedure. [Figure 12] It is a flowchart showing the workflow of the LMCS decoding process in an example of the second chroma sample reconstruction procedure where DMVR, BDOF, and CIIP are not applied to generate luminance prediction samples for chroma scaling. [Figure 13] It is a flowchart showing the workflow of the LMCS decoding process in a second example of the second chroma sample reconstruction procedure where an initial single prediction signal is applied to generate luminance prediction samples for chroma scaling. [Figure 14] It is a flowchart showing the steps of the chroma residual sample reconstruction procedure. [Figure 15] It is a flowchart showing the workflow of the LMCS decoding process in one or more embodiments of the chroma residual sample reconstruction procedure. [Figure 16] A flowchart showing the workflow of LMCS decoding processing in another embodiment of the chroma residual sample reconstruction procedure. [Figure 17] A flowchart showing the steps of a second chroma residual sample reconstruction procedure. [Figure 18] A flowchart showing the steps of the non-clipping chroma residual scaling coefficient derivation procedure. [Figure 19] A diagram of the area involved in an example of the non-clipping chroma residual scaling coefficient derivation procedure. [Figure 20] A flowchart showing the steps of the non-clipping chroma sample decoding procedure. [Figure 21] A flowchart showing the steps of one or more embodiments of the non-clipping chroma sample decoding procedure. [Figure 22] A flowchart showing the steps of one or more embodiments of the non-clipping chroma sample decoding procedure. [Figure 23] A flowchart showing the steps of the first aspect of the present disclosure.

Figure 24

Figure 25

Figure 26

Figure 27

DETAILED DESCRIPTION OF THE INVENTION

[0012] The terms used in the present disclosure are not intended to limit the present disclosure, but are for illustrative purposes of specific examples. Thus, the singular forms "a", "an", and "the" used in the present disclosure and the appended claims The term "the" also refers to the plural form unless the context clearly dictates otherwise. As used herein, the term "and / or" is to be understood to refer to any and all possible combinations of one or more of the associated listed items.

[0013] In this specification, terms such as "first", "second", "third", etc. may be used to describe various information, but it should be understood that the information should not be limited by these terms. These terms are only used to distinguish one category of information from another. For example, without departing from the scope of the present disclosure, the first information may be referred to as the second information, and vice versa. When used in this specification, the term "if" may, depending on the context, be understood to mean "when", "upon", or "in response to".

[0014] Throughout this specification, references to one or more "embodiments", "an embodiment", "another embodiment", etc. mean that one or more specific features, structures, or characteristics described in connection with the embodiment are included in at least one embodiment of the present disclosure. Thus, appearances of the phrases "in one embodiment" or "in an embodiment", "in another embodiment", etc. at various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the specific features, structures, or characteristics in one or more embodiments can be combined in any suitable manner.

[0015] The first version of the HEVC standard was completed in October 2013, which provides about 50% bitrate savings compared to the previous-generation video coding standard H.264 / MPEG AVC, or equivalent perceptual quality. The HEVC standard provides significant coding improvements over its predecessor, but there is evidence that excellent coding efficiency can be achieved by using additional coding tools on top of HEVC. Based on this, both VCEG and MPEG initiated a search for new coding technologies for future video coding standardization. To initiate important research on advanced technologies that have the potential to significantly improve coding efficiency, a Joint Video Exploration Team (JVET) was formed in October 2015 by ITU-T VECG and ISO / IEC MPEG. A reference software called the Joint Exploration Model (JEM) was maintained by the JVET by integrating some additional coding tools on top of the HEVC Test Model (HM). In October 2017, a Call for Proposals (CfP) regarding video compression with capabilities beyond HEVC was jointly solicited by ITU-T and ISO / IEC. In April 2018, 23 CfP responses were received and evaluated in 10 JVET meetings, demonstrating an increase in compression efficiency of about 40% over HEVC. Based on such evaluation results, the JVET newly launched a development project for the Versatile Video Coding (VVC), a next-generation video coding standard. In the same month, a reference software called the VVC Test Model (VTM) was established to demonstrate the reference implementation of the VVC standard. (Joint Video Exploration Team, JVET) was formed

[0016] ​​​​​​​​​​​​​​​

[0017] The prediction methods used in video coding typically perform spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to reduce or remove the redundancy inherent in video data and are generally associated with block-based video coding. Similar to HEVC, VVC is also constructed based on a block-based hybrid video coding framework. In block-based video coding, the input video signal is processed block by block. For each block, spatial prediction and / or temporal prediction can be performed. In newer video coding standards such as the current V

[0018] VC design, the blocks can be further divided based on a multi-type tree structure that includes not only quad-trees but also binary trees and / or ternary trees. This allows for better adaptation to changing local characteristics. Spatial prediction (also known as "intra prediction") uses pixels from samples of already encoded adjacent blocks (referred to as reference samples) within the same video picture / slice to predict the current block. Spatial prediction reduces the spatial redundancy inherent in the video signal. During decoding, the video bitstream is first entropy decoded in the entropy decoding unit. The coding mode and prediction information are sent to either the spatial prediction unit (when intra-coded) or the temporal prediction unit (when inter-coded) to form the predicted block. The residual transform coefficients are used to reconstruct the residual block.

[0019]

[0020] ​​​​​​​​​Therefore, it is sent to the inverse quantization unit and the inverse transform unit. Then, the prediction block and the residual block are added. The reconstructed block may further pass through in-loop filtering before being stored in the reference picture store. After that, the reconstructed video in the reference picture store is sent to drive the display device and is used to predict future video blocks.

[0021] In newer video coding standards such as the current VVC design, the coding tool of luminance mapping with chroma scaling (LMCS) can be applied before in-loop filtering. LMCS aims to adjust the dynamic range of the input signal to improve coding efficiency.

[0022] However, in the current design of LMCS, the mapping accuracy of the inter-prediction samples of the luminance component may exceed the dynamic range of the internal coding depth.

[0023] Conceptually, many video coding standards, including those described above in the background section, are similar. For example, substantially all video coding standards use block-based processing and share similar video coding block diagrams to achieve video compression.

[0024] FIG. 1 shows a block diagram of an exemplary block-based hybrid video encoder 100 that can be used with many video coding standards. In encoder 100, a video frame is divided into a plurality of video blocks for processing. For each given video block, a prediction is formed based on either an inter-prediction method or an intra-prediction method. For each, a prediction is formed based on either an inter-prediction technique or an intra-prediction technique. In inter prediction, one or more predictors are formed by motion estimation and motion compensation based on pixels from previously reconstructed frames. In intra prediction, a predictor is formed based on reconstructed pixels within the current frame. By mode decision, the best predictor can be selected to predict the current block. The prediction residual representing the difference between the current video block and its predictor is sent to the transform circuit 102. The transform coefficients are then sent from the transform circuit 102 to the quantization circuit 104 for entropy reduction. The quantized coefficients are then supplied to the entropy encoding circuit 106 to generate a compressed video bitstream. As shown in FIG. 1, video block

[0025] partitioning information, motion vectors, reference picture indices, and any prediction-related information 110 from the inter prediction circuit and / or intra prediction circuit 112 are also supplied via the entropy encoding circuit 106 and stored in the compressed video bitstream 11 4. In the encoder 100, decoder-related circuitry is also required to reconstruct pixels for prediction purposes. First, the prediction residual is reconstructed by the inverse quantization circuit 116 and the inverse transform circuit 118. This reconstructed prediction residual is combined with the block predictor 120 to generate the unfiltered reconstructed pixels of the current video block. Temporal prediction (also referred to as "inter prediction" or "motion compensated prediction") uses reconstructed pixels from previously encoded video pictures to predict the current video block. block. 4.

[0026] In the encoder 100, decoder-related circuitry is also required to reconstruct pixels for prediction purposes. First, the prediction residual is reconstructed by the inverse quantization circuit 116 and the inverse transform circuit 118. This reconstructed prediction residual is combined with the block predictor 120 to generate the unfiltered reconstructed pixels of the current video block. First, the prediction residual is reconstructed by the inverse quantization circuit 116 and the inverse transform circuit 118. This reconstructed prediction residual is combined with the block predictor 120 to generate the unfiltered reconstructed pixels of the current video block. This reconstructed prediction residual is combined with the block predictor 120 to generate the unfiltered reconstructed pixels of the current video block. This reconstructed prediction residual is combined with the block predictor 120 to generate the unfiltered reconstructed pixels of the current video block.

[0027] Temporal prediction (also referred to as "inter prediction" or "motion compensated prediction") uses reconstructed pixels from previously encoded video pictures to predict the current video block. It reduces the temporal redundancy inherent in the video signal. The temporal prediction of a given CU is usually signaled by one or more motion vectors (MVs) indicating the amount and direction of motion between the current CU and its temporal reference. Also, if multiple reference pictures are supported, one reference picture index is further transmitted, which is used to identify from which reference picture in the reference picture store the temporal prediction signal comes.

[0028] After spatial and / or temporal prediction is performed, the intra / inter mode decision circuit 121 in the encoder 100 selects the best prediction mode, for example, based on a rate-distortion optimization method. Next, the block predictor 120 is subtracted from the current video block. Then, the resulting prediction residual is decorrelated using the transform circuit 102 and the quantization circuit 104. The resulting quantized residual coefficients are inverse quantized by the inverse quantization circuit 116 and inverse transformed by the inverse transform circuit 118 to form a reconstructed residual, which is then added to the prediction block to form the reconstructed signal of the CU. The reconstructed CU is placed in the reference picture store of the picture buffer 117 and, before being used to encode future video blocks, further in-loop filtering 115 such as a deblocking filter, sample adaptive offset (SAO), and / or adaptive in-loop filter (ALF) can be applied to the reconstructed CU. To form the output video bitstream 114, encoding mode (inter or intra), prediction mode information, motion information ​​and quantization residual coefficients are all further compressed and sent to the entropy encoding unit 106 to be packetized to form a bitstream.

[0029] For example, deblocking filters are available in current versions of AVC, HEVC, and VVC. In HEVC, an additional in-loop filter called SAO (sample adaptive offset) is defined to further improve coding efficiency. In the current version of the VVC standard, yet another in-loop filter called ALF (adaptive loop filter) is being actively studied and is likely to be included in the final standard. These in-loop filter operations are optional. Performing these operations helps improve coding

[0030] efficiency and visual quality. They may also be turned off as a decision provided by the encoder 100 to reduce computational complexity. Intra prediction is usually based on non-filtered reconstructed pixels, while

[0031] inter prediction is based on filtered reconstructed pixels. Note that these filter options are turned on by the encoder 100.

[0032] Figure 2 is a block diagram showing an exemplary video decoder 200 that can be used with many video coding standards. This decoder 200 is similar to the reconstruction-related section present in the encoder 100 of FIG. 1. In decoder 200 (FIG. 2), the input video bitstream 201 is entropy decoded to derive quantized coefficient levels and prediction-related information. ​It is first decoded through the tropy decoder 202. Then, the quantized coefficient levels are processed through inverse quantization 204 and inverse transformation 206 to obtain the reconstructed prediction residual. The block predictor mechanism implemented in the intra / inter mode selector 212 performs either intra prediction 208 or motion compensation 210 based on the decoded prediction information. It is configured to do so. The set of unfiltered reconstructed pixels is obtained by using the adder 21 4 to sum the reconstructed prediction residual from the inverse transformation 206 and the prediction output generated by the block predictor mechanism.

[0033] The reconstructed block can further pass through the in-loop filter 209 before being stored in the picture buffer 213 that functions as a reference picture store. The reconstructed video in the picture buffer 213 can then be sent to drive the display device and can be used to predict future video blocks. In the situation where the in-loop filter 209 is turned on, a filtering operation is performed on these reconstructed pixels to derive the final reconstructed video output 222.

[0034] In video coding standards such as HEVC, the block may be divided based on a quadtree. In newer video coding standards such as the current VVC, more splitting methods are adopted to split one coding tree unit (CTU) into CUs and adapt to various local characteristics based on quadtree, binary tree, or ternary tree. The separation of CUs, prediction units (PUs), and transform units (TUs) is for most coding modes of the current VVC. ​ It does not exist in the LCU, and each CU is always used as the basic unit for both prediction and transformation without further division. However, in some specific coding modes such as the intra-subdivision coding mode, each CU can still contain multiple TUs. In the multi-type tree structure, one CTU is first divided by a quadtree structure. Next, each quadtree leaf node can be further divided by a binary tree structure and a ternary tree structure. It is always used. However, in some specific coding modes such as the intra-subdivision coding mode, each CU can still contain multiple TUs. In the multi- type tree structure, one CTU is first divided by a quadtree structure. Next, each quadtree leaf node can be further divided by a binary tree structure and a ternary tree structure. .

[0035] Figure 3 shows five splitting types currently adopted in the VVC, namely, quad-splitting 301, horizontal bi-splitting 302, vertical bi-splitting 303, horizontal tri-splitting 304, and vertical value splitting 305. In the situation where the multi-type tree structure is utilized, one CTU is first divided by a quadtree structure. Next, each quadtree leaf node can be further divided by a binary tree structure and a ternary tree structure. In the situation where the multi-type tree structure is utilized, one CTU is first divided by a quadtree structure. Next, each quadtree leaf node can be further divided by a binary tree structure and a ternary tree structure. In the situation where the multi-type tree structure is utilized, one CTU is first divided by a quadtree structure. Next, each quadtree leaf node can be further divided by a binary tree structure and a ternary tree structure. can be further divided.

[0036] Using one or more of the exemplary block splittings 301, 302, 303, 304, or 305 in FIG. 3, the configuration shown in FIG. 1 can be used to perform spatial prediction and / or temporal prediction. Spatial prediction (or "intra prediction") predicts the current video block using samples from encoded adjacent blocks (referred to as reference samples) within the same video picture / slice. Spatial prediction reduces the spatial redundancy inherent in the video signal. Using one or more of the exemplary block splittings 301, 302, 303, 304, or 305 in FIG. 3, the configuration shown in FIG. 1 can be used to perform spatial prediction and / or temporal prediction. Spatial prediction (or "intra prediction") predicts the current video block using samples from encoded adjacent blocks (referred to as reference samples) within the same video picture / slice. Spatial prediction reduces the spatial redundancy inherent in the video signal. Using one or more of the exemplary block splittings 301, 302, 303, 304, or 305 in FIG. 3, the configuration shown in FIG. 1 can be used to perform spatial prediction and / or temporal prediction. Spatial prediction (or "intra prediction") predicts the current video block using samples from encoded adjacent blocks (referred to as reference samples) within the same video picture / slice. Spatial prediction reduces the spatial redundancy inherent in the video signal. Using one or more of the exemplary block splittings 301, 302, 303, 304, or 305 in FIG. 3, the configuration shown in FIG. 1 can be used to perform spatial prediction and / or temporal prediction. Spatial prediction (or "intra prediction") predicts the current video block using samples from encoded adjacent blocks (referred to as reference samples) within the same video picture / slice. Spatial prediction reduces the spatial redundancy inherent in the video signal. Using one or more of the exemplary block splittings 301, 302, 303, 304, or 305 in FIG. 3, the configuration shown in FIG. 1 can be used to perform spatial prediction and / or temporal prediction. Spatial prediction (or "intra prediction") predicts the current video block using samples from encoded adjacent blocks (referred to as reference samples) within the same video picture / slice. Spatial prediction reduces the spatial redundancy inherent in the video signal. Using one or more of the exemplary block splittings 301, 302, 303, 304, or 305 in FIG. 3, the configuration shown in FIG. 1 can be used to perform spatial prediction and / or temporal prediction. Spatial prediction (or "intra prediction") predicts the current video block using samples from encoded adjacent blocks (referred to as reference samples) within the same video picture / slice. Spatial prediction reduces the spatial redundancy inherent in the video signal.

[0037] In new video coding standards such as the current VVC, a new coding tool, luminance mapping with chroma scaling (LMCS), has been added. Loop filters (e.g., Loop filters (e.g., , deblocking filter, SAO, and ALF) LMCS will be added as a tool for simplification.

[0038] In general, LMCS has two main modules. First, adaptive piecewise linear model and secondly, a luminance-dependent chroma residual scale It's a ring.

[0039] FIG. 4 shows the modified decoding process using LMCS. In FIG. 4, a particular block is Decoding module executed in the pinged domain, entropy decoding 401, inverse Quantization 402, inverse transform 403, luma intra prediction 404, and luma sample reconstruction 40 5 (i.e., the luminance prediction sample Y' pred and the luminance residual sample Y' res Add and and the reconstructed luminance sample Y' recon (generating certain other blocks) represents a decryption module that runs in the original (i.e., unmapped) domain. 4, the motion compensation prediction 409, the chroma intra prediction 412, and the chroma sample reconstruction 413 (i.e. That is, saturation prediction sample C pred and chroma residual sample C res and are added together to reconstruct Saturation sample C recon ), and in-loop filtering 407 (generating A further group of blocks includes , a forward mapping 410 and a reverse (or inverse) mapping 406 of the luma samples; and new operational models introduced by LMCS, including chroma residual scaling 411. Also, as shown in FIG. 4, a decoded picture buffer (DPB) 40 All reference pictures stored in 8 (for luminance) and 415 (for chroma) are in the original domain. They are mainly there.

[0040] The in-loop mapping of LMCS aims to adjust the dynamic range of the input signal to improve the coding efficiency. The in-loop mapping of luminance samples in existing LMCS designs is constructed based on two mapping functions, one forward mapping function FwdMap, and one corresponding inverse mapping function InvMap. The forward mapping function uses a piecewise linear model with 16 equally sized segments to signal from the encoder to the decoder. The inverse mapping function can be directly derived from the forward mapping function and thus does not need to be signaled. The parameters of the luminance mapping model are signaled at the slice level. To indicate whether the luminance mapping model should be signaled for the current slice, an existence flag is signaled first. If the luminance mapping model exists for the current slice, the corresponding piecewise linear model parameters are further signaled. Based on the piecewise linear model, the dynamic range of the input signal is divided into 16 segments of equal size in the original domain, and each segment is mapped to a corresponding segment. For a given segment in the original domain, its corresponding segment in the mapped domain can have the same or different sizes. The size of each segment in the mapped domain is the number of codewords of that segment (i.e., They are mainly there. The forward mapping function uses a piecewise linear model with 16 equally sized segments to signal from the encoder to the decoder. The inverse mapping function can be directly derived from the forward mapping function and thus does not need to be signaled. The inverse mapping function can be directly derived from the forward mapping function and thus does not need to be signaled.

[0041] The parameters of the luminance mapping model are signaled at the slice level. To indicate whether the luminance mapping model should be signaled for the current slice, an existence flag is signaled first. If the luminance mapping model exists for the current slice, the corresponding piecewise linear model parameters are further signaled. Based on the piecewise linear model, the dynamic range of the input signal is divided into 16 segments of equal size in the original domain, and each segment is mapped to a corresponding segment. For a given segment in the original domain, its corresponding segment in the mapped domain can have the same or different sizes. The size of each segment in the mapped domain is the number of codewords of that segment (i.e., , is indicated by the mapped sample values). For each segment in the original domain the number of codewords within the corresponding segment in the mapped domain Based on, the linear mapping parameters can be derived. For example, when the input is 10-bit depth, each of the 16 segments in the original domain has 64 pixel values and each of the segments in the mapped domain also has 64 codewords assigned to it, which indicates a simple one-to-one mapping (i.e., a mapping that does not change each sample value). The number of signaled codewords for each segment in the mapped domain is used to calculate the scaling factor and adjust the mapping function accordingly for that segment. Further, at the slice level, another LMCS control flag is signaled to enable / disable the LMCS of the slice.

[0042] For each segment, the corresponding piecewise linear model is defined as described in the box immediately following this paragraph. For the i-th segment (i = 0...15), the corresponding piecewise linear model is defined by two input pivot points InputPivot[i] and InputPivot[i + 1], and also, two output (mapped) pivot points MappedPivot[i] and Ma ppedPivot[i + 1]. Further, assuming a 10-bit input video the values of InputPivot[i] and MappedPivot[i] ( i = 0...15) are calculated as follows. 1. Set the variable OrgCW = 64. ​​When \(i = 0:16\), \(InputPivot[i]=i*OrgCW\) When \(i = 0:16\), \(MappedPivot[i]\) is calculated as follows. \(MappedPivot[0]=0;\) For the case of \((i = 0;i<16;i++)\) \(MappedPivot[i + 1]=MappedPivot[i]+Signale\) \(dCW[i]\) Here, \(SignaledCW[i]\) is the signaled number of the codeword for the \(i\)-th segment.

[0043] As shown in Figure 4, it is necessary to operate in two different domains during LMCS processing. For each CU encoded through the inter prediction mode ("inter CU"), its motion compensation prediction is executed in the original domain. However, the reconstruction of the luminance component (i.e., the addition of the luminance prediction samples and the luminance residual samples) is executed in the mapped domain. Therefore, the motion compensated luminance prediction \(Y\) is mapped to the value \(Y'\) pred in the mapped domain through the forward mapping function \(410\), i.e., \(Y'\) pred is used for pixel reconstruction \(405\) before the forward mapping function \(410\), i.e., \(Y'\) pred \(=FwdMap(Y\) pred ) from the original domain. On the other hand, considering that for each CU encoded through the intra prediction mode ("intra CU"), the intra prediction \(404\) is executed in the mapped domain (shown in Figure 4) before \(Y'\) pred is used for pixel reconstruction \(405\), the mapping of the prediction samples is not necessary. Finally, the reconstructed luminance samples \(Y'\) pred reco n After generating n , the inverse mapping function 406 is applied to obtain the reconstructed luminance sample Y ’ recon and convert it to the value Y of the original domain recon before proceeding to the luminance DPB 408, that is, Y = InvMap(Y’ recon )). Different from the forward mapping 410 of the prediction samples that only needs to be applied to the inter CU, the inverse mapping 406 of the reconstructed samples needs to be applied to both the inter CU and the intra CU recon ). To summarize, on the decoder side, the in-loop luminance mapping of the current LMCS is performed such that the luminance prediction sample Y is first converted to the mapped domain as needed. Y’pred = FwdMap(Ypred). Next, the mapped prediction sample is added to the decoded luminance residual to form the reconstructed luminance sample in the mapped domain. Y’recon = Y’pred + Y’rez. Finally, the inverse mapping is applied to convert the reconstructed luminance sample Y’recon back to the original domain . Yrecon = InvMap(Y’recon). On the encoder side, since the luminance residual is encoded in the mapped domain, it is generated as the difference between the original sample of the mapped luminance and the predicted sample of the mapped luminance. Y’rez = FwdMap(Yorg) - FwdMap(Ypred).

[0044] Luminance-dependent chrominance residual scaling, which is the second step of LMCS, is related to the quantization accuracy between the luminance signal and its corresponding chrominance signal when the in-loop mapping is applied to the luminance signal sample Y pred . Y’pred = FwdMap(Ypred). Next, the mapped prediction sample is added to the decoded luminance residual to form the reconstructed luminance sample in the mapped domain . Y’recon = Y’pred + Y’rez. Finally, the inverse mapping is applied to convert the reconstructed luminance sample Y’recon back to the original domain . Yrecon = InvMap(Y’recon). On the encoder side, since the luminance residual is encoded in the mapped domain, it is generated as the difference between the original sample of the mapped luminance and the predicted sample of the mapped luminance . Y’res = FwdMap(Yorg) - FwdMap(Ypred). . . . .

[0045] Luminance-dependent chrominance residual scaling, which is the second step of LMCS, is related to the quantization accuracy between the luminance signal and its corresponding chrominance signal when the in-loop mapping is applied to the luminance signal . is designed to compensate for the interaction. Whether the chroma residual scaling is effective or not is also signaled in the slice header. If the luminance mapping is enabled and the parallel tree splitting of the luminance component and the chroma component is disabled for the current slice an additional flag is signaled to indicate whether the luminance-dependent chroma residual scaling is applied. If the luminance mapping is not used or the parallel tree splitting is enabled for the current slice, the luminance-dependent chroma residual scaling is always disabled. Further for a CU containing four or fewer chroma samples, the chroma residual scaling is always disabled .

[0046] For both intra CUs and inter CUs, the scaling parameters used to scale the chroma residual depend on the average of the corresponding mapped luminance prediction samples . The scaling parameters are derived as described in the box immediately following this paragraph . avg’ Y is represented as the average of the luminance prediction samples in the mapped domain . The scaling parameter C ScaleInv is calculated according to the following steps 1. Find the segment index Y Y in the piecewise linear model to which avg’ belongs in the mapped domain. Here, Y 1dx has an integer value in the range from 0 to 15 1dx . 2. C ScaleInv = cScaleInv[Y 1dx , where cScaleIn v[i], i = 0...15 are 16 pre-calculated look-up tables (LUTs​​ ) it is. Intra prediction is performed in the mapped domain of LMCS, so for intra, combined inter-intra prediction (CIIP), or when the CU is coded as an intra-block-copy (IBC) mode, avg’ is calculated as the average of the luminance prediction samples. Otherwise, avg’ Y is calculated as the average of the forward-mapped inter-prediction luminance samples. Otherwise, avg’ Y is calculated as the average of the forward-mapped inter-prediction luminance samples. is calculated as the average of the forward-mapped inter-prediction luminance samples.

[0047] Figure 4 also shows the calculation of the average of the luminance prediction samples for luminance-dependent chroma residual scaling. For an inter-CU, the forward-mapped luminance prediction Y’ is supplied to the chroma residual scaling 411 together with the scaled chroma residual C pred to derive the chroma residual C which is supplied to the chroma reconstruction 413 together with the chroma prediction C resScale to derive the reconstructed chroma value C For an intra-CU, the intra prediction 404 generates Y’ res which is already in the mapped domain and is supplied to the chroma residual scaling 411 in the same way as for an inter-CU. recon to derive the reconstructed chroma value C For an intra-CU, the intra prediction 404 generates Y’ pred which is already in the mapped domain and is supplied to the chroma residual scaling 411 in the same way as for an inter-CU. For an intra-CU, the intra prediction 404 generates Y’ pred which is already in the mapped domain and is supplied to the chroma residual scaling 411 in the same way as for an inter-CU. which is already in the mapped domain and is supplied to the chroma residual scaling 411 in the same way as for an inter-CU.

[0048] Unlike the luminance mapping performed on a sample-by-sample basis, C ScaleInv is fixed for the entire chroma CU. Given C the chroma residual scaling is applied as described in the box immediately following this paragraph. ScaleInv Given C the chroma residual scaling is applied as described in the box immediately following this paragraph. Encoder side:

Number

Number

[0049] In the current new video coding standards such as VVC, new coding tools have been introduced. Some examples of the new coding tools are Bi-Directional Optical Flow (BDOF), Decoder-side Motion Vector Refinement (DMVR), Combined Inter and Intra Prediction (CIIP), Affine mode, and Prediction Refinement with Optical Flow (PROF) for the Affine mode.

[0050] In the current VVC, Bi-Directional Optical Flow (BDOF) is applied to correct the predicted samples of the bi-predicted coding blocks.

[0051] Figure 5 is an explanatory diagram of the BDOF process. BDOF is a fine adjustment of the motion regarding the samples executed on the block-based motion compensation prediction when bi-prediction is used. The motion fine adjustment of each 4×4 sub-block 501

Number

Number

[0052] Specifically, as described in the box immediately following this paragraph, the motion fine-tuning

Number

Number

Number

Number

Number

[0053] The value in the box immediately above

Number

Number

Number

Number

Number

Number

Number

[0054] the value in the box immediately above

Number

Number

Number

Number

Number

Number

Number

Number

Number

[0055] The above-mentioned movement fine adjustment

Number

Number

Number

Number

Number

Number

[0056] Based on the above-mentioned bit-depth control method, it is guaranteed that the maximum bit depth of the intermediate parameters of the entire BDOF process does not exceed 32 bits, and the maximum input to the multiplication is within 15 bits. That is, a 15-bit multiplier is sufficient for BDOF implementation. 。

[0057] DMVR further corrects by using bilateral matching prediction. The dual prediction used for merge blocks with two initially signaled MVs that can be done.

[0058] Specifically, in DMVR, bilateral matching is used to find the best match between two blocks along the motion trajectory of the current CU within two different reference pictures. Thereby, it is used to derive the motion information of the current CU. The cost function used for the matching process is the row subsampled SAD (Sum of Absolute Differences). After the matching process is performed, the corrected MV is used for motion compensation in the prediction stage. The temporal motion vector prediction of subsequent pictures and the uncorrected MV are used for motion vector prediction between the motion vector of the current CU and the motion vectors of its spatial neighbors.

[0059] Under the assumption of a continuous motion trajectory, the motion vectors MV0 and MV1 pointing to the two reference blocks are assumed to be proportional to the temporal distances between the current picture and the two reference pictures, i.e., TD0 and TD1. As a special case, when the current picture is temporally between the two reference pictures and the temporal distances from the current picture to the two reference pictures are the same, the bilateral matching becomes a mirror-based bidirectional MV.

[0060] ​​​​​​In the current VVC, the inter-prediction method and intra- prediction method are used in the hybrid video coding method, and each PU can only select inter-prediction or intra-prediction to utilize the correlation in either the temporal domain or the spatial domain, but not both. However, as pointed out in previous literature, the residual signals generated by the inter-prediction block and the intra- prediction block may present very different characteristics from each other. Therefore, if the two types of predictions can be efficiently combined, another more accurate prediction can be expected to reduce the energy of the prediction residual and improve the coding efficiency. Furthermore, in natural video content, the motion of moving objects may become complex. For example, there may be regions that contain both old content (e.g., objects included in previously encoded pictures) and new content (e.g., objects excluded in previously encoded pictures). In such a scenario, neither inter-prediction nor intra-prediction can provide an accurate prediction of the current block. To further improve the prediction efficiency, combined inter-intra prediction (CIIP), which combines the intra-prediction and inter-prediction of one CU encoded in the merge mode, is adopted in the VVC standard. Specifically, for each merge CU, one additional flag is signaled to indicate whether CIIP is enabled for the current CU. When the flag is equal to 1, CIIP applies only the planar mode to generate the intra-prediction samples of the luminance and chrominance components. Furthermore, equal weights (i.e., 0.5 are used to combine the intra-prediction and inter-prediction results to obtain the final prediction value. residual energy to improve the coding efficiency. In addition, in natural video content, the motion of moving objects may become complex. For example, there may be regions that contain both old content (e.g., objects included in previously encoded pictures) and new content (e.g., objects excluded in previously encoded pictures). In such a scenario, neither inter-prediction nor intra-prediction can provide an accurate prediction of the current block. To further improve the prediction efficiency, combined inter-intra prediction (CIIP), which combines the intra-prediction and inter-prediction of one CU encoded in the merge mode, is adopted in the VVC standard. Specifically, for each merge CU, one additional flag is signaled to indicate whether CIIP is enabled for the current CU. When the flag is equal to 1, CIIP applies only the planar mode to generate the intra-prediction samples of the luminance and chrominance components. Furthermore, equal weights (i.e., 0.5 are used to combine the intra-prediction and inter-prediction results to obtain the final prediction value. For example, there may be regions that contain both old content (e.g., objects included in previously encoded pictures) and new content (e.g., objects excluded in previously encoded pictures). In such a scenario, neither inter-prediction nor intra-prediction can provide an accurate prediction of the current block. To further improve the prediction efficiency, combined inter-intra prediction (CIIP), which combines the intra-prediction and inter-prediction of one CU encoded in the merge mode, is adopted in the VVC standard. Specifically, for each merge CU, one additional flag is signaled to indicate whether CIIP is enabled for the current CU. When the flag is equal to 1, CIIP applies only the planar mode to generate the intra-prediction samples of the luminance and chrominance components. Furthermore, equal weights (i.e., 0.5 are used to combine the intra-prediction and inter-prediction results to obtain the final prediction value. In such a scenario, neither inter-prediction nor intra-prediction can provide an accurate prediction of the current block. To further improve the prediction efficiency, combined inter-intra prediction (CIIP), which combines the intra-prediction and inter-prediction of one CU encoded in the merge mode, is adopted in the VVC standard. Specifically, for each merge CU, one additional flag is signaled to indicate whether CIIP is enabled for the current CU. When the flag is equal to 1, CIIP applies only the planar mode to generate the intra-prediction samples of the luminance and chrominance components. Furthermore, equal weights (i.e., 0.5

[0061] To further improve the prediction efficiency, combined inter-intra prediction (CIIP), which combines the intra-prediction and inter-prediction of one CU encoded in the merge mode, is adopted in the VVC standard. Specifically, for each merge CU, one additional flag is signaled to indicate whether CIIP is enabled for the current CU. When the flag is equal to 1, CIIP applies only the planar mode to generate the intra-prediction samples of the luminance and chrominance components. Furthermore, equal weights (i.e., 0.5 are used to combine the intra-prediction and inter-prediction results to obtain the final prediction value. is adopted in the VVC standard. Specifically, for each merge CU, one additional flag is signaled to indicate whether CIIP is enabled for the current CU. When the flag is equal to 1, CIIP applies only the planar mode to generate the intra-prediction samples of the luminance and chrominance components. Furthermore, equal weights (i.e., 0.5 are used to combine the intra-prediction and inter-prediction results to obtain the final prediction value. are used to combine the intra-prediction and inter-prediction results to obtain the final prediction value. are used to combine the intra-prediction and inter-prediction results to obtain the final prediction value. ) is applied as the final prediction sample of CIIP CU by averaging the inter-prediction sample and the intra- prediction sample.

[0062] VVC also supports an affine mode for motion compensation prediction. In HEVC, only a translational motion model is applied to motion compensation prediction. In the real world, there are many types of motions such as zoom-in / zoom-out, rotation, perspective motion, and other irregular motions. In VVC, by signaling one flag for each inter-coded block, affine motion compensation prediction is applied to indicate whether a translational motion or an affine motion model is applied to inter-prediction. In the current VVC design, two affine modes including a 4-parameter affine mode and a 6-parameter affine mode are supported for one affine-coded block.

[0063] The 4-parameter affine model has the following parameters. Two parameters for each translational motion in the horizontal and vertical directions, one parameter for zoom motion, and one parameter for rotational motion in both directions. The horizontal zoom parameter is equal to the vertical zoom parameter. The horizontal rotation parameter is equal to the vertical rotation parameter. To achieve better adaptation of motion vectors and affine parameters, in VVC, those affine parameters are transformed into two MVs (also called control point motion vectors (CPMVs)) located at the upper left corner and the upper right corner of the current block. The affine motion field of

[0064] the block is described by two control point MVs (V0, V1). Motion of control pointsVector Based on this, the motion vector (v x , v y ) of one affine-coded block is calculated as described in the box immediately following this paragraph.

Number

[0065] The 6-parameter affine mode has the following parameters: two parameters for each translational motion in the horizontal and vertical directions, one parameter for zoom motion, and one parameter for rotational motion in the horizontal direction, one parameter for zoom motion, and one parameter for rotational motion in the vertical direction. The 6-parameter affine motion model is coded with three MVs in three CPMVs.

[0066] Affine movement mode The three control points of one 6-parameter affine block are located at the upper left, upper right, and lower left corners of the block. The motion of the upper left control point is related to translational motion, the motion of the upper right control point is related to horizontal rotation and zoom motion, and the motion of the lower left control point is related to vertical rotation and zoom motion. Compared with the 4-parameter affine motion model, the horizontal rotation and zoom motions of the 6-parameter

[0067] Assume that (V0, V1, V2) are the MVs at the upper left, upper right, and lower left corners of the current block. Then, the motion vectors of each sub-block (v x , v y ) are derived using the three MVs at the control points as described in the box immediately following this paragraph.

Number

[0068] To improve the affine motion compensation accuracy, predictive fine-tuning (P ROF) using optical flow is currently being studied in the current VVC, which corrects sub-block-based affine motion compensation based on an optical flow model. Specifically, after performing sub-block-based affine motion compensation, the luminance prediction samples of one affine block are corrected by one sample fine-tuning value derived based on the optical flow equation. Specifically, the operation of PROF can be summarized into the following four steps. Based on the optical flow equation. Specifically, after performing sub-block-based affine motion compensation, the luminance prediction samples of one affine block are corrected by one sample fine-tuning value derived based on the optical flow equation. Specifically, the operation of PROF can be summarized into the following four steps. After performing sub-block-based affine motion compensation, the luminance prediction samples of one affine block are corrected by one sample fine-tuning value derived based on the optical flow equation. Specifically, the operation of PROF can be summarized into the following four steps.

[0069] In step 1, sub-block-based affine motion compensation is performed, and the sub-block prediction For the four-parameter affine model in the above equation (6) and the six-parameter affine model in the above equation (7), the derived sub-block MV is used to generate sub-block prediction

Number

[0070] In step 2, the spatial gradients

Number

Number

Number

[0071] Furthermore, in step 2, to calculate the gradient, one additional row / column of the prediction samples needs to be generated on each side of one sub-block. To reduce the memory bandwidth and complexity, the samples on the extended boundary are copied from the nearest integer pixel position within the reference picture to avoid additional interpolation processing.

[0072] In step 3, the luminance prediction fine-tuning value is calculated as described in the box immediately following this paragraph.

Number

Number

Number

Number

Number

[0073] Furthermore, in the current PROF design, after adding the prediction fine-tuning to the original prediction samples, as described in the box immediately following this paragraph, a single clipping operation is performed as the fourth step to clip the values of the corrected prediction samples within 15 bits.

Number

Number

Number

Number

[0074] Since the affine model parameters and the pixel positions with respect to the sub-block centers do not change for each sub-block,

Number

Number

Number

Number

Number

Number

[0075] Based on the above affine sub-block MV derivation formulas (6) and (7), the MV difference​

Number

Number

Number

Number

Number

Number

Number

Number

Number

Number

[0076] According to the current LMCS design, the chroma residual samples are their corresponding luminance prediction samples It is scaled based on the luma. Newer coding tools are effective for inter-CUs When this is done, the luma prediction samples used to scale the chroma residual samples through the LMCS within this inter-CU are obtained last after applying the cross of these newer coding tools

[0077] Figure 6 is a flowchart showing the workflow of chroma residual scaling in LM CS when all of DMVR, BDOF, and CIIP are enabled. The outputs from the luma L 0 prediction value 601 and the luma L1 prediction value 602 are sequentially supplied to DMVR 603 and BD OF 604, and the obtained luma inter-prediction value 621 is supplied to the average 606 together with the luma intra-prediction value 622 from the luma intra-prediction 605 to generate the average luma prediction value 6 23, which is supplied to the chroma residual scaling 607 together with the chroma residual 608, As a result, the chroma residual scaling 607, chroma prediction 610, and chroma reconstruction 609 can cooperate to generate the final result.

[0078] The current LMCS design presents three problems for video decoding processing. First, the mapping between different domains (different domain mapping) requires additional computational complexity and on-chip memory. Second, the fact that the derivation of the luma and chroma scaling coefficients uses different luma prediction values brings additional complexity. Third, the interaction between LMCS and newer coding tools introduces a delay in the decoding process, that is, a delay related to LMCS problem.

[0079] First, in the current LMCS design, the reconstructed samples in the original domain are mapped ​​Both of the domains obtained are used in various decoding modules. As a result, these samples often need to be converted from one domain to another between different decoding modules, which can lead to both higher computational complexity and more on-chip memory.

[0080] Specifically, in the case of the intra mode, CIIP mode, and IBC mode, the mapped domain reference samples from the adjacent reconstructed domains of one current CU are used to generate prediction samples. However, in the case of the inter mode, motion compensation prediction is performed using the original domain reconstructed samples of the temporal reference picture as a reference. The reconstructed samples stored in the DPB are also in the original domain. As shown in FIG. 4, such a mixed representation of the reconstructed samples under different prediction modes results in additional forward and inverse luminance mapping operations.

[0081] For example, in the case of an inter CU, the luminance reconstruction operation (i.e., adding the prediction sample and the residual sample together) is performed in the mapped domain, so the inter prediction luminance samples generated in the original domain need to be converted to the mapped domain before they are used for luminance sample reconstruction. In another example, for both the intra CU and the inter CU, inverse (or reverse) mapping is always applied so that the reconstructed luminance samples are converted from the mapped domain to the original domain and then stored in the DPB afterwards. Such a design not only increases the computational complexity due to the additional forward / reverse mapping operations, but also maintains multiple versions of the reconstructed samples for this purpose. ​​​​​ requires more on-chip memory.

[0082] Based on the above description, in some LMCS designs, luminance mapping and luminance-dependent chroma residual scaling are performed to encode the luminance component and the chroma component, respectively. In an actual hardware implementation, the forward and inverse (or reverse) mapping functions FwdMap and InvMap can be implemented using a look-up table (LUT) or calculated on-the-fly. If a LUT-based solution is used, the possible output elements from the functions FwdMap, InvMap, and cScaleInv can be pre-calculated and stored in advance as a LUT, and then used for the luminance mapping and chroma residual scaling operations of all CUs within the current slice. Assuming the input video is 10-bit, each LUT for FwdMap and InvMap has 2 10 = 1024 elements, and each element of the LUT has 10 bits. Therefore, the total memory capacity of the LUTs for forward and inverse luminance mapping is equal to 2 * 102 10 4 * 10 = 20480 bits = 2560 bytes. On the other hand, to derive the chroma scaling parameter C a 16-entry LUT table cScaleInv needs to be maintained in the encoder and decoder, and each chroma scaling parameter is stored in 32 bits. Correspondingly, the memory size used to store the LUT cScaleInv is equal to 16 * 32 = 512 bits = 64 bytes. The difference between 2560 and 64 is due to the forward and inverse (reverse) mapping operations. ScaleInv To derive the chroma scaling parameter C, a 16-entry LUT table cScaleInv needs to be maintained in the encoder and decoder, and each chroma scaling parameter is stored in 32 bits. Correspondingly, the memory size used to store the LUT cScaleInv is equal to 16 * 32 = 512 bits = 64 bytes. The difference between 2560 and 64 is due to the forward and inverse (reverse) mapping operations. is equal to 16 * 32 = 512 bits = 64 bytes. The difference between 2560 and 64 is due to the forward and inverse (reverse) mapping operations. The difference between 2560 and 64 is due to the forward and inverse (reverse) mapping operations. Shows the scale of the additional on-chip memory required.

[0083] Furthermore, in new video coding standards such as the current VVC, both intra prediction and deblocking filters use the reconstructed samples of the adjacent blocks described above. Therefore, one extra row of reconstructed samples at the width of the current picture / slice needs to be maintained in a buffer, which is also known as the "line buffer" in video coding. The reconstructed samples in the line buffer are used as a reference for at least the intra prediction and deblocking operations of the CUs located in the first row within one CTU. According to the existing LMCS design, the intra prediction and deblocking filters use reconstructed samples from different domains. Therefore, additional on-chip memory is required to store both the original domain reconstructed samples and the mapped domain reconstructed samples, which may approximately double the size of the line buffer.

[0084] In addition to increasing the line buffer size, another implementation option to avoid doubling the line buffer size is to perform the domain mapping operation on the fly. However, this comes at the cost of an unacceptable increase in computational complexity.

[0085] Therefore, the current design of LMCS requires additional computational complexity and on-chip memory for the necessary mapping between different domains.

[0086] Second, in the proposed adaptive luminance residual scaling, both the luminance component and the chrominance component ​​​It has a scaling operation on these prediction residuals. The luminance and chroma scaling coefficient derivation methods in the current design of LMCS both use luminance prediction sample values to derive the corresponding scaling coefficients, but there are differences between their corresponding operations.

[0087] In the case of luminance residual scaling, the scaling coefficient is derived for each sample by enabling each luminance residual sample to have its own scaling coefficient. However, in the case of chroma residual scaling, the scaling coefficient is fixed for the entire CU, i.e., all chroma residual samples within the CU share the same scaling coefficient calculated based on the average of the mapped luminance prediction samples.

[0088] Also, two different LUTs are used to calculate the scaling coefficients for luminance and chroma residuals. Specifically, the input to the luminance LUT is the mapping model segment index of the original luminance prediction sample value, and the input to the chroma LUT is the mapping model segment index of the average value of the mapped luminance prediction samples. In some examples, it is possible to scale both luminance and chroma residuals using one LUT without the need to map the luminance prediction samples to the mapped domain.

[0089] Such differences bring extra complexity to the encoding process, and a harmonized approach to luminance and chroma scaling coefficient derivation is desirable. Therefore, several methods can be proposed to harmonize the scaling methods for luminance and chroma residuals in order to achieve one unified design. ​

[0090] Thirdly, as described above regarding "luminance-dependent chroma residual scaling", for the current LMCS design, chroma residual samples are scaled based on their corresponding luminance prediction samples. This means that until all luminance prediction samples of a CU are fully generated, the chroma residual samples of one LMCS CU cannot be reconstructed. Also, as described above, in order to improve the efficiency of inter prediction, DMVR, BDOF, and CIIP can be applied. As shown in FIG. 6, for the chroma residual scaling of the current design of LMCS, all three modules of DMVR, BDOF, and CIIP and other new coding tools can be sequentially called to generate the luminance prediction samples used to determine the scaling coefficient of the chroma residual. Considering the high computational complexity of the three modules, waiting for their successful completion before performing the chroma residual scaling of LMCS may cause significant delays for the decoding of chroma samples. For the affine CU, since each affine CU can perform the PROF process and subsequently execute LMCS, the PROF process may also have a delay problem, which may also cause a delay problem in the decoding of chroma samples. Furthermore, in the current design of LMCS, unnecessary clipping operations are performed during the derivation of the chroma residual scaling coefficient, further increasing the computational complexity and the additional requirements for on-chip memory. This disclosure solves or alleviates these problems presented by the current design of LMCS. Considering the high computational complexity of the three modules, waiting for their successful completion before performing the chroma residual scaling of LMCS may cause significant delays for the decoding of chroma samples. For the affine CU, since each affine CU can perform the PROF process and subsequently execute LMCS, the PROF process may also have a delay problem, which may also cause a delay problem in the decoding of chroma samples. For the affine CU, since each affine CU can perform the PROF process and subsequently execute LMCS, the PROF process may also have a delay problem, which may also cause a delay problem in the decoding of chroma samples. For the affine CU, since each affine CU can perform the PROF process and subsequently execute LMCS, the PROF process may also have a delay problem, which may also cause a delay problem in the decoding of chroma samples. For the affine CU, since each affine CU can perform the PROF process and subsequently execute LMCS, the PROF process may also have a delay problem, which may also cause a delay problem in the decoding of chroma samples.

[0091] Furthermore, in the current design of LMCS, unnecessary clipping operations are performed during the derivation of the chroma residual scaling coefficient, further increasing the computational complexity and the additional requirements for on-chip memory. Furthermore, in the current design of LMCS, unnecessary clipping operations are performed during the derivation of the chroma residual scaling coefficient, further increasing the computational complexity and the additional requirements for on-chip memory. Furthermore, in the current design of LMCS, unnecessary clipping operations are performed during the derivation of the chroma residual scaling coefficient, further increasing the computational complexity and the additional requirements for on-chip memory.

[0092] This disclosure solves or alleviates these problems presented by the current design of LMCS. For this purpose, and more particularly, the present disclosure discusses a method that can reduce the complexity of LMCS for hardware codec implementation while maintaining coding gain. Instead of using the existing LMCS framework that converts prediction / reconstruction samples by a mapping operation, a new method called Prediction Dependent Residual Scaling (PDRS) is proposed, which directly scales the prediction residual without sample mapping. The proposed method can achieve similar effects and coding efficiency as LMCS, but the implementation complexity is much lower.

[0093] In the PDRS procedure, as shown in FIG. 7, luminance prediction samples for decoding luminance residual samples are obtained (701), scaling factors are derived using the luminance prediction samples (702), the luminance residual samples are scaled using the scaling factors (703), and the reconstructed luminance samples are calculated by adding the luminance prediction samples and the scaled luminance residual samples (704). Instead of using the existing LMCS framework that converts prediction / reconstruction samples by a mapping operation, a new method called Prediction Dependent Residual Scaling (PDRS) is proposed, which directly scales the prediction residual without sample mapping. The proposed method can achieve similar effects and coding efficiency as LMCS, but the implementation complexity is much lower. In the PDRS procedure, as shown in FIG. 7, luminance prediction samples for decoding luminance residual samples are obtained (701), scaling factors are derived using the luminance prediction samples (702), the luminance residual samples are scaled using the scaling factors (703), and the reconstructed luminance samples are calculated by adding the luminance prediction samples and the scaled luminance residual samples (704). In some examples, an adaptive luminance residual scaling method is proposed to reduce the implementation complexity of LMCS. Specifically, different from the existing LMCS method that directly converts predicted / reconstructed luminance samples to the mapped domain before calculating the luminance prediction residual, in the proposed method of the PDRS procedure, the luminance prediction residual samples are derived in the same way as the normal prediction process in the original domain without a mapping operation, and then a scaling operation on the luminance prediction residual is performed. The scaling of the luminance prediction residual is based on the corresponding luminance prediction samples. In some examples, an adaptive luminance residual scaling method is proposed to reduce the implementation complexity of LMCS. Specifically, different from the existing LMCS method that directly converts predicted / reconstructed luminance samples to the mapped domain before calculating the luminance prediction residual, in the proposed method of the PDRS procedure, the luminance prediction residual samples are derived in the same way as the normal prediction process in the original domain without a mapping operation, and then a scaling operation on the luminance prediction residual is performed. The scaling of the luminance prediction residual is based on the corresponding luminance prediction samples.

[0094] In the PDRS procedure, as shown in FIG. 7, luminance prediction samples for decoding luminance residual samples are obtained (701), scaling factors are derived using the luminance prediction samples (702), the luminance residual samples are scaled using the scaling factors (703), and the reconstructed luminance samples are calculated by adding the luminance prediction samples and the scaled luminance residual samples (704). In the PDRS procedure, as shown in FIG. 7, luminance prediction samples for decoding luminance residual samples are obtained (701), scaling factors are derived using the luminance prediction samples (702), the luminance residual samples are scaled using the scaling factors (703), and the reconstructed luminance samples are calculated by adding the luminance prediction samples and the scaled luminance residual samples (704). In some examples, an adaptive luminance residual scaling method is proposed to reduce the implementation complexity of LMCS. Specifically, different from the existing LMCS method that directly converts predicted / reconstructed luminance samples to the mapped domain before calculating the luminance prediction residual, in the proposed method of the PDRS procedure, the luminance prediction residual samples are derived in the same way as the normal prediction process in the original domain without a mapping operation, and then a scaling operation on the luminance prediction residual is performed. The scaling of the luminance prediction residual is based on the corresponding luminance prediction samples. In some examples, an adaptive luminance residual scaling method is proposed to reduce the implementation complexity of LMCS. Specifically, different from the existing LMCS method that directly converts predicted / reconstructed luminance samples to the mapped domain before calculating the luminance prediction residual, in the proposed method of the PDRS procedure, the luminance prediction residual samples are derived in the same way as the normal prediction process in the original domain without a mapping operation, and then a scaling operation on the luminance prediction residual is performed. The scaling of the luminance prediction residual is based on the corresponding luminance prediction samples. In the PDRS procedure, as shown in FIG. 7, luminance prediction samples for decoding luminance residual samples are obtained (701), scaling factors are derived using the luminance prediction samples (702), the luminance residual samples are scaled using the scaling factors (703), and the reconstructed luminance samples are calculated by adding the luminance prediction samples and the scaled luminance residual samples (704).

[0095] In some examples, an adaptive luminance residual scaling method is proposed to reduce the implementation complexity of LMCS. Specifically, different from the existing LMCS method that directly converts predicted / reconstructed luminance samples to the mapped domain before calculating the luminance prediction residual, in the proposed method of the PDRS procedure, the luminance prediction residual samples are derived in the same way as the normal prediction process in the original domain without a mapping operation, and then a scaling operation on the luminance prediction residual is performed. The scaling of the luminance prediction residual is based on the corresponding luminance prediction samples. In some examples, an adaptive luminance residual scaling method is proposed to reduce the implementation complexity of LMCS. Specifically, different from the existing LMCS method that directly converts predicted / reconstructed luminance samples to the mapped domain before calculating the luminance prediction residual, in the proposed method of the PDRS procedure, the luminance prediction residual samples are derived in the same way as the normal prediction process in the original domain without a mapping operation, and then a scaling operation on the luminance prediction residual is performed. The scaling of the luminance prediction residual is based on the corresponding luminance prediction samples. In some examples, an adaptive luminance residual scaling method is proposed to reduce the implementation complexity of LMCS. Specifically, different from the existing LMCS method that directly converts predicted / reconstructed luminance samples to the mapped domain before calculating the luminance prediction residual, in the proposed method of the PDRS procedure, the luminance prediction residual samples are derived in the same way as the normal prediction process in the original domain without a mapping operation, and then a scaling operation on the luminance prediction residual is performed. The scaling of the luminance prediction residual is based on the corresponding luminance prediction samples. In some examples, an adaptive luminance residual scaling method is proposed to reduce the implementation complexity of LMCS. Specifically, different from the existing LMCS method that directly converts predicted / reconstructed luminance samples to the mapped domain before calculating the luminance prediction residual, in the proposed method of the PDRS procedure, the luminance prediction residual samples are derived in the same way as the normal prediction process in the original domain without a mapping operation, and then a scaling operation on the luminance prediction residual is performed. The scaling of the luminance prediction residual is based on the corresponding luminance prediction samples. In some examples, an adaptive luminance residual scaling method is proposed to reduce the implementation complexity of LMCS. Specifically, different from the existing LMCS method that directly converts predicted / reconstructed luminance samples to the mapped domain before calculating the luminance prediction residual, in the proposed method of the PDRS procedure, the luminance prediction residual samples are derived in the same way as the normal prediction process in the original domain without a mapping operation, and then a scaling operation on the luminance prediction residual is performed. The scaling of the luminance prediction residual is based on the corresponding luminance prediction samples. In some examples, an adaptive luminance residual scaling method is proposed to reduce the implementation complexity of LMCS. Specifically, different from the existing LMCS method that directly converts predicted / reconstructed luminance samples to the mapped domain before calculating the luminance prediction residual, in the proposed method of the PDRS procedure, the luminance prediction residual samples are derived in the same way as the normal prediction process in the original domain without a mapping operation, and then a scaling operation on the luminance prediction residual is performed. The scaling of the luminance prediction residual is based on the corresponding luminance prediction samples. It depends on the value and the piecewise linear model. As a result, the forward and inverse luminance mapping operations in the current LMCS design can be completely removed, and all the prediction samples and reconstruction samples involved in the decoding process are maintained in the original sample domain. Based on the above characteristics, the proposed method is called predictive dependent residual scaling. Furthermore, in order to improve the latency of chroma residual scaling derivation, several methods can be proposed to completely or partially exclude the

[0096] DMVR, BDOF, and CIIP operations from the generation of the luminance prediction samples used to calculate the scaling parameters of the chroma residual samples. Figure 8 is a flowchart showing the decoding process workflow when the PDRS procedure is applied in the LMCS process. This shows the elimination of the need for mapping between different domains. Here, all other decoding modules (including intra prediction 804, 809, 812 and inter prediction 816, reconstruction 806 and 813, and all in-loop filters 807 and 814) operate in the original domain, except for the residual decoding modules (e.g., entropy decoding res 801, inverse quantization 802 and inverse transform 803). Specifically, in order to reconstruct the luminance samples, the method proposed in the PDRS procedure pred is to inverse scale the prediction residual samples Y of luminance to the

[0097] original amplitude level, and then simply add them to the luminance prediction samples Y. This completely removes the forward and inverse luminance sample Reduce the potential memory capacity size to save S parameters. For example, when the LUT-based solution is used to perform luminance mapping, the previously used memory capacity to store two mapping LUTs FwdMap[] and InvMap[] (about 2560 bytes) is no longer required in the proposed method. Furthermore, unlike existing luminance mapping methods that require storing reconstructed luminance samples in both the original domain and the mapped domain, the proposed method of the PDRS procedure generates and maintains all prediction samples and reconstruction samples only in the original domain. Correspondingly, compared with existing luminance mapping, the proposed method in the PDRS procedure can efficiently reduce the size of the line buffer used to store samples reconstructed for intra prediction and deblocking by half.

[0098] According to one or more embodiments of the PDRS procedure, the luminance prediction samples and luminance residual samples are from the same location within one of the luminance prediction block and its associated residual block.

[0099] According to one or more embodiments of the PDRS procedure, the step of deriving the scaling factor using the luminance prediction samples includes dividing the entire range of possible luminance prediction sample values into a plurality of luminance prediction sample segments, calculating one scaling factor for each of the plurality of luminance prediction sample segments based on a predefined Section linear model, and determining the scaling factor of the luminance prediction samples based on the scaling factors of the plurality of luminance prediction sample segments.

[0100] In one example, luminance prediction is based on the scaling factors of a plurality of luminance prediction sample segments. The step of determining the scaling factor of the sample includes assigning the luminance prediction sample to one of the plurality of luminance prediction sample segments, and calculating the scaling factor of the luminance prediction sample as the scaling factor of the assigned luminance prediction sample segment. This includes.

[0101] In this example, the plurality of luminance prediction sample segments include 16 segments in a pre-defined 16-entry LUT table scaleForward, and a pre-defined Section linear model for calculating one scaling factor for each of the plurality of luminance prediction sample segments includes 16 values corresponding to the 16 segments in the pre-defined LUT table scaleForward.

[0102] To maintain operational accuracy, the scaling parameters used to scale / inverse-scale the luminance residual samples may be determined based on the corresponding luminance prediction samples located in the same place. In one example, if Pred is the value of one luminance prediction sample, the scaling factor of its corresponding residual sample is calculated through the following steps. Y For one luminance prediction sample value, the scaling factor of its corresponding residual sample is calculated through the following steps. This is calculated through.

[0103] In the same example, the scaling factor (e.g., the luminance residual scaling factor Scale Y ) is calculated based on the assigned luminance prediction sample segment as described in the box immediately following this paragraph. This is calculated based on. Scale Y =scaleForward[Idx Y ​Here, Y is the luminance residual value for which the scaling coefficient is calculated, and Scale Y is , the scaling coefficient, and scaleForward[i] (i = 0...15) is the pre - defined 16 LUT tables, and Idx Y is the segment index of the segment assigned to the luminance prediction sample domain value. scaleForward[i] (i = 0...15) is pre - calculated as follows . scaleForward[i]=(OrgCW << SCALE_FP_PREC) / SignaledCW[i] Here, OrgCW and SignaledCW[i] are the number of codewords of the i - th segment in the original domain and the mapped domain respectively, and SCALE _FP_PREC is the accuracy of the scaling coefficient.

[0104] In the same example, when the luminance scaling coefficient Scale Y is given, the luminance residual sample scaling method can be applied as described in the box immediately following this paragraph .

[0105] The motivation behind this example is that the forward mapping in the current LMCS is based on a single - segment linear model. When both the original luminance sample and the luminance prediction sample are located in the same segment (i.e., the same segment defined by two pivot points InputPivot[i] and InputPi vot[i + 1]), the two forward mapping functions of the original luminance sample and the predicted luminance sample will be exactly the same. Corresponding to this Then, Y’res = FwdMap(Yorg) - FwdMap(Ypred) = FwdMa p(Yorg - Ypred) == FwdMap(Yres). By applying the inverse mapping to both sides of this equation, the corresponding decoder-side reconstruction operation can be expressed as follows and can be written as. Yrecon = Ypred + InvMap(Y’res). That is, in the situation where both the original luminance sample and the luminance prediction sample are located in the same fragment

[0106] In other words, in the situation where both the original luminance sample and the luminance prediction sample are located in the same fragment in LMCS, the luminance mapping method can be achieved by one residual scaling operation in the decoding process, as implemented in this possible implementation form, as shown in, for example, Figure 8 can be achieved Encoder side:

Number

Number

[0107] Such a conclusion is derived based on the assumption that both the original luminance sample and the luminance prediction sample are located in the same segment defined by two pivot points InputPivot[i] and InputPivot[i + 1]. However, this possible implementation form of this example can still be used as a simplification and / or approximation of the existing luminance mapping operations in VVC even when the original luminance sample and the luminance prediction sample are located in different segments of the piecewise linear model The experimental results show that such simplification and / or approximation have little impact on the coding performance is the case where they are located in different segments of the piecewise linear model of the original luminance sample and the luminance prediction sample, and even when they are located in different segments of the piecewise linear model approximation can still be used as a simplification and / or approximation of the existing luminance mapping operations in VVC The experimental results show that such simplification and / or approximation have little impact on the coding performance

[0108] Although it is repetitive, this example is based on the assumption that both the original luminance sample value and the predicted luminance sample value are located in the same segment of the piecewise linear mode. In this case, the forward / inverse mapping functions applied to the original luminance sample and the predicted luminance sample are the same . Therefore, it is safe to simply calculate the corresponding residual scaling coefficient depending on the luminance prediction sample.

[0109] However, when the prediction sample of the CU is not accurate enough (for example, samples far from the reference sample are usually not predicted very accurately, in the case of an intra-predicted CU ), the prediction sample and the original sample are often located in different segments of the piecewise linear model. In this case, the scaling coefficient derived based on the predicted sample value may not be reliable when reflecting the original mapping relationship between the residual sample in the original (i.e., unmapped) domain and the residual sample in the mapped domain.

[0110] FIG. 9 is a diagram showing the residual mapping error caused by using only the predicted sample to derive the scaling coefficient. In FIG. 9, the filled triangular points represent the pivot control points of different segments in the piecewise linear function, and the filled circular points represent the original sample value and the predicted sample value. Y and Y are the original samples and the predicted samples in the original (i.e., unmapped) domain. Y' org and Y' pred are the original samples and the predicted samples in the original (i.e., unmapped) domain. Y' and Y' org and Y' pred are the mapped samples corresponding to Y org and Y pre d is the mapped sample. Y res and Y' res are the original domain and the mapped domain in the case where the existing sample-based luminance mapping method in VVC is applied, respectively. The corresponding residuals in the original domain and the mapped domain are as follows. Y' is the corresponding residual in the original domain and the mapped domain when the existing sample-based luminance mapping method in VVC is applied. Y' resScale is the mapped residual sample derived based on the proposed luminance residual scaling method. As shown in FIG. 9, since the original sample and the predicted sample are not within the same segment of the piecewise linear model, the scaling coefficient derived based on the predicted sample may not be accurate enough to approximate the original residual in the mapped domain (i.e., Y' ) to generate the scaled residual (i.e., res ) that approximates the original residual within the mapped domain (i.e., Y' ). resScale ) within the mapped domain (i.e., Y'

[0111] In the second example, it is not necessary to assume that both the original luminance sample value and the predicted luminance sample value are located in the same segment of the piecewise linear model.

[0112] In this second example, in order to improve the accuracy of the residual scaling coefficient, instead of directly deriving the scaling coefficient from the segment of the piecewise linear model where the luminance prediction sample is located, the scaling coefficient is calculated as the average of the scaling coefficients of N (N is a positive integer) adjacent segments.

[0113] In this second example, the step of determining the scaling coefficient of the luminance prediction sample based on the scaling coefficients of multiple luminance prediction sample segments includes the step of assigning the luminance prediction sample to one of the multiple luminance prediction sample segments, and the step of predicting the luminance sample based on the scaling coefficients of the multiple The scaling factor of the measurement sample is calculated as the average of the scaling factors of several luminance prediction sample segments adjacent to the assigned luminance prediction sample segment and includes the step of calculating as the average of the scaling factors of several luminance prediction sample segments adjacent to the assigned luminance prediction sample segment and, including

[0114] More specifically, in one possible implementation of this second example, the scaling factor can be calculated based on the assigned luminance prediction sample segment as described in the steps below For example, on the decoder side, when the luminance prediction sample Pred and the luminance residual Y are given, the scaling factor applied to the inverse scaling

Number

Number

Number

Number

Number

[0115] In a second possible implementation of this second example, which is identical to the above implementation in other respects, the scaling coefficient can be calculated based on the assigned luminance prediction sample segments as described in the box immediately following this paragraph. 1) Find or obtain the corresponding segment index Idx Y of the piecewise linear model to which Pred belongs in the original domain Y 2) The luminance residual scaling coefficient is calculated as follows.

Equation

[0116] These two possible implementations of this second example differ only in the selection of N luminance prediction sample domain values segments based on the assigned segments.

[0117] ​​​​In one chroma sample reconstruction procedure, as shown in FIG. 10, a luminance prediction sample value is obtained to decode both the luminance residual sample and the chroma residual sample at the input position (1 001), then a luminance prediction sample associated with the luminance residual sample is obtained (10 02), then a chroma prediction sample associated with the chroma residual sample is obtained (100 3), the first scaling coefficient of the luminance residual sample and the second scaling coefficient of the chroma residual sample are derived using the luminance prediction sample (1004), the luminance residual sample is scaled using the first scaling coefficient (1005), the chroma residual sample is scaled using the second scaling coefficient (1006), the reconstructed luminance sample is calculated by adding the chroma prediction sample and the scaled chroma residual sample (1007), and the reconstructed chroma sample is calculated by adding the chroma prediction sample and the scaled chroma residual sample (1008). The chroma sample reconstruction procedure aims to harmonize the scaling methods of the luminance residual and the chroma residual so as to achieve a more unified design. According to one or more embodiments of the chroma sample reconstruction procedure, the luminance prediction sample value is the average of all the luminance prediction samples within the coding unit (CU) including the input position. In these embodiments, the chroma scaling derivation method is used to calculate the scaling coefficient of the luminance residual. More specifically, instead of separately deriving one scaling coefficient for each luminance residual sample, one scaling coefficient calculated based on the average of the luminance prediction samples is used. The chroma sample reconstruction procedure aims to harmonize the scaling methods of the luminance residual and the chroma residual so as to achieve a more unified design. The reconstructed luminance sample is calculated by adding the chroma prediction sample and the scaled chroma residual sample (1007), and the reconstructed chroma sample is calculated by adding the chroma prediction sample and the scaled chroma residual sample (1008). The chroma sample reconstruction procedure aims to harmonize the scaling methods of the luminance residual and the chroma residual so as to achieve a more unified design. The reconstructed luminance sample is calculated by adding the chroma prediction sample and the scaled chroma residual sample (1007), and the reconstructed chroma sample is calculated by adding the chroma prediction sample and the scaled chroma residual sample (1008).

[0118] According to one or more embodiments of the chroma sample reconstruction procedure, the luminance prediction sample value is the average of all the luminance prediction samples within the coding unit (CU) including the input position. In these embodiments, the chroma scaling derivation method is used to calculate the scaling coefficient of the luminance residual. More specifically, instead of separately deriving one scaling coefficient for each luminance residual sample, one scaling coefficient calculated based on the average of the luminance prediction samples is used.

[0119] According to one or more embodiments of the chroma sample reconstruction procedure, the luminance prediction sample value is the average of all the luminance prediction samples within the coding unit (CU) including the input position. In these embodiments, the chroma scaling derivation method is used to calculate the scaling coefficient of the luminance residual. More specifically, instead of separately deriving one scaling coefficient for each luminance residual sample, one scaling coefficient calculated based on the average of the luminance prediction samples is used. In these embodiments, the chroma scaling derivation method is used to calculate the scaling coefficient of the luminance residual. More specifically, instead of separately deriving one scaling coefficient for each luminance residual sample, one scaling coefficient calculated based on the average of the luminance prediction samples is used. In these embodiments, the chroma scaling derivation method is used to calculate the scaling coefficient of the luminance residual. More specifically, instead of separately deriving one scaling coefficient for each luminance residual sample, one scaling coefficient calculated based on the average of the luminance prediction samples is used. In these embodiments, the chroma scaling derivation method is used to calculate the scaling coefficient of the luminance residual. More specifically, instead of separately deriving one scaling coefficient for each luminance residual sample, one scaling coefficient calculated based on the average of the luminance prediction samples is used. The total scaling factor is used to scale the luminance residual samples of the entire CU. It is used.

[0120] According to another embodiment of the chroma sample reconstruction procedure, the luminance prediction sample value is the average of all luminance prediction samples within a predefined sub-block subdivided from a coding unit (CU) containing the input position. In this embodiment, a sub-block based method can be proposed to derive the scaling factors for both the luminance residual and the chroma residual. Specifically, one CU is first equally divided into a plurality of M×N sub-blocks. For each sub-block, all or partial luminance prediction samples are used, and the corresponding scaling factor used to scale the residuals of both the luminance and chroma of the sub-block is derived. Compared with the first method, the second method can improve the spatial accuracy of the estimated scaling factor because the low-correlation luminance prediction samples outside the sub-block are excluded from the calculation of the sub-block scaling factor. Specifically, one CU is first equally divided into a plurality of M×N sub-blocks. For each sub-block, all or partial luminance prediction samples are used, and the corresponding scaling factor used to scale the residuals of both the luminance and chroma of the sub-block is derived. In this embodiment, a sub-block based method can be proposed to derive the scaling factors for both the luminance residual and the chroma residual. Compared with the first method, the second method can improve the spatial accuracy of the estimated scaling factor because the low-correlation luminance prediction samples outside the sub-block are excluded from the calculation of the sub-block scaling factor. On the other hand, considering that the second method can also start scaling the luminance residual and chroma residual within one sub-block immediately after the luminance prediction of the sub-block is completed, that is, without waiting for the complete generation of the luminance prediction samples of the entire CU, the delay of luminance and chroma residual reconstruction can be reduced. On the other hand, considering that the second method can also start scaling the luminance residual and chroma residual within one sub-block immediately after the luminance prediction of the sub-block is completed, that is, without waiting for the complete generation of the luminance prediction samples of the entire CU, the delay of luminance and chroma residual reconstruction can be reduced. On the other hand, considering that the second method can also start scaling the luminance residual and chroma residual within one sub-block immediately after the luminance prediction of the sub-block is completed, that is, without waiting for the complete generation of the luminance prediction samples of the entire CU, the delay of luminance and chroma residual reconstruction can be reduced.

[0121] According to a third embodiment of the chroma sample reconstruction procedure, the luminance prediction sample domain value includes the luminance prediction samples arranged at the same location. In this embodiment, the luminance residual scaling method is extended to scale the chroma residual, and different scales of each chroma residual sample In this embodiment, the luminance residual scaling method is extended to scale the chroma residual, and different scales of each chroma residual sample The kering coefficient is derived based on the luminance prediction sample values arranged at the same location .

[0122] In the above embodiment of the chroma sample reconstruction procedure, in order to perform scaling of the chroma residual , it is proposed to use the same LUT as that used for the calculation of the luminance scaling coefficient . In one example, the scaling coefficient Scale C at the CU level of the chroma residual is derived as follows . 1) Calculate the average of the luminance prediction samples (represented in the original domain) within the CU, denoted as avg Y . 2) Find or obtain the corresponding segment index Idx Y of the piecewise linear model to which avg Y belongs . 3) Calculate the value of Scale C as follows Scale C = scaleForward[Idx Y where scaleForward[i] (i = 0...15) is one pre - defined 16 - entry LUT, which is calculated as follows scaleForward[i]=(OrgCW << SCALE_FP_PREC) / SignaledCW[i] where OrgCW and SignaledCW[i] are the number of codewords of the i - th segment in the original domain and the mapped domain respectively, and S CALE_FP_PREC is the accuracy of the scaling coefficient .

[0123] In the above example, the scaling coefficient of the chroma residual is derived for each sub - block of the current CU ​​When it can be extended easily. In that case, in the above first step, av gY is calculated as the average of the luminance prediction samples in the original domain of the sub-block, but steps 2 and 3 remain the same.

[0124] In the second chroma sample reconstruction procedure, as shown in FIG. 11, during the luminance prediction process for the coding unit (CU), skip some pre-defined intermediate luminance prediction stages, so that a plurality of luminance prediction samples are obtained (1101). Using the obtained plurality of luminance prediction samples, derive the scaling coefficients for the chroma residual samples within the CU (11 02), and use the scaling coefficients to scale the chroma residual samples within the CU (1 103). The reconstructed chroma samples are calculated by adding the chroma prediction samples and the scaled chroma residual samples within the CU (1104).

[0125] According to one or more embodiments of the second chroma sample reconstruction procedure, the pre-defined intermediate luminance prediction stages include one or more of decoder-side motion vector derivation (DMVR), bidirectional optical flow (BDOF), and combined inter-intra prediction (CIIP). In these embodiments, to solve the latency problem, use the inter-prediction samples derived before DMVR, BDOF / PROF, CIIP intra / inter combination processing to derive the scaling coefficients of the chroma residuals.

[0126] FIG. 12 shows this embodiment of the second chroma sample reconstruction procedure where DMVR, BDOF, and CIIP are not applied to generate luminance prediction samples for chroma scaling. ​​​A flowchart showing the workflow of LMCS decoding processing in an example. Here , instead of waiting for the complete termination of DMVR 1203, BDOF 1204 and / or the luminance intra prediction of CIIP section 1205, as soon as the prediction samples 1221 and 1222 based on the initial L0 and L1 luminance predictions 1201 and 1202 become available, the chroma residual scaling process 1208 can be started.

[0127] In FIG. 12, before DMVR 1203, BDOF 1204, and / or CIIP 1 205, in addition to the original averaging operation 1206, one additional averaging operation 1211 is required to combine the initial L0 and L1 prediction samples 1221 and 1222 .

[0128] In a second example of this embodiment of the second chroma sample reconstruction procedure to reduce complexity , the initial L0 prediction sample is always used and the scaling coefficient for the chroma residual can be derived.

[0129] FIG. 13 is a flowchart showing the workflow of LMCS decoding processing in a second example of this embodiment of the second chroma sample reconstruction procedure, where the initial single prediction signal is applied to generate the luminance prediction sample for chroma scaling. No additional averaging operation is required in addition to the original averaging operation 1306. The prediction sample 1321 of the initial L0 is used to derive the scaling coefficient of the chroma residual before DMVR 1303, BDOF 1304, and / or CIIP 1305. 1303, BDOF 1304, and / or the s caling coefficient of the chroma residual before CIIP 1305.

[0130] In a third example of this embodiment of the second chroma sample reconstruction procedure, chroma residual scaling As a luminance prediction sample used to derive coefficients, one initial prediction signal (L0 or L1) is adaptively selected. In one possible implementation of this example, among the initial prediction signals (L0 or L1), the one with a picture order count (POC) distance whose reference picture is smaller than the current picture is selected to derive the chroma residual scaling coefficient.

[0131] In another embodiment of the second chroma sample reconstruction procedure, it is proposed to disable only DMVR and BDOF / PROF while enabling CIIP to generate the inter-prediction sample used to determine the chroma residual scaling coefficient. Specifically, in this method, the inter-prediction samples derived before DMVR and BDOF / PROF are first averaged. Next, they are combined with the intra-prediction samples of CIIP. Finally, the combined prediction samples are used as the prediction samples for determining the chroma residual scaling coefficient.

[0132] In yet another embodiment of the second chroma sample reconstruction procedure, it is proposed to disable only BDOF / PROF while maintaining DMVR and CIIP to generate the prediction samples used to determine the chroma residual scaling coefficient.

[0133] In yet another embodiment of the second chroma sample reconstruction procedure, it is proposed to maintain BDOF / PROF and CIIP while disabling DMVR when deriving the luminance prediction samples used to determine the chroma residual scaling coefficient.

[0134] Furthermore, the method in the above embodiment of the second chroma sample reconstruction procedure is shown because it is designed to reduce the delay of chroma prediction residual scaling. It is worth mentioning that those methods can also be used to reduce the delay of luminance prediction residual scaling. For example, all methods can also be applied to the PDRS method described in the section of "Luminance mapping based on prediction-dependent residual scaling". According to the existing DMVR design, in order to reduce the computational complexity, the prediction samples used for DMVR motion fine-tuning are generated using a 2-tap bilinear filter instead of the default 8-tap interpolation. After the corrected motion is determined, the default 8-tap filter is applied to generate the final prediction samples of the current CU. Therefore, in order to reduce the chroma residual decoding delay caused by DMVR, it is proposed to determine the scaling coefficient of the chroma residual using the luminance prediction samples (the average of the L0 and L1 prediction samples when the current CU is bi-predicted) generated by the bilinear filter. According to one chroma residual sample reconstruction procedure, as shown in FIG. 14, one or more luminance prediction sample values are selected (1401) from the output of the bilinear filter of decoder-side motion vector derivation (DMVR), and one or more selected luminance prediction sample values are

[0135] adjusted (1402) to another one or more luminance prediction sample values having the same bit depth as the original encoded bit depth of the input video, and one or more chroma residual samples are recovered using the luminance prediction sample values having the same bit depth as the original encoded bit depth of the input video. After the corrected motion is determined, the default 8-tap filter is applied to generate the final prediction samples of the current CU. Therefore, in order to reduce the chroma residual decoding delay caused by DMVR, it is proposed to determine the scaling coefficient of the chroma residual using the luminance prediction samples (the average of the L0 and L1 prediction samples when the current CU is bi-predicted) generated by the bilinear filter. According to one chroma residual sample reconstruction procedure, as shown in FIG. 14, one or more luminance prediction sample values are selected (1401) from the output of the bilinear filter of decoder-side motion vector derivation (DMVR), and one or more selected luminance prediction sample values are adjusted (1402) to another one or more luminance prediction sample values having the same bit depth as the original encoded bit depth of the input video, and one or more chroma residual samples are recovered using the luminance prediction sample values having the same bit depth as the original encoded bit depth of the input video.

[0136] According to one chroma residual sample reconstruction procedure, as shown in FIG. 14, one or more luminance prediction sample values are selected (1401) from the output of the bilinear filter of decoder-side motion vector derivation (DMVR), and one or more selected luminance prediction sample values are adjusted (1402) to another one or more luminance prediction sample values having the same bit depth as the original encoded bit depth of the input video, and one or more chroma residual samples are recovered using the luminance prediction sample values having the same bit depth as the original encoded bit depth of the input video. prediction sample values having the same bit depth as the original encoded bit depth of the input video, and one or more chroma residual samples are recovered using the luminance prediction sample values having the same bit depth as the original encoded bit depth of the input video. prediction sample values having the same bit depth as the original encoded bit depth of the input video, and one or more chroma residual samples are recovered using the luminance prediction sample values having the same bit depth as the original encoded bit depth of the input video. Derive a scaling factor for scaling (1403), and use the scaling factor to scale one or more chroma residual samples (1404), and one or more chroma residual samples are reconstructed by adding them to one or more scaled chroma residual samples and their corresponding chroma prediction samples (1405).

[0137] In one or more embodiments of the chroma residual sample reconstruction procedure, the step of selecting one or more luminance prediction sample values from the output of the bilinear filter of DMVR includes the step of selecting the luminance prediction samples of L0 and L1 from the output of the bilinear filter of DMVR. This includes the step of selecting the luminance prediction samples of L0 and L1 from the output of the bilinear

[0138] Figure 15 is a flowchart showing the workflow of the LMCS decoding process in such an embodiment of the chroma residual sample reconstruction procedure. The L0 and L1 prediction samples 1521 and 1522 from the output of the bilinear filter component of DMVR 1503 are supplied to the average 1511 in order to derive the chroma residual scaling input 1523 to be used in the chroma residual scaling 1507 for decoding one or more chroma residual samples.

[0139] In these embodiments, there is a problem with the bit code depth. To save the internal memory capacity size used by DMVR, the intermediate L0 and L1 prediction samples generated by the bilinear filter of DMVR are 10-bit accurate. This is different from the representation bit depth of the immediate prediction samples of normal dual prediction which is equivalent to 14 bits. Therefore, due to their different accuracies, the intermediate prediction samples output from the bilinear filter cannot be directly used for chroma residual scaling, filter cannot be directly used for chroma residual scaling due to their different accuracies.​​​​​ It cannot be directly applied to determine the scaling coefficient.

[0140] To address this problem, first, the DMVR intermediate bit depth is made consistent with the intermediate bit depth used in normal motion compensation interpolation, that is, it is proposed to increase the bit depth from 10 bits to 14 bits. Then, the existing averaging process applied to generate the normal dual-prediction signal is reused to generate the corresponding prediction samples for determining the chroma residual scaling coefficient. In one example of these embodiments, adjusting one or more selected luminance prediction sample values to another one or more luminance prediction sample values having the same bit depth as the original encoded bit depth of the input video can increase the internal bit depth of the L0 and L1 luminance prediction samples from the output of the bilinear filter of the DMVR to 14 bits by left shifting, averaging the 14-bit shifted L0 and L1 luminance prediction sample values to obtain a 14-bit average luminance prediction sample value, and converting the 14-bit average luminance prediction sample value by right shifting to change the internal bit depth of the 14-bit average luminance prediction sample value to the original encoded bit depth of the input video. More specifically, in this example, the chroma scaling coefficient is determined by the steps described in the box immediately following this paragraph. 1) Internal bit depth alignment. As shown below, the internal bit depth of the L0 and L1 prediction samples generated by the bilinear filter is increased from 10 bits to 14 bits. It can be used to generate corresponding prediction samples for determining the chroma residual scaling coefficient.

[0141] In an example of these embodiments, adjusting one or more selected luminance prediction sample values to another one or more luminance prediction sample values having the same bit depth as the original encoded bit depth of the input video. The internal bit depth of the L0 and L1 luminance prediction samples from the output of the bilinear filter of the DMVR is increased to 14 bits by left shifting. And averaging the 14-bit shifted L0 and L1 luminance prediction sample values to obtain a 14-bit average luminance prediction sample value. By right shifting, the internal bit depth of the 14-bit average luminance prediction sample value is changed to the original encoded bit depth of the input video. To convert the 14-bit average luminance prediction sample value. This includes obtaining a 14-bit average luminance prediction sample value. And converting the 14-bit average luminance prediction sample value. It includes changing the internal bit depth of the 14-bit average luminance prediction sample value to the original encoded bit depth of the input video.

[0142] More specifically, in this example, the chroma scaling coefficient is determined by the steps described in the box immediately following this paragraph. It is determined by the steps described in the box immediately following this paragraph. 1) Internal bit depth alignment. As shown as follows, the internal bit depth of the L0 and L1 prediction samples generated by the bilinear filter is increased from 10 bits to 14 bits. The internal bit depth of the L0 and L1 prediction samples generated by the bilinear filter is increased from 10 bits to 14 bits. Increase it. [Number] Here, [Number] and [Number] are the predicted samples output from the bilinear filter, [Number] and [Number] are the scaled predicted samples after bit-depth alignment. [Number] is the constant used to compensate for the shifted dynamic range of the predicted samples resulting from the following averaging operation for compensation. 2) Average of the scaled predicted samples of L0 and L1. Chroma residual scaling The final luminance sample used to determine the scaling factor is calculated by averaging the two scaled luminance predicted samples as follows. [Number] Here, the bit depth is the encoded bit depth of the input video.

[0143] In other embodiments of the chroma residual sample reconstruction procedure, one or more luminance predicted sample values are selected from the output of the bilinear filter of the DMVR, and one or more other luminance predicted sample values having the same bit depth as the original encoded bit depth of the input video are added to one or more of the other luminance predicted sample values. The step of adjusting the selected luminance prediction sample value is a bilinear filter of DMVR selecting one of the luminance prediction samples of L0 and L1 from the output of the step of, and changing the internal bit depth of the one luminance prediction value selected by the shift to the original coded bit depth of the input video to adjust one selected luminance prediction sample by changing it to the original coded bit depth of the input video the step of adjusting, and using the luminance prediction sample adjusted as the luminance prediction sample having the same bit depth as the original coded bit depth of the input video including the step of.

[0144] Figure 16 is a flowchart showing the workflow of the LMC S decoding process in such another embodiment of the chroma residual sample reconstruction procedure. The L0 prediction sample 1621 from the output of the bilinear filter 1612 component of the DMVR 1603 is used in the chroma residual scaling 1607 for decoding one or more chroma residual samples. In this embodiment it is proposed to directly use the initial single prediction sample (i.e., the L0 prediction sample) to derive the scaling coefficient of the chroma residual.

[0145] In an example of such another embodiment of the chroma residual sample reconstruction procedure, assuming that the current CU is bi-predicted, the chroma scaling coefficient is determined by shifting the luminance sample output from the bilinear filter to the original coded bit depth of the input video as described in the box immediately following this paragraph. When the bit depth is 10 or less otherwise

Equation

Equation

Equation

[0146] Finally, instead of generating luminance prediction samples, reference samples (i.e., samples at integer positions taken from external reference pictures) are directly used to determine the scaling factor of the chroma residual. It is proposed to use the average of the reference samples at L0 and L1 to determine the chroma residual scaling factor. In one or more embodiments, it is proposed that the chroma residual scaling factor can be calculated using only the reference samples in one direction (e.g., list L0). According to the second chroma residual sample reconstruction procedure, as shown in FIG. 17, one or more luminance reference sample values are selected from the reference picture (1701), one or more selected luminance reference sample values are converted into luminance sample values (1702), the converted luminance samples are used to derive a scaling factor (1703), the scaling factor is used to scale one or more chroma residual samples (1704), and one or more chroma residual samples are reconstructed by adding one or more scaled chroma residual samples and their corresponding chroma prediction samples (1705). In one or more embodiments of the second chroma residual sample reconstruction procedure, the step of selecting one or more luminance reference sample values from the reference picture and converting one or more selected luminance reference sample values into luminance sample values includes obtaining both the luminance reference sample values of L0 and L1 from the L0 and L1 reference pictures, and the converted luminance samples

[0147]

[0148] In one or more embodiments of the second chroma residual sample reconstruction procedure, the step of selecting one or more luminance reference sample values from the reference picture and converting one or more selected luminance reference sample values into luminance sample values includes obtaining both the luminance reference sample values of L0 and L1 from the L0 and L1 reference pictures, and the converted luminance samples ​​​​​​​​​​​ including the step of averaging the luminance reference sample values of L0 and L1 as the ru value.

[0149] In other embodiments of the second chroma residual sample reconstruction procedure, one or a plurality of luminance reference samples are selected from the reference picture, and one or a plurality of selected luminance reference samples are converted into luminance sample values. The step of converting includes selecting one luminance reference sample value from among the L0 and L1 luminance reference sample values from the L0 and L1 reference pictures, and using the one selected luminance reference sample value as the converted luminance sample value.

[0150] According to the existing LMCS design, the reconstructed luminance samples adjacent to the 64x64 region where the current CU is located are used to calculate the chroma residual scaling coefficients of the CUs within the region. Furthermore, one clipping operation, namely Clip1(), is applied to clip the reconstructed luminance adjacent samples within the dynamic range of the internal bit depth ([0, ([1<<bitDepth)-1]) before the average is calculated.

[0151] Specifically, the method first fetches 64 left adjacent luminance samples and 64 upper adjacent luminance samples of the corresponding 64x64 region to which the current CU belongs. Then, the average of the left and upper adjacent samples, i.e., avgY, is calculated, and the segment index Y of avgY within the LMCS piecewise linear model is found. Finally, the chroma residual C = cScaleInv[Y] is derived. 1dx Scal eInv 1dx

[0152] ​​Specifically, in the current VVC draft, how to derive the corresponding average luminance is as follows and is described as follows. The clipping operation Clip1() is applied as shown in prominent font size . For the derivation of the variable varScale, the following sequential steps are applied The variable invAvgLuma is derived as follows - The array recLuma[i] (i = 0(2*sizeY - 1)), and the variable cnt are derived as follows - The variable cnt is set equal to 0 - If availL is equal to TRUE, the array recLuma[i] (i = 0..size Y - 1) is set equal to currPic[xCuCb - 1][Min(yCuCb + i, pic_ height_in_luma_samples - 1)] (i = 0..sizeY - 1) and cnt is set equal to sizeY - If availT is equal to TRUE, the array recLuma[cnt + i] (i = 0 ..sizeY - 1) is set equal to currPic[Min(xCuCb + i, pic_widt h_in_luma_samples - 1)][yCuCb - 1] (i = 0..size Y - 1) and cnt is set equal to (cnt + sizeY) - The variable invAvgLuma is derived as follows - If cnt is greater than 0, the following applies [Number] - Otherwise (cnt is equal to 0), the following applies invAvgLuma = 1 << (BitDepth - 1) In the above description, sizeY is 64. recLuma[i] is the upper and left neighbor It is a reconstructed sample of the luminance sample in contact. invAvgLuma is the calculated luminance average.

[0153] However, in the reconstruction process, after adding the predicted sample to the residual sample of one CU the resulting sample value is already clipped to the dynamic range of the internal bit depth This means that all adjacent reconstructed luminance samples around the current 64x64 region are guaranteed to be within the range of the internal bit depth. Therefore, their average, that is, avgY cannot exceed this range either. As a result, the existing clipping (i.e., Clip 1()) does not need to calculate the corresponding chroma residual scaling factor. To further reduce the complexity and memory requirements of the LMCS design, it is proposed to remove the clipping operation and derive the chroma residual scaling factor since calculating the average of adjacent reconstructed luminance samples Figure 18 is a flowchart showing the steps of the non-clipping chroma residual scaling factor derivation procedure. In Figure 18, a plurality of reconstructed luminance samples from a first predetermined region adjacent to a second predetermined region where the CU is located are selected (1801) during the decoding of the CU, the average of the plurality of reconstructed luminance samples is calculated (1802), and the average of the plurality of reconstructed luminance samples is directly used without clipping when deriving the chroma residual scaling factor for decoding the CU (1803).

[0154] In one or more embodiments of the non-clipping chroma residual scaling factor derivation procedure, the average of the plurality of reconstructed luminance samples is the arithmetic mean of the plurality of reconstructed luminance samples.

[0155] ​​​​​​​

[0156] In one or more embodiments of the non-clipping chroma residual scaling coefficient derivation procedure, when deriving the chroma residual scaling coefficient for decoding the CU, the step of directly using the average of a plurality of reconstructed luminance samples without clipping comprises identifying the segment index of the average in a predefined segmented model and deriving the chroma residual scaling coefficient for decoding the CU based on the gradient of the segment model.

[0157] In one or more embodiments of the non-clipping chroma residual scaling coefficient derivation procedure, generating luminance prediction samples and luminance residual samples within a first predetermined region, adding the luminance residual samples to the luminance prediction samples, and clipping the added luminance samples to the dynamic range of the coded bit depth, generates a plurality of reconstructed luminance samples within the first predetermined region.

[0158] In one or more embodiments of the non-clipping chroma residual scaling coefficient derivation procedure, the plurality of reconstructed luminance samples are a plurality of forward mapped inter-luminance reconstruction samples.

[0159] In one or more embodiments of the non-clipping chroma residual scaling coefficient derivation procedure, the second predetermined region is the 64×64 region in which the CU is located.

[0160] In one example, as shown in FIG. 19, the first predetermined region may include upper adjacent samples within a 1x64 region 1902 directly above the second predetermined region 1904. Alternatively, or in addition to this, ​​​​​In addition, the first predetermined area is a 64x1 area immediately to the left of the second predetermined area 1904. 1903.

[0161] According to the existing LMCS design, reconstruction of both the original domain and the mapped domain is The generated samples are used for CUs coded in different modes. Several LMCS transformations are used to convert the predicted and reconstructed luminance samples between the two domains. conversion is involved in the current encoding / decoding process.

[0162] Specifically, in the intra mode, CIIP mode, and IBC mode, The neighboring reconstruction domains of one current CU used to generate the prediction samples These reference samples are maintained within the mapped domain. In contrast, the CIIP model For both 1D and all inter modes, motion compensated prediction is generated from temporal reference pictures. The samples are in the original domain. The intensity reconstruction operation is performed in the mapped domain. Therefore, those inter-predicted samples of the luma component are added together before being added with the residual samples. On the other hand, intra and inter-domain For both domains, reverse mapping converts the mapped domain to the original domain. The filter is always applied to the reconstructed luma samples that have been filtered.

[0163] Furthermore, the clipping operation, i.e. Clip1(), clips the inter predicted samples as After being converted to the mapped domain, the dynamic range of the internal bit depth (all That is, [0,(1< <bitDepth)-1]の範囲内にクリッピングするために適 is used. On the other hand, for both intra and inter modes, after the reconstructed luminance sample pull is converted to the original domain, the same clipping operation is applied to the reconstructed luminance sample as well.

[0164] However, based on the existing LMCS design, there is a bitstream constraint to ensure that the samples obtained from the forward LMCS are always within the dynamic range of the internal bit depth. This means that the mapped luminance prediction samples of the inter CU cannot exceed such a dynamic range. Therefore, the existing clipping operation applied to the mapped luminance prediction samples in the inter mode is redundant. As an example, it can be proposed to remove the clipping operation after the forward conversion of the inter prediction samples in the inter mode and the CIIP mode. In another example, it can be proposed to remove the clipping operation from the inverse LMCS mapping process when converting the reconstructed luminance sample from the mapped domain to the original domain. is redundant. As an example, it can be proposed to remove the clipping operation after the forward conversion of the inter prediction samples in the inter mode and the CIIP mode. In another example, it can be proposed to remove the clipping operation from the inverse LMCS mapping process when converting the reconstructed luminance sample from the mapped domain to the original domain. More specifically, in the current VVC draft, the inverse mapping process of the luminance sample is described as follows, and the clipping operation Clip1() in Equation (1242) is applied as shown in the prominent font size. 8.8.2.2 Inverse mapping process of luminance samples The input to this process is the luminance sample lumaSample. The output of this process is the modified luminance sample invLumaSample.

[0165] The value of invLumaSample is derived as follows. - slice_lmcs_enab of the slice containing the luminance sample lumaSample is applied as shown in the prominent font size. 8.8.2.2 Inverse mapping process of luminance samples The input to this process is the luminance sample lumaSample. The output of this process is the modified luminance sample invLumaSample. The value of invLumaSample is derived as follows. - slice_lmcs_enab of the slice containing the luminance sample lumaSample When led_flag is equal to 1, the following ordered steps are applied. 1. The variable idxYInv is derived by calling the identification of the luminance sample partitioning function index process, as specified in Clause 8.8.2.3, with lumaSample as the input and idxYInv as the output. 2. The variable invSample is derived as follows.

Number

Number

[0166] Furthermore, in the current VVC draft, the weighted sample prediction process for merge and intra prediction is described as follows, where the clipping operation Clip 1() is applied as shown in the prominent font size. 8.5.6.7 Weighted Sample Prediction Process for Merge and Intra Prediction The input to this process is as follows. - The luminance position (xCb, yCb) that specifies the top-left sample of the current luminance coding block for the top-left luminance sample of the current picture - The width cbWidth of the current coding block, - The height cbHeight of the current coding block - Two (cbWidth) x (cbHeight) arrays predSamplesIn ter and predSamplesIntra - Variable cIdx that specifies the color component index The output of this process is a (cbWidth) × (cbHeight) array of predicted sample values predSamplesComb. The variable scallFact is derived as follows. [Number] The adjacent luminance positions (xNbA, yNbA) and (xNbB, yNbB) are respectively , (xCb - 1, yCb - 1+(cbHeight << scallFactY)) and (xCb - 1+(cbWidth << scallFactX), yCb - 1), are equally set. When X is replaced by either A or B, the variables availableX and i sIntraCodedNeighbourX are derived as follows. - In the derivation process of adjacent block availability specified in clause 6.4.4, the position (xCurr, yCurr) set equal to (xCb, yCb ), the adjacent position (xNbX, yNbX) set equal to, the adjacent position (xNbY, yNbY) set equal to FALSE, the chec kPredModeY set equal to FALSE, and the cIdx set equal to 0, are called as inputs , and the output is assigned to availableX. - The variable isIntraCodedNeighbourX is derived as follows. - When availableX is equal to TRUE and CuPredMode[0][xNbX [yNbX] is equal to MODE_INTRA, isIntraCodedNei ghbourX is set equal to TRUE. - Otherwise, isIntraCodedNeighbourX is set equal to FALSE . The weight w is derived as follows. - When both isIntraCodedNeighbourA and isIntraCodedNeighbourB are equal to TRUE, w is set equal to 3. - Otherwise, when both isIntraCodedNeighbourA and isIntraCodedNeighbourB are equal to FALSE, w is set equal to 1. - Otherwise, w is set equal to 2. When cIdx is equal to 0 and slice_lmcs_enabled_flag is equal to 1, predSamplesInter[x][y] for x = 0..cbWidth-1 and y = 0..cbHeight-1 is modified as follows. The prediction samples predSamplesComb[x][y] for x = 0..cbWidth-1 and y = 0..cbHeight-1 are derived as follows.

Number

Number

[0167] Furthermore, in the current VVC draft, the picture reconstruction using the mapping process of the luminance samples is described as follows, and the clipping operation Clip1() is applied as shown in prominent fonts. 8.7.5.2 Picture Reconstruction by Mapping Process of Luminance Samples The input to this process is as follows. - The position (xCurr, yCurr) of the top-left sample of the current block relative to the top-left sample of the current picture - The variable nCurrSw that defines the block width ​​​​​​​​​ - Variable nCurrSh that specifies the block height - (nCurrSw) x (nCurrS h) array predSamples that specifies the luminance prediction samples of the current block - (nCurrSw) x (nCurrS h) array resSamples that specifies the luminance residual samples of the current block The output of this process is the reconstructed luminance picture sample array recSamples There is. The (nCurrSw ) × (nCurrSh) array of the mapped predicted luminance samples predMapSamples is derived as follows. - If one of the following conditions is true, predMapSamples[i][j is set equal to pred Samples[i][j] for i = 0..nCurrSw - 1, j = 0..nCurrSh - 1. - CuPredMode[0][xCurr][yCurr] is MODE_INTRA Equal to. - CuPredMode[0][xCurr][yCurr] is MODE_IBC and Equal to. - CuPredMode[0][xCurr][yCurr] is MODE_PLT and Equal to. - CuPredMode[0][xCurr][yCurr] is MODE_INTER Equal to, and ciip_flag[xCurr][yCurr] is equal to 1. - Otherwise (CuPredMode[0][xCurr][yCurr] is MO Equal to DE_INTER, and ciip_flag[xCurr][yCurr] is equal to 0 ), the following applies.

Number

Equation

[0168] These redundant clipping operations incur computational complexity in existing LMCS designs and additional requirements for on-chip memory. To further reduce the complexity and memory requirements of the LMCS design, it is proposed to remove these redundant clipping operations and memory requirements. To further reduce the complexity and memory requirements of the LMCS design, it is proposed to remove these redundant clipping operations and memory requirements. To further reduce the complexity and memory requirements of the LMCS design, it is proposed to remove these redundant clipping operations

[0169] According to the non-clipping chroma sample decoding procedure, as shown in FIG. 20, during the decoding of an encoded unit (CU) encoded by the inter mode or combined inter-intra prediction (CIIP) mode under the luminance mapping (LMCS) framework with chroma scaling a plurality of reconstructed samples of the luminance component are obtained (2001) within the mapped domain, and by converting a plurality of reconstructed samples of the luminance component from the mapped domain back to the original domain, a plurality of converted samples of the luminance component are obtained (2002) within the original domain, and when deriving the chroma scaling coefficients for decoding the chroma samples of the CU a plurality of samples of the converted luminance component in the original domain are used (2003) without clipping a plurality of reconstructed samples of the luminance component are obtained (2001) within the mapped domain, and by converting a plurality of reconstructed samples of the luminance component from the mapped domain back to the original domain, a plurality of converted samples of the luminance component are obtained (2002) within the original domain, and when deriving the chroma scaling coefficients for decoding the chroma samples of the CU a plurality of reconstructed samples of the luminance component are obtained (2001) within the mapped domain, and by converting a plurality of reconstructed samples of the luminance component from the mapped domain back to the original domain, a plurality of converted samples of the luminance component are obtained (2002) within the original domain, and when deriving the chroma scaling coefficients for decoding the chroma samples of the CU a plurality of samples of the converted luminance component in the original domain are used (2003) without clipping a plurality of samples of the converted luminance component in the original domain are used (2003) without clipping a plurality of samples of the converted luminance component in the original domain are used (2003) without clipping a plurality of samples of the converted luminance component in the original domain are used (2003) without clipping

[0170] In one or more embodiments of the non-clipping chroma sample decoding procedure, as shown in FIG. 21 When the CU is encoded in inter mode, the step of obtaining a plurality of reconstructed samples of the luminance component in the mapped domain includes the step (2101) of calculating a plurality of inter-prediction samples of the luminance component in the original domain and the step (2102) of converting, without clipping, a plurality of inter-prediction samples of the luminance component from the original domain to the mapped domain to obtain a plurality of converted inter-prediction samples of the luminance component in the mapped domain, and the step (2103) of adding the plurality of converted inter-prediction samples of the luminance component in the mapped domain to a plurality of residual samples of the luminance component in the mapped domain, and as a result, obtaining a plurality of reconstructed samples of the luminance component in the mapped domain. In one or more other embodiments of the non-clipping chroma sample decoding procedure, as shown in FIG. 22, when the CU is encoded in CIIP mode, the step of obtaining a plurality of reconstructed samples of the luminance component in the mapped domain includes the step (2201) of calculating a plurality of inter-prediction samples of the luminance component in the original domain and the step (2202) of converting, without clipping, a plurality of inter-prediction samples of the luminance component from the original domain to the mapped domain to obtain a plurality of converted inter-prediction samples of the luminance component in the mapped domain, and

[0171] the step (2203) of calculating a plurality of intra-prediction samples of the luminance component in the mapped domain, and the step of adding the plurality of converted inter-prediction samples and the plurality of intra-prediction samples, and as a result, obtaining a plurality of reconstructed samples of the luminance component in the mapped domain. In the original domain, the step of obtaining a plurality of converted inter-prediction samples of the luminance component in the mapped domain includes the step of converting, without clipping, a plurality of inter-prediction samples of the luminance component from the original domain to the mapped domain, and the step (2203) of calculating a plurality of intra-prediction samples of the luminance component in the mapped domain, and the step of adding the plurality of converted inter-prediction samples and the plurality of intra-prediction samples, and Derive a predicted sample of the luminance component in the mapped domain by weighted average Step (2204), and Add the derived predicted sample of the luminance component in the mapped domain to a plurality of residual samples of the luminance component in the mapped domain, resulting in a plurality of reconstructed samples of the luminance component in the mapped domain, step (2 205), and includes.

[0172] When the current LMCS is enabled, the forward and reverse luminance mappings are performed using one LUT table defined with an accuracy of 11 bits . For example, taking the forward luminance mapping as an example, the current forward mapping scaling coefficient is defined as follows . It is defined as follows ScaleCoeff[i]=(lmcsCW[i]*(1<<11)+(1<<(L og 2(OrgCW)-1)))>>(Log 2(OrgCW)) where lmcsCW is the length of one segment of the mapped luminance domain , and OrgCW is the length of one segment of the original luminance domain equal to 1<<(BitDepth-4) .

[0173] However, such an accuracy of 11 bits has been found to be sufficient only for internal coding bit depths less than 16 bits . When the internal coding bit depth is 16 bits, the value of L og 2(OrgCW) becomes 12. In such a case, the improvement in 11-bit accuracy is not sufficient to support scaling coefficient derivation. Thus, when the current bitstream conformance is applied, i.e., in the segments in the mapped luminance domain Even when the total length of the ment is less than or equal to (1<<BitDepth)-1, the forward luminance The predicted luminance sample value after mapping may exceed the dynamic range of the internal coding bit depth.

[0174] Based on such considerations, two solutions are proposed.

[0175] In one solution, it is proposed to always apply a clipping operation to the mapped luminance predictions in the inter mode and the CIIP mode. Moreover, the current bit stream compatibility can be removed.

[0176] Under this solution, the weighted sample prediction process for merge and intra prediction is as follows. Compared with the specification of the same procedure in the current VVC draft, the clipping operation Clip1 () is always applied to the mapped luminance prediction sample so that it includes the clipping value in Equation (1028a). 8.5.6.7 Weighted Sample Prediction Process for Merge and Intra Prediction The input to this process is as follows. When cIdx is equal to 0 and slice_lmcs_enabled_flag is equal to 1, for x = 0..cbWidth-1 and y = 0..cbHeight-1, predSamplesInter[x][y] is modified as follows.

Equation

Equation

[0177] Even under this solution, picture reconstruction using the mapping process of the luminance samples is as follows. Compared with the specification of the same procedure in the current VVC draft, a clipping operation Clip1() is added so that the mapped luminance prediction samples include the clipping values. 8.7.5.2 Picture Reconstruction by Luminance Sample Mapping Process The input to this process is as follows. - The position (xCurr, yCurr) of the top-left sample of the current block relative to the top-left sample of the current picture 8.7.5.2 Picture Reconstruction by Luminance Sample Mapping Process The input to this process is as follows. - The position (xCurr, yCurr) of the top-left sample of the current block relative to the top-left sample of the current picture Curr, yCurr), - The variable nCurrSw defining the block width - The variable nCurrSh specifying the block height - The (nCurrSw) x (nCurrSh) array predSamples specifying the luminance prediction samples of the current block h) array predSamples - The (nCurrSw) x (nCurrSh) array resSamples specifying the luminance residual samples of the current block h) array resSamples The output of this process is the reconstructed luminance picture sample array recSamples. The (nCurrSw )×(nCurrSh) array of the mapped predicted luminance samples predMapSamples is derived as follows. - If one of the following conditions is true, predMapSamples[i][j is set equal to pred Samples[i][j] for i = 0..nCurrSw - 1, j = 0..nCurrSh - 1. - CuPredMode[0][xCurr][yCurr] is equal to MODE_INTRA - CuPredMode[0][xCurr][yCurr] is equal to MODE_INTRA and equal. -CuPredMode[0][xCurr][yCurr] is equal to MODE_IBC and so on. -CuPredMode[0][xCurr][yCurr] is equal to MODE_PLT and so on. -CuPredMode[0][xCurr][yCurr] is equal to MODE_INTER and ciip_flag[xCurr][yCurr] is equal to 1. - Otherwise (CuPredMode[0][xCurr][yCurr] is equal to MO DE_INTER and ciip_flag[xCurr][yCurr] is equal to 0) the following applies.

Number

Number

[0178] In the second solution, it is proposed to increase the accuracy of the derivation of the LMCS scaling factor beyond 11 bits (denoted as M bits) .

[0179] Under this second solution, the forward luminance mapping is as follows. ScaleCoeff[i]=(lmcsCW[i]*(1<<M)+(1<<(Lo g 2(OrgCW)-1)))>>(Log 2(OrgCW)) idxY=predSamples[i][j]>>Log2(OrgCW) predMapSamples[i][j]=LmcsPivot[idxY]+ (ScaleCoeff[idxY]*(predSamples[i][j]-In putPivot[idxY])+(1<<(M-1)))>>M(1219) with i=0..nCurrSw-1,j=0..nCurrSh-1

[0180] In one embodiment of the second solution, scaling of both forward and inverse luminance mapping is proposed to improve the accuracy of coefficient derivation. In another embodiment of the second solution, only improving the accuracy of scaling coefficient derivation of forward luminance mapping is proposed.

[0181] When either of the above two solutions is applied, the current clipping operation applied to the mapped luminance prediction samples in inter mode and CIIP mode can also be safely removed .

[0182] According to the first aspect of the present disclosure, as shown in FIG. 23, a plurality of prediction samples in the mapped domain of the luminance component of a CU encoded by inter mode or CIIP mode under the LMCS framework are obtained (2301), and a plurality of residual samples in the mapped domain of the luminance component of the CU are received from the bitstream (2302), a plurality of prediction samples in the mapped domain are added to a plurality of residual samples in the mapped domain, and as a result, a plurality of reconstruction samples in the mapped domain of the luminance component are obtained (2303), and a plurality of reconstruction samples of the luminance component are based on a plurality of predefined inverse mapping scaling coefficients . is converted from the mapped domain to the original domain (2304).

[0183] In one or more embodiments of the first aspect of the present disclosure, as illustrated in FIG. 24, C U is encoded by an inter mode, and obtaining a plurality of prediction samples in the mapped domain of the luminance component of CU is to derive a plurality of inter prediction samples of the luminance component of CU from the temporal reference picture of CU in the original domain (2401) and then, based on a predefined coding bit depth and a plurality of predefined forward mapping scaling coefficients within a predefined forward mapping accuracy transforming a plurality of inter prediction samples of the luminance component from the original domain to the mapped domain (2402). This includes and

[0184] In one or more other embodiments of the first aspect of the present disclosure, as illustrated in FIG. 25 CU is encoded by the CIIP mode, and obtaining a plurality of prediction samples in the mapped domain of the luminance component of CU is to derive a plurality of inter prediction samples of the luminance component of CU from the temporal reference picture of CU in the original domain (250 1) and, based on a predefined coding bit depth and a predefined forward mapping accuracy within a predefined plurality of forward mapping scaling coefficients, transforming a plurality of inter prediction samples of the luminance component from the original domain to the mapped domain (2502) and calculating a plurality of intra prediction samples in the mapped domain of the luminance component of CU (2503) and transforming the plurality of converted inter prediction samples ​​Deriving, as a weighted average of a pull and a plurality of intra prediction samples, a prediction sample of the luminance component of a CU in the mapped domain (2504). includes.

[0185] In one example, as shown in FIG. 26, based on a predefined coding bit depth and a plurality of predefined forward mapping scaling factors within a predefined forward mapping accuracy, converting a plurality of inter prediction samples of the luminance component from the original domain to the mapped domain means using the plurality of predefined forward mapping scaling factors to convert a plurality of inter prediction samples of the luminance component from the original domain to the mapped domain (2601), determining whether a clipping operation is necessary based on the predefined coding bit depth and the predefined forward mapping accuracy (2602), and in response to the determination that a clipping operation is necessary, clipping a plurality of inter prediction samples of the luminance component in the mapped domain to the predefined coding bit depth (2603), and bypassing the clipping of a plurality of inter prediction samples of the luminance component in response to the determination that a clipping operation is not necessary (2604). predefined forward mapping scaling factors from the original domain to the mapped domain Converting a plurality of inter prediction samples of the luminance component from the original domain to the mapped domain using the plurality of predefined forward mapping scaling factors (2601), determining whether a clipping operation is necessary based on the predefined coding bit depth and the predefined forward mapping accuracy (2602), and in response to the determination that a clipping operation is necessary, clipping a plurality of inter prediction samples of the luminance component in the mapped domain to the predefined coding bit depth (2603), and bypassing the clipping of a plurality of inter prediction samples of the luminance component in response to the determination that a clipping operation is not necessary (2604). from the original domain to the mapped domain using the plurality of predefined forward mapping scaling factors, determining whether a clipping operation is necessary based on the predefined coding bit depth and the predefined forward mapping accuracy (2602), and in response to the determination that a clipping operation is necessary, clipping a plurality of inter prediction samples of the luminance component in the mapped domain to the predefined coding bit depth (2603), and bypassing the clipping of a plurality of inter prediction samples of the luminance component in response to the determination that a clipping operation is not necessary (2604). Based on the predefined coding bit depth and the predefined forward mapping accuracy, determining whether a clipping operation is necessary If a clipping operation is necessary, then In response to the determination that a clipping operation is necessary, clipping a plurality of inter prediction samples of the luminance component in the mapped domain to the predefined coding bit depth (2603), and bypassing the clipping of a plurality of inter prediction samples of the luminance component in response to the determination that a clipping operation is not necessary (2604). In response to the determination that a clipping operation is not necessary, bypassing the clipping of a plurality of inter prediction samples of the luminance component includes.

[0186] In one or more cases, determining whether a clipping operation is necessary includes determining that a clipping operation is necessary if the predefined coding bit depth is greater than the predefined forward mapping accuracy .

[0187] In one or more cases, determining whether a clipping operation is necessary includes determining that a clipping operation is not necessary when a predefined encoded bit depth is less than a predefined forward mapping accuracy.

[0188] In one or more examples, determining whether a clipping operation is necessary regardless of a predefined encoded bit depth and a predefined forward mapping accuracy includes determining that a clipping operation is necessary.

[0189] In one or more examples, determining whether a clipping operation is necessary regardless of a predefined encoded bit depth and a predefined forward mapping accuracy includes determining that a clipping operation is not necessary.

[0190] In one or more examples, the predefined forward mapping accuracy is 15 bits.

[0191] In one or more examples, the predefined forward mapping accuracy is 11 bits.

[0192] FIG. 27 illustrates a block diagram of an apparatus for video encoding according to some implementations of the present disclosure. The apparatus 2700 may be a terminal such as a mobile phone, a tablet computer, a digital broadcast terminal, a tablet device, or a personal digital assistant.

[0193] As shown in FIG. 27, the apparatus 2700 may include one or more of the following components: a processing component 2702, a memory 2704, a power supply component 2706, a multimedia component 2708 , an audio component 2710, an input / output (I / O) interface 2712, a sensor component 2714, and a communication component 2716.

[0194] ​​​The processing component 2702 generally controls the overall operations of the device 2700, such as operations related to display, call, data communication, camera operation, and recording operation. The processing component 2702 can include one or more processors 2720 to execute instructions to complete all or part of the steps of the above method. Further, the processing component 2702 can include one or more modules to facilitate the interaction between the processing component 2702 and other components. For example, the processing component 2702 can include a multimedia module to facilitate the interaction between the multimedia component 2708 and the processing component 2702.

[0195] The memory 2704 is configured to store different types of data to support the operations of the device 2700. Examples of such data include instructions for any application or method operating on the device 2700, contact data, phone book data, messages, photos, videos, etc. The memory 2704 can include any type of temporary or non - temporary storage medium or a combination thereof, and the memory 2704 can be a static random - access memory (SRAM), an electrically erasable programmable read - only memory (EEPROM), an erasable programmable read - only memory (EPROM), a programmable read - only memory (PROM), a read - only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or a compact disk. The non - temporary storage medium can be, for example, a hard disk drive (HDD), a solid - state drive (SSD), a flash memory, a hybrid drive, or a solid - state hybrid (EEPROM), an erasable programmable read - only memory (EPROM), a programmable read - only memory (PROM), a read - only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or a compact disk. The non - temporary storage medium can be, for example, a hard disk drive (HDD), a solid - state drive (SSD), a flash memory, a hybrid drive, or a solid - state hybrid (SSD), a flash memory, a hybrid drive, or a solid - state hybrid drive. drive. Solid State Hybrid Drive (SSHD), Read Only Memory (ROM), Compact Disc Read Only Memory (CD-ROM), magnetic tape, floppy (registered trademark) disk, etc. may be used.

[0196] The power supply component 2706 supplies power to different components of the device 2700. The power supply component 2706 can include a power management system, one or more power supplies, and other components related to the generation, management, and distribution of power for the device 2700.

[0197] The multimedia component 2708 includes a screen that provides an output interface between the device 2700 and the user. In some examples, the screen can include a Liquid Crystal Display (LCD) and a Touch Panel (TP). When the screen includes a touch panel, the screen may be implemented as a touch screen that receives input signals from the user. The touch panel can include one or more touch sensors for detecting touches, swipes, and gestures on the touch panel. The touch sensors can not only sense the boundaries of touch or swipe operations, but also detect the duration and pressure associated with touch or swipe operations. In some examples, the multimedia component 2708 can include a front camera and / or a rear camera. When the device 2700 is in an operation mode such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. mode, the front camera and / or the rear camera can receive external multimedia data.

[0198] The audio component 2710 is configured to output and / or input audio signals. It is composed of. For example, the audio component 2710 includes a microphone (MIC). When the device 2700 is in an operating mode such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal may be further stored in the memory 2704 or transmitted via the communication component 27 16. In some examples, the audio component 2710 further includes a speaker for outputting an audio signal.

[0199] The I / O interface 2712 provides an interface between the processing component 2702 and the peripheral interface module. The above-mentioned peripheral interface module may be a keyboard, a click wheel, buttons, etc. These buttons can include, but are not limited to, home buttons, volume buttons, start buttons, and lock buttons.

[0200] The sensor component 2714 includes one or more sensors for providing state evaluation in different aspects of the device 2700. For example, the sensor component 2714 can detect the on / off state of the device 2700 and the relative position of the components. For example, the components are the display and the keypad of the device 2700. The sensor component 2714 can also detect the position change of the device 2700 or the components of the device 2700, the presence or absence of user contact on the device 2700, the orientation or acceleration / deceleration of the device 2700, and the temperature change of the device 2700. The sensor component 2714 can include a proximity sensor configured to detect the presence of nearby objects without physical contact. The sensor component 2714 can include an imaging component. further includes an optical sensor such as a CMOS or CCD image sensor used for the purpose and can. In some examples, the sensor component 2714 can further include an acceleration sensor, gyro scope sensor, magnetic sensor, pressure sensor, or temperature sensor.

[0201] The communication component 2716 is configured to facilitate wired or wireless communication between the device 2700 and other devices. The device 2700 can access a wireless network based on communication standards such as WiFi, 4G, or a combination thereof. In one example, the communication component 2716 can receive a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In one example, the communication component 2716 can further include a Near Field Communication (NFC) module for facilitating short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra-Wideband (UWB) technology, Bluetooth (R) (BT) technology, and other technologies and can be implemented. and can be implemented based on the above technologies. and can be implemented. and can be implemented.

[0202] In one example, the device 2700 can be implemented by one or more of an Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, micro controller, microprocessor, or other electronic elements that execute the above methods. and can be implemented by one or more of the others.

[0203] In one or more examples, the described functions can be implemented in hardware, software, firmware It can be implemented in software, or any combination thereof. In the case of software implementation, the function can be stored in a computer-readable medium or transmitted via a computer-readable medium as one or more instructions or codes and executed by a hardware-based processing unit. The computer-readable medium can include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium, or a communication medium including any medium that facilitates the transfer of a computer program from one location to another, for example, in accordance with a communication protocol. In this way, the computer-readable medium can generally correspond to (1) a tangible computer-readable storage medium that is non-transitory, or (2) a communication medium such as a signal or a carrier wave. The data storage medium can be any available medium that can be accessed by one or more computers or one or more processors to retrieve the instructions, codes, and / or data structures for implementing the implementation forms described in this application. A computer program product can include a computer-readable medium. Furthermore, the above method can be implemented using the apparatus 2700. The present disclosure can include dedicated hardware implementations such as application-specific integrated circuits, programmable logic arrays, and other hardware devices. The hardware implementation can be configured to implement one or more of the methods described herein. Examples that can include various implementation forms of devices and systems can include various electronic and computing systems. For example, in accordance with a communication protocol, any medium that facilitates the transfer of a computer program from one location to another. In this way, the computer-readable medium can generally correspond to (1) a tangible computer-readable storage medium that is non-transitory, or (2) a communication medium such as a signal or a carrier wave. The data storage medium can be any available medium that can be accessed by one or more computers or one or more processors to retrieve the instructions, codes, and / or data structures for implementing the implementation forms described in this application. A computer program product can include a computer-readable medium. In this way, the computer-readable medium can generally correspond to (1) a tangible computer-readable storage medium that is non-transitory, or (2) a communication medium such as a signal or a carrier wave. The data storage medium can be any available medium that can be accessed by one or more computers or one or more processors to retrieve the instructions, codes, and / or data structures for implementing the implementation forms described in this application. A computer program product can include a computer-readable medium. In this way, the computer-readable medium can generally correspond to (1) a tangible computer-readable storage medium that is non-transitory, or (2) a communication medium such as a signal or a carrier wave. The data storage medium can be any available medium that can be accessed by one or more computers or one or more processors to retrieve the instructions, codes, and / or data structures for implementing the implementation forms described in this application. A computer program product can include a computer-readable medium. The data storage medium can be any available medium that can be accessed by one or more computers or one or more processors to retrieve the instructions, codes, and / or data structures for implementing the implementation forms described in this application. A computer program product can include a computer-readable medium. Furthermore, the above method can be implemented using the apparatus 2700. The present disclosure can include dedicated hardware implementations such as application-specific integrated circuits, programmable logic arrays, and other hardware devices. The hardware implementation can be configured to implement one or more of the methods described herein. Examples that can include various implementation forms of devices and systems can include various electronic and computing systems. Furthermore, the above method can be implemented using the apparatus 2700. The present disclosure can include dedicated hardware implementations such as application-specific integrated circuits, programmable logic arrays, and other hardware devices. The hardware implementation can be configured to implement one or more of the methods described herein. Examples that can include various implementation forms of devices and systems can include various electronic and computing systems. Furthermore, the above method can be implemented using the apparatus 2700. The present disclosure can include dedicated hardware implementations such as application-specific integrated circuits, programmable logic arrays, and other hardware devices. The hardware implementation can be configured to implement one or more of the methods described herein. Examples that can include various implementation forms of devices and systems can include various electronic and computing systems. Furthermore, the above method can be implemented using the apparatus 2700. The present disclosure can include dedicated hardware implementations such as application-specific integrated circuits, programmable logic arrays, and other hardware devices. The hardware implementation can be configured to implement one or more of the methods described herein. Examples that can include various implementation forms of devices and systems can include various electronic and computing systems.

[0204] Furthermore, the above method can be implemented using the apparatus 2700. The present disclosure can include dedicated hardware implementations such as application-specific integrated circuits, programmable logic arrays, and other hardware devices. The hardware implementation can be configured to implement one or more of the methods described herein. Examples that can include various implementation forms of devices and systems can include various electronic and computing systems. Furthermore, the above method can be implemented using the apparatus 2700. The present disclosure can include dedicated hardware implementations such as application-specific integrated circuits, programmable logic arrays, and other hardware devices. The hardware implementation can be configured to implement one or more of the methods described herein. Examples that can include various implementation forms of devices and systems can include various electronic and computing systems. Furthermore, the above method can be implemented using the apparatus 2700. The present disclosure can include dedicated hardware implementations such as application-specific integrated circuits, programmable logic arrays, and other hardware devices. The hardware implementation can be configured to implement one or more of the methods described herein. Examples that can include various implementation forms of devices and systems can include various electronic and computing systems. Furthermore, the above method can be implemented using the apparatus 2700. The present disclosure can include dedicated hardware implementations such as application-specific integrated circuits, programmable logic arrays, and other hardware devices. The hardware implementation can be configured to implement one or more of the methods described herein. Examples that can include various implementation forms of devices and systems can include various electronic and computing systems. Furthermore, the above method can be implemented using the apparatus 2700. The present disclosure can include dedicated hardware implementations such as application-specific integrated circuits, programmable logic arrays, and other hardware devices. The hardware implementation can be configured to implement one or more of the methods described herein. Examples that can include various implementation forms of devices and systems can include various electronic and computing systems. can be widely included. One or more examples described in this specification are between modules and through modules, or as part of an application-specific integrated circuit, can function with two or more specific interconnected hardware modules or devices having related control and data signals. Therefore, the disclosed apparatus or system can include software, firmware, and hardware implementations. The terms "module", "sub-module", "circuit", "sub-circuit" ,"circuitry", "sub-circuitry", " "unit", or "sub-unit" can include a memory (shared, dedicated, or grouped) that stores code or instructions that can be executed by one or more processors. The modules referred to in this specification can include one or more circuits, regardless of the presence of stored code or instructions. A module or circuit can include one or more connected components and can include one or more circuits. A module or circuit can include one or more connected components and can include

[0205] Other embodiments of the invention will be apparent to those skilled in the art from the consideration of this specification and the practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention that depart from the present disclosure but are within the known or customary practice in the art. This specification and the examples are intended to be considered only as examples, and the true scope and spirit of the invention are set forth It is indicated by the range of demand.

[0206] The present invention is not limited to the exact examples described above and shown in the accompanying drawings, and it will be understood that various modifications and changes can be made without departing from its scope. The scope of the present invention is intended to be limited only by the appended claims. It will be understood that various modifications and changes can be made without departing from its scope. The scope of the present invention is intended to be limited only by the appended claims.

Claims

1. Obtaining a plurality of prediction samples in a mapped domain of a luminance component of an encoded unit (CU) encoded by an inter mode under a framework of luminance mapping with chroma scaling (LMCS); Obtaining a plurality of residual samples in the mapped domain of the luminance component of the CU; Adding the plurality of prediction samples in the mapped domain to the plurality of residual samples in the mapped domain to obtain a plurality of reconstructed samples of the luminance component in the mapped domain; Converting the plurality of reconstructed samples of the luminance component from the mapped domain to the original domain based on a plurality of predefined inverse mapping scaling coefficients; comprising; Obtaining the plurality of prediction samples in the mapped domain of the luminance component of the CU comprises: Deriving a plurality of inter prediction samples of the luminance component of the CU in the original domain from a temporal reference picture of the CU; Converting the plurality of inter prediction samples of the luminance component from the original domain to the mapped domain based on a predefined encoding bit depth and a plurality of predefined forward mapping scaling coefficients within a predefined forward mapping accuracy; A method for video decoding.

2. Converting the plurality of inter prediction samples of the luminance component from the original domain to the mapped domain based on the predefined encoding bit depth and the plurality of predefined forward mapping scaling coefficients within the predefined forward mapping accuracy comprises: Converting the plurality of inter prediction samples of the luminance component from the original domain to the mapped domain using the plurality of predefined forward mapping scaling coefficients without performing a clipping operation; The method according to claim 1.

3. The method according to claim 1, wherein the predefined forward mapping accuracy is 11 bits.

4. A computing device, comprising: One or more processors; A non-transitory storage device coupled to the one or more processors; When executed by the one or more processors, a plurality of programs stored in the non-transitory storage device that cause the computing device to execute the method according to any one of claims 1 to 3, and A computing device comprising.

5. A method for storing a bitstream, comprising Performing an encoding method to generate a bitstream, The encoding method includes Dividing the current picture into one or more coding units (CUs); Obtaining a plurality of prediction samples in the mapped domain of the luminance component of the current coding unit (CU) encoded in inter mode under the framework of luminance mapping with chroma scaling (LMCS); Obtaining a plurality of residual samples in the mapped domain of the luminance component of the current CU; Adding the plurality of prediction samples in the mapped domain to the plurality of residual samples in the mapped domain to obtain a plurality of reconstructed samples in the mapped domain of the luminance component of the current CU; Converting the plurality of reconstructed samples of the luminance component from the mapped domain to the original domain based on a plurality of predefined inverse mapping scaling coefficients; Obtaining prediction information of the current CU based on the plurality of reconstructed samples in the original domain and generating a bitstream; Including Obtaining the plurality of prediction samples in the mapped domain of the luminance component of the current CU includes Deriving a plurality of inter prediction samples in the original domain of the luminance component of the current CU from a temporal reference picture of the current CU; Converting the plurality of inter prediction samples of the luminance component from the original domain to the mapped domain based on a predefined coding bit depth and a plurality of predefined forward mapping scaling coefficients within a predefined forward mapping accuracy; Storing the bitstream; Including The bitstream is decoded by the method according to claim 1, A method for storing a bitstream.

Images