METHOD FOR VIDEO ENCODING, NON-TRANSITORY COMPUTER-READABLE STORAGE MEDIUM, AND METHOD FOR STORING BITSTREAMS - Patent application

Abstract

To provide methods and devices for rectifying a forward mapping coding bit length issue introduced by luma mapping with chroma scaling (LMCS).SOLUTION: In a method, a plurality of prediction samples, in a mapped domain, of a luma component of a coded unit (CU) that is coded by an inter mode or combined inter-intra prediction (CIIP) mode under LMCS framework is obtained, a plurality of residual samples, in the mapped domain, of the luma component of the CU is received from the bitstream, the plurality of prediction samples in the mapped domain are added to the plurality of residual samples in the mapped domain, resulting in a plurality of reconstructed samples, in the mapped domain, of the luma component, and the plurality of reconstructed samples of the luma component is converted from the mapped domain into an original domain based on a pre-defined plurality of inverse mapping scaling factors.SELECTED DRAWING: Figure 20

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation of U.S. Provisional Patent Application No. 63 / 043,569, filed June 24, 2020. The entire disclosure of the aforementioned application is hereby incorporated by reference in its entirety for all purposes. The bodies are incorporated herein by reference.

[0002] The present invention relates generally to video encoding and compression. More particularly, the present invention relates to encoding and compression. Prediction-dependent residual scaling for the scaling unit Video encoding is performed using PDRS (Performance Residual Scaling) The present invention relates to a system and method for [Background technology]

[0003] This section provides background information related to the present disclosure. The information should not necessarily be construed as prior art.

[0004] Use any of a variety of video encoding techniques to compress the video data Video encoding may be performed according to one or more video encoding standards. Some exemplary video coding standards are Versatile Video Coding (Versatile Video Coding) tile Video Coding (VVC), joint search test model (joint exploration model (JEM) coding, high-efficiency video coding (H.26 5 / HEVC), Advanced Video Coding (H.264 / AVC), and Video Extraction Includes Part Group (MPEG) encoding.

[0005] Video coding generally involves predictive coding that exploits the redundancy inherent in a video image or sequence. Video coding techniques utilize various prediction methods (e.g., inter-prediction, intra-prediction, etc.). One of them is to convert video data to lower bitrates while avoiding or minimizing video quality degradation. The goal is to compress the data into a format that is usable for the target. Summary of the Invention [Problem to be solved by the invention]

[0006] This section provides a general overview of the disclosure and does not provide a complete description of its full scope or all of its features. It is not a comprehensive disclosure. [Means for solving the problem]

[0007] According to a first aspect of the present application, under the framework of luma mapping with chroma scaling (LMCS), , conclusion A plurality of prediction samples in a mapped domain of a luma component of a coding unit (CU) coded by a combined inter and intra prediction (CIIP) mode is obtained, and a plurality of residual samples in the mapped domain of the luma component of the CU are obtained. acquired the plurality of prediction samples in the mapped domain are added to the plurality of residual samples in the mapped domain to obtain a plurality of reconstructed samples in the mapped domain of the luma component, and the plurality of reconstructed samples of the luma component are transformed from the mapped domain to the original domain based on a plurality of predefined inverse mapping scaling coefficients. According to a second aspect of the present application, there is provided a method for video decoding, the method including: obtaining a plurality of prediction samples in a mapped domain of a luma component of a coding unit (CU) coded by a joint inter-intra prediction (CIIP) mode under a luma mapping with chroma scaling (LMCS) framework; obtaining a plurality of residual samples in the mapped domain of the luma component of the CU; adding the plurality of prediction samples in the mapped domain to the plurality of residual samples in the mapped domain to obtain a plurality of reconstructed samples in the mapped domain of the luma component; and transforming the plurality of reconstructed samples of the luma component from the mapped domain to an original domain based on a plurality of predefined inverse mapping scaling coefficients. In one embodiment, obtaining a plurality of predicted samples in the mapped domain of the luma component of the CU includes deriving a plurality of inter-predicted samples in the original domain of the luma component of the CU from a temporal reference picture of the CU, and transforming the plurality of inter-predicted samples of the luma component from the original domain to the mapped domain based on a predefined coding bit depth and a predefined plurality of forward mapping scaling factors within a predefined forward mapping precision. In one embodiment, converting the plurality of inter-predicted samples of the luma component from the original domain to the mapped domain based on a predefined coding bit depth and a predefined plurality of forward mapping scaling factors within a predefined forward mapping precision includes converting the plurality of inter-predicted samples of the luma component from the original domain to the mapped domain using the predefined plurality of forward mapping scaling factors without a clipping operation. In one embodiment, the predefined forward mapping precision is 11 bits.

[0008] No. of this application 3 According to an aspect of the present application, a computing device includes one or more processors, a memory, and a plurality of programs stored in the memory. The programs, when executed by the one or more processors, cause the computing device to perform the functions of the first aspect of the present application. or the second aspect to perform the operations described above.

[0009] No. of this application 4 According to an aspect of the present application, a non-transitory computer-readable storage medium stores a plurality of programs for execution by a computing device having one or more processors, the programs, when executed by the one or more processors, causing the computing device to perform the functions of the first aspect of the present application. or the second aspect to perform the operations described above.

[0010] A set of exemplary, non-limiting embodiments of the present disclosure are described below in conjunction with the accompanying drawings. Structural, methodological, or functional variations may be made by those skilled in the art based on the examples presented herein. may be implemented, and all such variations are within the scope of this disclosure. Where appropriate, the teachings of different embodiments may, but are not necessarily, combined with each other. There is no need to mix them. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram illustrating an example block-based hybrid video encoder that can be used with many video coding standards. [Figure 2] 1 is a block diagram illustrating an exemplary video decoder that can be used with many video encoding standards. [Figure 3] FIG. 1 is a diagram of block partitioning in a multi-type tree structure that can be used in conjunction with many video coding standards. [Figure 4]10 is a flowchart showing a decoding process to which LMCS is applied. [Figure 5] FIG. 1 is a diagram of bi-directional optical flow (BDOF) processing. [Figure 6] 1 is a flowchart illustrating the workflow of chroma residual scaling in LMCS when Decoder-side Motion Vector Refinement (DMVR), BDOF, and CIIP are all enabled. [Figure 7] 1 is a flowchart illustrating the steps of the Prediction Dependent Residual Scaling (PDRS) procedure. [Figure 8] 10 is a flowchart showing a workflow of a decryption process when a PDRS procedure is applied in LMCS processing. [Figure 9] FIG. 10 illustrates the residual mapping error resulting from using only predicted samples to derive scaling factors. [Figure 10] 10 is a flowchart illustrating steps in a chroma sample reconstruction procedure. [Figure 11] 10 is a flow chart illustrating steps of a second chroma sample reconstruction procedure. [Figure 12] 10 is a flowchart illustrating a workflow of an LMCS decoding process in an example of a second chroma sample reconstruction procedure in which DMVR, BDOF, and CIIP are not applied to generate luma prediction samples for chroma scaling. [Figure 13] 10 is a flowchart illustrating a workflow of an LMCS decoding process in a second example of a second chroma sample reconstruction procedure in which an initial single prediction signal is applied to generate luma prediction samples for chroma scaling. [Figure 14] 10 is a flowchart illustrating steps in a chroma residual sample reconstruction procedure. [Figure 15] 10 is a flowchart illustrating a workflow of an LMCS decoding process in one or more embodiments of a chroma residual sample reconstruction procedure. [Figure 16] 10 is a flowchart illustrating a workflow of an LMCS decoding process in another embodiment of the chroma residual sample reconstruction procedure. [Figure 17] 10 is a flowchart illustrating steps of a second chroma residual sample reconstruction procedure. [Figure 18] 10 is a flowchart illustrating steps in an unclipping chroma residual scaling factor derivation procedure. [Figure 19] FIG. 10 is a diagram of regions involved in an example of an unclipping chroma residual scaling factor derivation procedure. [Figure 20] 10 is a flowchart illustrating steps in an unclipping chroma sample decoding procedure. [Figure 21] 10 is a flowchart illustrating the steps of one or more embodiments of an unclipping chroma sample decoding procedure. [Figure 22] 10 is a flowchart illustrating steps of one or more embodiments of an unclipping chroma sample decoding procedure. [Figure 23] 1 is a flowchart showing the steps of a first aspect of the present disclosure. [Figure 24] 1 is a flowchart illustrating steps of one or more embodiments of the first aspect of the present disclosure. [Figure 25] 1 is a flowchart illustrating steps of one or more embodiments of the first aspect of the present disclosure. [Figure 26] 1 is a flowchart illustrating steps of one or more embodiments of the first aspect of the present disclosure. [Figure 27] 1 is a block diagram illustrating an example apparatus for video encoding, according to some implementations of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0012] The terms used in this disclosure are not intended to limit the disclosure but to illustrate specific examples. As used in this disclosure and the appended claims, the singular forms "a," "an," and "the" are and "the" also refer to the plural unless the context clearly dictates otherwise. The term "and / or" as used means one or more of the It is to be understood that any or all possible combinations of the listed items are referred to.

[0013] In this specification, terms such as "first," "second," and "third" are used to describe various pieces of information. It is understood that the information should not be limited by these terms, although it may be These terms are used only to distinguish one category of information from another. For example, the first information may be referred to as the second information without departing from the scope of this disclosure. Similarly, the second information may be referred to as the first information. Where applicable, the word "if" may be replaced with "when" or "whenever" depending on the context. means "upon" or "in response to" It can be understood.

[0014] Throughout this specification, the terms "in one embodiment," "in an embodiment," "another ... " or the like refers to one or more particular features, structures, or feature is included in at least one embodiment of the present disclosure. Therefore, the use of the words "in one embodiment" or "in an embodiment" in various places throughout this specification may refer to Appearances of phrases such as "in an embodiment," "in another embodiment," etc. do not necessarily all refer to the same embodiment. Furthermore, references to specific features, structures, or The or characteristics may be combined in any suitable manner.

[0015] The first version of the HEVC standard was completed in October 2013, and is the first version of the previous generation of HEVC. Approximately 50% bitrate savings compared to H.264 / MPEG AVC video encoding standards or equivalent perceptual quality. The HEVC standard represents a significant coding improvement over its predecessor. It offers the benefits of HEVC, but by using additional coding tools on top of it, it offers superior coding efficiency. Based on this, both VCEG and MPEG are We have begun work to explore new coding techniques for standardizing video coding. ITU-T VECG and and ISO / IEC MPEG in October 2015 as a joint video exploration team. (Joint Video Exploration Team, JVET) was formed. One reference software, called the Joint Exploration Model (JEM), is the HEVC test model. By integrating some additional coding tools on top of Dell (HM), JVET This was maintained.

[0016] October 2017: Video Compression Proposal (CfP) with Capabilities Beyond HEVC The 23rd International Standards Conference (ISC) was held in April 2018 and was jointly solicited by ITU-T and ISO / IEC. CfP responses were received and evaluated in 10 JVET meetings, which resulted in HEVC being ranked at approximately 40 Based on these evaluation results, JVET has decided to A new development project for Versatile Video Coding (VVC), an alternative video coding standard, has been launched. In the same month, a VVC test model (VTM) was launched. A single reference software release has been established to demonstrate a reference implementation of the VVC standard.

[0017] Prediction methods used in video coding typically aim to reduce or eliminate the redundancy inherent in video data. performs spatial (intra-frame) and / or temporal (inter-frame) prediction to remove and is generally associated with block-based video coding. Similarly, VVC is based on a block-based hybrid video coding framework. It is being constructed.

[0018] In block-based video coding, the input video signal is processed block by block. For a block, spatial prediction and / or temporal prediction can be performed. In newer video coding standards, such as VC design, blocks are coded as binary trees rather than just quad trees. and / or further splitting based on multi-type tree structures, including ternary trees. This allows for better adaptation to changing local characteristics.

[0019] Spatial prediction (also known as "intra prediction") is used to predict the current block. , and already coded neighboring blocks (called reference samples) in the same video picture / slice. Spatial prediction uses pixels from samples in a video signal. Reduce sexuality.

[0020] During the decoding process, the video bitstream is first decoded in the entropy decoding unit. The coding mode and prediction information are then entropy decoded to form a prediction block. , spatial prediction unit (if intra-coded) or temporal prediction unit (if inter-coded) The residual transform coefficients are transmitted to either the The prediction block and the residual are then sent to the inverse quantization unit and the inverse transform unit. The difference block is added to the reconstructed block. The reconstructed block is stored in the reference picture store. Before that, the reference picture store may go through an additional in-loop filtering pass. The reconstructed video in is sent to drive a display device and , used to predict future video blocks.

[0021] Newer video coding standards, such as the current VVC design, use brightness scaling with chroma scaling. The coding tool for degree mapping (LMCS) can be applied before in-loop filtering. LMCS adjusts the dynamic range of the input signal to improve coding efficiency. The purpose is to:

[0022] However, in the current design of LMCS, the mapping of inter-predicted samples of the luma component is The coding precision can exceed the dynamic range of the intra-coding depth.

[0023] Conceptually, many video coding standards, including those mentioned above in the Background section, For example, virtually all video coding standards use block-based processing. They use the same video coding block diagram to achieve video compression.

[0024] FIG. 1 illustrates an exemplary block-based high-speed video coding scheme that may be used with many video coding standards. 1 shows a block diagram of a hybrid video encoder 100. The encoder 100 A frame is divided into multiple video blocks for processing. For a given video block: The predictions are formed based on either inter-prediction or intra-prediction techniques. In center prediction, motion estimation and motion estimation are performed based on pixels from previously reconstructed frames. Intra prediction involves the formation of one or more predictors. A predictor is formed based on the reconstructed pixels in the frame. The mode decision determines the The best predictor can be selected to predict the lock.

[0025] A prediction residual, which represents the difference between the current video block and its predictor, is sent to the transform circuit 102. The transform coefficients are then sent from the transform circuit 102 to a quantization circuit for entropy reduction. 104. The quantized coefficients are then transmitted to generate a compressed video bitstream. As shown in FIG. block partition information, motion vectors, reference picture indexes, and intra prediction modes. Prediction-related information 110 from any inter-prediction and / or intra-prediction circuit 112 is also fed through an entropy coding circuit 106 to produce a compressed video bitstream 11 It will be saved in 4.

[0026] The encoder 100 also includes decoder-related circuitry to reconstruct pixels for prediction purposes. First, the prediction residual is reproduced by the inverse quantization circuit 116 and the inverse transform circuit 118. This reconstructed prediction residual is then used to construct the filtered prediction residual for the current video block. The block predictor 120 is combined with the block predictor 120 to generate the unmodified reconstructed pixel.

[0027] Temporal prediction (also called "inter prediction" or "motion compensated prediction") is a method for predicting the time course of a current video. Using reconstructed pixels from an already coded video picture to predict a block. Temporal prediction reduces the temporal redundancy inherent in video signals. The temporal prediction signal for a given CU is The symbol is usually one or more symbols that indicate the amount and direction of movement between the current CU and its time reference. It is signaled by multiple motion vectors (MVs) and multiple reference pictures. If supported, one reference picture index is additionally transmitted, which is Used to identify which reference picture in the reference picture store the temporal prediction signal comes from. It is used.

[0028] After spatial and / or temporal prediction is performed, the intra / inter prediction in the encoder 100 The mode decision circuit 121 determines the best prediction mode, for example based on a rate-distortion optimization method. The block predictor 120 is then subtracted from the current video block. The resulting prediction residuals are then de-correlated using a transform circuit 102 and a quantizer circuit 104. The resulting quantized residual coefficients are inversely quantized by the inverse quantization circuit 116 and then subjected to an inverse transform. The inverse transform is performed by path 118 to form a reconstructed residual, which is then added to the predicted block. The reconstructed CU is stored in the picture buffer 117. before being put into the reference picture store and used to encode future video blocks. Then, the reconstructed CU is applied with a deblocking filter and a sample adaptive offset (sample adaptive offset (SAO), and / or adaptive in-loop filter Further loops, such as adaptive in-loop filters (ALF) Intra-filtering 115 can be applied to the output video bitstream 114. To form the coding mode (inter or intra), prediction mode information, motion information , and the quantized residual coefficients are all further compressed and The data is sent to the entropy coding unit 106 to be packed.

[0029] For example, the deblocking filter works in current versions of AVC, HEVC, and VVC. In order to further improve coding efficiency, HEVC uses SAO An additional in-loop filter called (sample adaptive offset) The current version of the VVC standard defines an adaptive loop filter (ALF). Yet another in-loop filter, called the Highly likely.

[0030] These in-loop filter operations are optional. Performing these operations is They help improve efficiency and visual quality. They also reduce computational complexity. Alternatively, the encoder 100 may provide a decision to turn it off.

[0031] Intra prediction is usually based on unfiltered reconstructed pixels, ,Inter prediction is based on filtered reconstructed pixels, and these filters Note that the filter option is turned on by the encoder 100.

[0032] FIG. 2 illustrates an exemplary video decoder 200 that can be used with many video coding standards. This decoder 200 is a block diagram showing the reconstruction of the encoder 100 of FIG. In the decoder 200 (FIG. 2), the input video bitstream is Stream 201 uses the encoder to derive the quantized coefficient levels and prediction related information. The quantized coefficient levels are first decoded via tropy decoding 202. Then, the quantized coefficient levels are decoded via inverse It is processed through quantization 204 and inverse transform 206 to obtain a reconstructed prediction residual. The block predictor mechanism implemented in the intra / inter mode selector 212 Based on the prediction information, either intra prediction 208 or motion compensation 210 is performed. The set of unfiltered reconstructed pixels is then fed to adder 21. 4 to the reconstructed prediction residual from the inverse transform 206 and the block predictor mechanism. The predicted output is obtained by summing it with the predicted output generated by the

[0033] The reconstructed blocks are stored in a picture buffer 213 which acts as a reference picture store. The picture may be further passed through an in-loop filter 209 before being stored in the The reconstructed video in buffer 213 is then used to drive a display device. and used to predict future video blocks. In the situation where the in-loop filter 209 is turned on, these reconstructed A filtering operation is performed on the reconstructed pixels to produce the final reconstructed video output 222. Put out.

[0034] In video coding standards such as HEVC, blocks are divided based on a quadtree. Good. Newer video coding standards, such as VVC, now use more partitioning methods. A coding tree unit (CTU) is divided into CUs, and a quadtree, binary tree, or is based on a ternary tree and can adapt to various local characteristics. CU, Prediction Unit The separation of the PU and TU is the most common coding mode in current VVC. Each CU is a basic unit for both prediction and transformation without further division. However, some modes, such as the intra subdivision coding mode, In certain coding modes, each CU can still contain multiple TUs. In the quadtree structure, one CTU is first divided into quadtrees. A ternary tree leaf node can be further divided by binary and ternary tree structures. .

[0035] Figure 3 shows the five division types currently used in VVC: quadrant 301; Horizontal halves 302, vertical halves 303, horizontal thirds 304, and vertical value divisions 305 are shown. In situations where a multi-type tree structure is used, one CTU is first converted to a quadtree structure. Then, each quadtree leaf node is divided into two parts by binary tree structure and ternary tree structure. can be further divided into

[0036] One of the exemplary block divisions 301, 302, 303, 304, or 305 of FIG. 1. One or more of the above methods may be used to perform spatial and / or temporal prediction using the configuration shown in FIG. Spatial prediction (or "intra prediction") can be performed on multiple frames within the same video picture / picture. The image is taken from samples of adjacent blocks (called reference samples) already coded within the slice. The current video block is predicted using spatial prediction. Reduces technical redundancy.

[0037] New video coding standards, such as VVC, now offer a new coding tool: chroma scaling. Luminance mapping with filtering (LMCS) has been added. , deblocking filter, SAO, and ALF) LMCS will be added as a tool for simplification.

[0038] In general, LMCS has two main modules: First, adaptive piecewise linear models and secondly, a luminance-dependent chroma residual scale It's a ring.

[0039] Figure 4 shows the modified decoding process using LMCS. In Figure 4, a particular block is Decoding module executed in the pinged domain, entropy decoding 401, inverse Quantization 402, inverse transform 403, luma intra prediction 404, and luma sample reconstruction 40 5 (i.e., the luminance prediction sample Y' pred and the luminance residual sample Y' res Add and and the reconstructed luminance samples Y' recon (generating certain other blocks) indicates that the decryption module runs in the original (i.e., unmapped) domain. 409, motion compensated prediction, 412, chroma intra prediction, 413 (i.e., chroma sample reconstruction), That is, saturation prediction sample C pred and saturation residual sample C res and are added together to form a reconstructed Saturation sample C recon ), and in-loop filtering 407 (generating (including locking, SAO, and ALF). A further group of blocks is , forward mapping 410 and inverse (or backward) mapping 406 of luma samples; and new operational models introduced by LMCS, including chroma residual scaling411. Also, as shown in FIG. 4, a decoded picture buffer (DPB) 40 All reference pictures stored in 8 (for luma) and 415 (for chroma) are It's in the main.

[0040] The in-loop mapping of LMCS is based on the dynamics of the input signal to improve coding efficiency. The purpose is to adjust the range of the luminance samples in the existing LMCS design. The intra-group mapping consists of two mapping functions: one forward mapping function FwdMap, and one corresponding inverse mapping function InvMap. The ping function is computed using one piecewise linear model with 16 equally sized pieces. The inverse mapping function is signaled from the encoder to the decoder. It can be derived directly from the number and therefore does not need to be signaled.

[0041] The parameters of the luminance mapping model are signaled at the slice level. to indicate whether a mapping model should be signaled for the current slice. The presence flag is signaled first. The presence flag indicates whether a luminance mapping model exists in the current slice. If present, the corresponding piecewise linear model parameters are further signaled. Based on the model, the dynamic range of the input signal has equal size in the original domain. Each segment is mapped to a corresponding segment. For a given segment in the original domain, The corresponding segments in the mapped image may have the same or different sizes. The size of each segment in the domain is the number of codewords in that segment (i.e. , the mapped sample values). For , the number of codewords in the corresponding segment in the mapped domain For example, if the input is 10 bits, the linear mapping parameters can be derived based on When the pixel depth is 16, each of the 16 segments in the original domain has 64 pixel values. and each of the segments in the mapped domain also has six segments assigned to it. If we have four codewords, it is a simple one-to-one mapping (i.e., each sample (mapping that does not change the domain value) The number of signaled codewords in the segment is calculated by the scaling factor. Additionally, at the slice level, another An LMCS control flag is signaled to enable / disable LMCS for the slice.

[0042] For each segment, the corresponding piecewise linear model is given in the box immediately following this paragraph. It is defined as follows. For the ith segment (i=0...15), the corresponding piecewise linear model is Input pivot points InputPivot[i] and InputPivot[i+1], etc. and two output (mapping) pivot points MappedPivot[i] and Ma ppedPivot[i+1], and the 10-bit input video Assuming that the values of InputPivot[i] and MappedPivot[i] ( i=0...15) is calculated as follows: 1. Set the variable OrgCW=64. 2. For i=0:16, InputPivot[i]=i*OrgCW 3. For i=0:16, MappedPivot[i] is calculated as follows: MappedPivot[0]=0; (i=0;i<16;i++) MappedPivot[i+1]=MappedPivot[i]+Signale dCW[i] where SignaledCW[i] is the codeword for the i-th segment. The signaled number.

[0043] As shown in Figure 4, during LMCS processing, we need to operate in two different domains. For each CU coded via inter-CU prediction mode ("inter-CU"), its motion The compensation prediction is performed in the original domain. However, the luminance component (i.e., luminance prediction The reconstruction of the luminance residual (addition of the luminance residual and the luminance samples) is performed in the mapped domain. Therefore, the motion compensated luminance prediction Y pred is Y' pred is used for pixel reconstruction 405 Previously, the forward mapping function 410, i.e., Y' pred =FwdMap(Y pred ) The value Y' in the domain mapped from the original domain via pred Nimappi On the other hand, the video must be coded via intra prediction mode ("intra CU") For each CU, Y' pred Before being used for pixel reconstruction 405, the mapping Considering that intra prediction 404 is performed in the domain (shown in FIG. 4), the prediction No sample mapping is necessary. Finally, the reconstructed luminance samples Y' reco n , an inverse mapping function 406 is applied to generate the reconstructed luma samples Y ' recon to the original domain value Y recon and then convert to luminance DPB 408 Mmm, that is, Y recon =InvMap(Y' recon ) to InterCU Unlike the forward mapping 410 of the predicted samples, which only needs to be applied to the reconstructed The backward mapping 406 of the selected samples is applied to both inter-CU and intra-CU. It needs to be done.

[0044] In summary, on the decoder side, the current in-loop luma mapping for LMCS is based on luma prediction. Sample Y pred is first converted to the mapped domain if necessary. Y'pred = FwdMap(Ypred). Then, the mapped prediction sample is The sample is added to the decoded luminance residual and the reconstructed Form the luminance samples: Y'recon=Y'pred+Y'res. Finally, the inverse mapping Applying ping to transform the reconstructed luminance samples Y'recon to the original domain Yrecon=InvMap(Y'recon). On the encoder side, the luminance residual is Since it is coded in the mapped domain, it is not matched with the original samples of the mapped luminance. It is generated as the difference between the predicted sample of the luminance signal and the sample of the predicted luminance signal. p(Yorg)-FwdMap(Ypred).

[0045] The second step of LMCS, luminance-dependent chroma residual scaling, is performed by in-loop mapping. The quantization accuracy between a luma signal and its corresponding chroma signal when filtering is applied to the luma signal. The saturation residual scaling is enabled or disabled. The slice header also indicates whether luminance mapping is enabled and the current slice If dual-tree splitting of luma and chroma components is disabled for Rice, luminance-dependent An additional flag is signaled to indicate whether chroma residual scaling is applied. If intensity mapping is not used, or if dual-tree partitioning is not used for the current slice, When enabled, luminance-dependent chroma residual scaling is always disabled. Chroma residual scaling is always disabled for CUs containing four or fewer chroma samples. .

[0046] For both intra and inter CUs, to scale the chroma residuals The scaling parameter used is the average of the corresponding mapped luma prediction samples. The scaling parameters are given in the box immediately following this paragraph. It is derived as follows. avg' Y Denote , as the mean of the luminance prediction samples in the mapped domain. The scaling parameter C ScaleInv is calculated according to the following steps: 1. Mapped domains avg' Y The segment in the piecewise linear model to which Dex Y 1dx where Y 1dx has an integer value ranging from 0 to 15 . 2.C ScaleInv =cScaleInv[Y 1dx ], where cScaleIn v[i], i=0...15, are 16 pre-computed lookup tables (LUTs) ) Since intra prediction is performed in the mapped domain of LMCS, the intra,results are Combined Inter-Intra Prediction (CIIP) or Intra Block Copy (IBC) model For CUs coded as a code, avg' Y is calculated as the average of the luminance prediction samples Otherwise, avg' Y is the forward mapped inter predicted luma sample It is calculated as the average of the

[0047] Figure 4 also shows the calculation of the average of the luma prediction samples for luma-dependent chroma residual scaling. For inter-CU, the forward-mapped luma prediction Y' pred is a scale Ringed chroma residual C resScale and is fed to the chroma residual scaling 411 , saturation residual C res is derived, which corresponds to the reconstructed saturation value C recon Derive For the saturation prediction C pred In the case of intra CU, , the intra prediction 404 is already in the mapped domain Y' pred Generate It is fed to chroma residual scaling 411 in the same way as for inter-CU.

[0048] Unlike brightness mapping, which is performed on a sample-by-sample basis, ScaleInv is the saturation It is fixed for the entire CU. ScaleInv Given this, the next paragraph Chroma residual scaling is applied as described in box. Encoder side:

number

number

[0049] New video coding standards, such as VVC, are now introducing new coding tools. Some examples of new coding tools are Bi-Directional Optical Flow (Bi-Dir Motion vector fine adjustment on decoder side Decoder-side Motion Vector Refinement, DMVR), Combined Inter and Intra Prediction tra Prediction,CIIP), for affine modes and Prediction Refinement by Optical Flow with Optical Flow,PROF).

[0050] In the current VVC, bidirectional optical flow (BDOF) is applied to achieve bidirectional predictive coding. The prediction samples of the coded block are corrected.

[0051] FIG. 5 is an explanatory diagram of BDOF processing. BDOF is a block-by-block process when bi-prediction is used. It is a sample-wise motion refinement performed on top of the block-based motion compensation prediction. 4 Fine-tuning the movement of subblock 501

number

[0052] Specifically, as described in the box immediately following this paragraph, the motion fine-tuning [Number] value is derived. [Number] Here, [Number] is the floor function, and clip3(min, max, x) is a function that clips any value x within the range [min, max]. The symbol >> represents a bitwise right shift operation . The symbol << represents a bitwise left shift operation. [Number] is a motion correction threshold for preventing propagated errors caused by irregular local motion, which is equal to 1 << max(5, bitDepth - 7), where bitDe pth is the internal bit depth. Furthermore, [Number] is.

[0053] The value in the box immediately above [Number]

number

number

number

number

number

number

[0054] The value in the box immediately above

number

number

number

number

number

number

number

number

number

[0055] Fine-tuning the above movement

number

number

number

number

number

number

[0056] Based on the above-mentioned bit depth control method, the maximum bit depth of the intermediate parameters in the entire BDOF process is guaranteed not to exceed 32 bits, and the maximum input to the multiplication is guaranteed to be within 15 bits. That is, a 15-bit multiplier is sufficient for BDOF implementation. 。

[0057] DMVR can be further corrected by using bilateral matching prediction. Bi-prediction is used for merging blocks with two early signaled MVs. It's technology.

[0058] Specifically, in DMVR, bilateral matching is performed using two different reference pictures. by finding the best match between two blocks along the motion trajectory of the current CU in Used to derive the motion information of the current CU. Cost used in the matching process The function is row-subsampled SAD (Sum of Absolute Differences). After being calculated, the corrected MVs are used for motion compensation in the prediction stage to correct the temporal motion of the subsequent pictures. The motion vector prediction and uncorrected MV are the motion vectors of the current CU and its spatial neighbors. It is used for motion vector prediction between the input vector and the output vector.

[0059] Under the assumption of continuous motion trajectories, the motion vectors MV0 and MV1 point to two reference blocks. and MV1 are the temporal distances between the current picture and the two reference pictures, i.e., TD0 and MV1. As a special case, if the current picture has two reference pictures, The time distance between the current picture and the two reference pictures is If they are the same, the bilateral matching becomes mirror-based two-way MV.

[0060] Currently, VVC uses both inter-prediction and intra-prediction methods in hybrid video coding. A prediction method is used, and each PU utilizes correlation in either the time domain or the spatial domain. You can only choose between inter prediction or intra prediction for use, but not both. However, as pointed out in previous papers, the inter-predicted blocks and the inter-predicted blocks The residual signals generated by the predictive blocks may exhibit very different characteristics. Therefore, if the two types of predictions can be efficiently combined, the prediction residual can be improved. To reduce the difference energy and improve coding efficiency, another more accurate prediction is Moreover, in natural video content, the movements of moving objects are complex. For example, older content (e.g., pictures previously coded) may contain objects) and new content (e.g., in a previously coded picture) There may be areas that contain both the object and the object. In this case, both inter and intra predictions provide one accurate prediction of the current block. I can't do that.

[0061] To further improve prediction efficiency, we use the merged mode to encode the input of one CU. Combined inter-intra prediction (CIIP) is a method that combines intra-prediction and inter-prediction. This is adopted in the VC standard. Specifically, for each merge CU, One additional flag is signaled to indicate whether IIP is enabled or not. If the flag is equal to 1, the CIIP will generate intra prediction samples for the luma and chroma components. Apply only planar modes to generate the ray. In addition, apply equal weights (i.e., 0.5 ) is the final predicted sample of the CIIP CU, and the inter-predicted sample and the intra-predicted sample are used. It is applied to average the predicted samples.

[0062] VVC also supports affine mode for motion compensated prediction. Only translational motion models are applied to motion compensated prediction. There are many types of motion, including zoom-out, rotation, perspective motion, and other irregular motions. In VVC, this is achieved by signaling one flag for each inter-coded block. Therefore, affine motion compensation prediction is applied, and translational motion or affine motion models are used. In the current VVC design, the 4-parameter affine model is used. Two affine modes, including a 6-parameter affine mode, are Supported for coding blocks.

[0063] The four-parameter affine model has the following parameters: horizontal and vertical Two parameters for each translation movement, one parameter for the zoom movement , and one parameter for rotational movement in both directions. Horizontal zoom parameter is equal to the vertical zoom parameter. The horizontal rotation parameter is equal to the vertical rotation parameter. In order to achieve better adaptation of motion vectors and affine parameters, The affine parameters are the coordinates of the top-left and top-right corners of the current block. , are converted into two MVs (also called control point motion vectors (CPMVs)). The affine motion field of is described by two control points MV(V0, V1).

[0064] Control point movement Vector Based on the motion field of one affine coded block (v x , v y ) is calculated as described in the box immediately following this paragraph.

number

[0065] The 6-parameter affine mode has the following parameters: horizontal and vertical Two parameters for each translation movement, one parameter for the zoom movement and one parameter for horizontal and rotational movement, and one parameter for zoom movement. 6 parameters, one for vertical and one for rotational movement. The affine motion model is coded with three MVs in three CPMVs.

[0066] The three control points of a six-parameter affine block are located at the top-left, top-right, and bottom-left corners of the block. The movement of the top-left control point is associated with translational movement, the movement of the top-right control point is associated with horizontal rotation and zoom movement, and the movement of the bottom-left control point is associated with vertical rotation and zoom movement. Compared to the four-parameter affine motion model, the six-parameter Affine Motion Mode The horizontal rotation and zoom movements of may not be the same as those in the vertical direction.

[0067] Let (V0,V1,V2) be the MVs in the top-left, top-right, and bottom-left corners of the current block. Then, each sub-block (v x , v y )'s motion vector is the box immediately following this paragraph. It is derived using three MVs at the control points as described in

number

[0068] To improve the accuracy of affine motion compensation, we refine the prediction by optical flow (P ROF) is currently being studied in VVC, which is based on the optical flow model. The sub-block-based affine motion compensation is performed based on the sub-block After performing the base affine motion compensation, the luma prediction samples of one affine block are Corrected by a one-sample fine-tuning value derived based on the optical flow equation. In detail, the operation of PROF can be summarized in the following four steps:

[0069] In step 1, sub-block based affine motion compensation is performed with four parameters. For the 6-parameter affine model, see equation (6) above, and for the 6-parameter affine model, see equation (6). For the subblock, the derived subblock MV is used in the above equation (7) to Lock Prediction

number

[0070] In step 2, the spatial gradient of each prediction sample

number

number

number

[0071] Additionally, in step 2, one additional row / Columns need to be generated on each side of one sub-block. This reduces memory bandwidth and complexity. To reduce the number of samples, the samples on the extended boundary are referenced to avoid additional interpolation. Copied from the nearest integer pixel location in the picture.

[0072] In step 3, the brightness prediction fine-tuning values are listed in the box immediately following this paragraph. It is calculated as follows.

number

number

number

number

number

[0073] Furthermore, in the current PROF design, after applying forecast fine-tuning to the original forecast samples, As stated in the box immediately after the drop, the value of the corrected predicted sample is converted to 15 bits. In order to clip within can be.

number

number

number

number

[0074] The affine model parameters and pixel positions relative to the sub-block center are It doesn't change every time,

number

number

number

number

number

number

[0075] Based on the above affine subblock MV derivation formulas (6) and (7), the MV difference

number

number

number

number

number

number

number

number

number

number

[0076] According to the current LMCS design, chroma residual samples are generated by their corresponding luma prediction samples. Newer coding tools are effective for inter-CU. When set to 1, the chroma residual samples are scaled via the LMCS within this inter-CU. The luma prediction samples used to estimate the luminance are based on the cross-application of these newer coding tools. It is retrieved last.

[0077] Figure 6 shows the LM with DMVR, BDOF, and CIIP all enabled. A flowchart showing the workflow of chroma residual scaling in CS. The outputs from the 0 predictor 601 and the luminance L1 predictor 602 are fed to the DMVR 603 and the BD The obtained luma inter prediction value 621 is sequentially supplied to the OF 604. 605 is fed to the average 606 together with the luma intra prediction 622 from 605 to produce the average luma prediction 6 23, which is fed into chroma residual scaling 607 along with chroma residual 608, As a result, chroma residual scaling 607, chroma prediction 610, and chroma reconstruction 609 work together. can be used to generate the final result.

[0078] The current LMCS design presents three challenges to the video decoding process. First, Mapping between domains (different domain mappings) requires additional computational complexity and on-chip Second, the derivation of the luma and chroma scaling factors requires a large amount of memory for different luma values. The fact that we use predicted values introduces additional complications. Third, the LMCS and newer The interaction between the encoding tool and the LMCS adds delay to the decoding process, i.e., the delay associated with LMCS. Introduce the problem.

[0079] First, the current LMCS design allows for the reconstruction of the original domain and the mapping Both of these domains are used in various decoding modules. Samples need to be transformed from one domain to another between different decoding modules. This often comes at the expense of both higher computational complexity and more on-chip memory. This may invite

[0080] Specifically, for intra-mode, CIIP mode, and IBC mode, one current The mapped domain reference samples from the neighboring reconstructed domains of the current CU are , are used to generate predicted samples. However, for inter modes, motion compensation The prediction is performed using the original domain reconstructed samples of the temporal reference picture as reference. The reconstructed samples stored in the DPB are also in the original domain. Such a mixed representation of reconstructed samples under different prediction modes involves additional forward and and inverse luminance mapping operations.

[0081] For example, in the case of inter-CU, the luma reconstruction operation (i.e., the predicted samples and the residual samples) The calculation (adding the original domain and the original domain together) is performed in the mapped domain. Inter-predicted luma samples generated in the previous step are used for luma sample reconstruction. In another example, the intra-CU and For both inter-CUs, the inverse (or backward) mapping is always applied to reconstruct After the mapped luminance samples are transformed from the original domain to the DP Such a design increases computational complexity due to the additional forward / reverse mapping operations. This not only increases the complexity but also maintains multiple versions of the reconstructed sample. This requires more on-chip memory.

[0082] Based on the above explanation, some LMCS designs have luminance mapping and luminance dependency Chroma residual scaling is performed to encode the luma and chroma components, respectively. In a practical hardware implementation, the forward and inverse (or reverse) mapping functions FwdMap and InvMap use lookup tables (LUTs) and can be calculated and implemented on the fly. A LUT-based solution is used. If so, the possible output elements from the functions FwdMap, InvMap, and cScaleInv are: The elements can be pre-computed and pre-stored as a LUT, and then the current slide Used for luma mapping and chroma residual scaling operations for all CUs in the device. Assuming the input video is 10-bit, FwdMap and InvMa Each LUT in p has 2 10 = 1024 elements, each element of the LUT has 10 bits Therefore, the total storage capacity of the forward and inverse luminance mapping LUTs is 2*102 This is equivalent to 4*10=20480 bits=2560 bytes. On the other hand, the saturation scaling parameter Data C ScaleInv To derive, 16 errors are required in the encoder and decoder. A LUT table cScaleInv for each chroma scaling entry must be maintained. The LUT cScaleInv parameter is stored in 32 bits. The memory size used to store this is 16*32=512 bits = 64 bytes. The difference between 2560 and 64 is calculated by the forward and reverse (backward) mapping operations. The scale of the additional on-chip memory required is shown in Figure 1.

[0083] Furthermore, new video coding standards such as VVC now use intra prediction and deblocking. Both filtering filters use the reconstructed samples of the neighboring blocks. So, one extra row of reconstructed samples in the current picture / slice width is The data must be kept in a buffer, also known as a "line buffer" in image encoding. The reconstructed samples in the line buffer are located in the first row of one CTU. The CUs that are used at least as a reference for intra prediction and deblocking operations are According to the existing LMCS design, the intra prediction and deblocking filters are different. Therefore, the original domain reconstruction sample is used. Additional on-chip storage is required to store both the pull and the mapped domain reconstruction samples. This requires additional memory, which can nearly double the size of the line buffer.

[0084] In addition to increasing the line buffer size, Another implementation choice to avoid doubling is to perform domain mapping operations on the fly. However, this comes at the cost of a non-negligible increase in computational complexity.

[0085] Therefore, the current design of LMCS requires a , which requires additional computational complexity and on-chip memory.

[0086] Second, in the proposed adaptive luminance residual scaling, both the luminance and chrominance components are scaled accordingly. The current design of LMCS involves a scaling operation on these prediction residuals. Both the saturation and chroma scaling coefficient derivation methods use the Although the two methods use luminance prediction sample values for the purpose of the luminance prediction, there are differences between their corresponding operations.

[0087] For luma residual scaling, the scaling factor is such that each luma residual sample is its own By allowing for a scaling factor to be derived for each sample. However, in the case of chroma residual scaling, the scaling factor is fixed for the entire CU. That is, all chroma residual samples in a CU are mapped to luma prediction samples. They share the same scaling factor, which is calculated based on the average of the

[0088] Also, two different LUTs calculate the scaling factors for the luma and chroma residuals. Specifically, the input to the luma LUT is a map of the original luma predicted sample values. The input to the saturation LUT is the mapped The mapping model segment index for the average value of the luminance prediction sample. In some cases, the need to map the luminance prediction samples to the mapped domain is In addition, it is possible to use one LUT to scale both luma and chroma residuals. becomes.

[0089] Such differences introduce extra complexity into the encoding process, and require additional luma and chroma scaling. A harmonized approach to the derivation of the coefficients is desirable. To achieve this, several steps are required to harmonize the scaling methods for luma and chroma residuals. The following method can be proposed.

[0090] Third, as mentioned above with regard to "luminance-dependent chroma residual scaling", the current LMCS By design, chroma residual samples are scheduled based on their corresponding luma prediction samples. This means that the CU will be queued until all luma prediction samples are completely generated. This means that it is not possible to reconstruct the chroma residual samples of one LMCS CU. As mentioned above, to improve the efficiency of inter-prediction, DMVR, BDOF, and CIIP can be applied. As shown in Figure 6, the saturation residual of the current design of LMCS For scaling, all three modules: DMVR, BDOF, and CIIP to determine the scaling factor of the chroma residual. It can generate luminance prediction samples that are used for the three modules. Given the complexity, we recommend that you wait for their successful completion before performing the LMCS chroma residual scaling. Waiting at 0 can cause significant delays due to the decoding of chroma samples. For affine CUs, each affine CU performs PROF processing followed by LMCS. PROF processing can also have latency issues, as it can take up to 100 samples. This can also cause delays in decoding.

[0091] Furthermore, the current design of LMCS introduces unnecessary crosstalk during the chroma residual scaling coefficient derivation process. A ripping operation is performed, which increases the computational complexity and the extra requirement for on-chip memory. Increase.

[0092] This disclosure addresses or mitigates these challenges presented by current designs of LMCS. More specifically, the present disclosure aims to achieve a hardware-based coding scheme while maintaining coding gain. This paper discusses a method that can reduce the complexity of LMCS for deck implementation.

[0093] The existing LMCS framework transforms predicted / reconstructed samples through a mapping operation. Instead of using a metric, we use a new method called Prediction-Dependent Residual Scaling (PDRS). A method is proposed to directly scale the prediction residual without sample mapping. The proposed method can achieve similar effectiveness and coding efficiency as LMCS, but is more complex to implement. Sex is much lower.

[0094] In the PDRS procedure, a luma prediction sample is used to decode the luma residual samples, as shown in Figure 7. The luminance prediction samples are taken (701) and used to derive the scaling factors (702). 02), scale the luminance residual samples using a scaling factor (703), and The reconstructed image is obtained by adding the luminance prediction samples and the scaled luminance residual samples. The resulting luminance samples are calculated (704).

[0095] In some cases, adaptive luminance residual scaling is used to reduce the implementation complexity of LMCS. Specifically, a method is proposed to calculate the predicted / reconstructed luminance before calculating the luminance prediction residual. Unlike existing LMCS methods that directly transform samples into the mapped domain, P In the proposed method of DRS procedure, the luminance prediction residual samples are converted to the original It is derived in the same way as normal prediction in the domain, followed by a scan on the luminance prediction residual. A scaling operation is performed. The scaling of the luma prediction residual is performed by scaling the corresponding luma prediction sample. As a result, the forward and The inverse luminance mapping operations can be completely removed, and all involved during the decoding process The predicted and reconstructed samples are maintained in the original sample domain. Based on this, the proposed method is called prediction-dependent residual scaling. To improve the delay of scaling derivation, the scaling parameter of the chroma residual samples is From the generation of the luminance prediction samples used to calculate DMVR, BDOF, and Several methods can be proposed to completely or partially eliminate the CIIP operation.

[0096] Figure 8 shows the workflow of the decryption process when the PDRS procedure is applied to the LMCS process. This is a flowchart showing how to eliminate the need for mapping between different domains. Here, the residual decoding module (e.g., entropy decoding 801, inverse quantization Except for the inverse transform 802 and the inverse transform 803, all other decoding modules (intra prediction 80 4,809,812 and inter prediction 816, reconstruction 806 and 813, and All in-loop filters (including 807 and 814) operate in the original domain. Specifically, to reconstruct the luminance samples, the method proposed in the PDRS procedure involves luminance prediction. Residual sample Y res are inversely scaled to their original amplitude levels and then used as the luminance prediction vectors. Sample Y pred Just add it to .

[0097] The PDRS procedure reduces the forward and reverse luminance sample mapping in existing LMCS designs. The filtering operation is completely eliminated, which not only saves / reduces the computational complexity but also It also reduces the potential storage size for saving S-parameters, e.g., LUT-based When a sparse solution is used to perform the luminance mapping, two mapping LUTs are used. Previously, to store FwdMap[] and InvMap[] (approximately 2560 bytes) The storage capacity used for the original domain is no longer required in the proposed method. Existing methods require storing reconstructed luminance samples in both the input and mapped domains. Unlike the brightness mapping method in, the proposed method of PDRS procedure is only in the original domain. generates and maintains all predicted and reconstructed samples in the Compared with the luminance mapping of the previous method, the proposed method in the PDRS procedure is and line buffers used to store the reconstructed samples for deblocking. This effectively reduces the size of the buffer by half.

[0098] According to one or more embodiments of the PDRS procedure, the luminance prediction samples and the luminance residuals The samples are from one and the same location within the luma prediction block and its associated residual block. That is why.

[0099] According to one or more embodiments of the PDRS procedure, deriving a scaling factor using luminance prediction samples includes dividing the full range of possible luminance prediction sample values into a plurality of luminance prediction sample segments and deriving a scaling factor using a predefined luminance prediction sample segment. classification The method includes calculating a scaling factor for each of the plurality of luma prediction sample segments based on a linear model, and determining a scaling factor for the luma prediction sample based on the scaling factors of the plurality of luma prediction sample segments.

[0100] In one example, a luminance prediction is performed based on a scaling factor for a plurality of luminance prediction sample segments. The step of determining a scaling factor for the sample comprises: The steps of assigning the sample to one of the sample segments and the luminance prediction sample Scaling factor for the luminance prediction sample segment assigned the scaling factor for the sample segment. and calculating:

[0101] In this example, the plurality of luma prediction sample segments includes 16 segments in 16 predefined LUT tables, scaleForward, and a predefined LUT table for calculating one scaling factor for each of the plurality of luma prediction sample segments. classification The linear model contains 16 values corresponding to the 16 segments in the predefined LUT table scaleForward.

[0102] To maintain the accuracy of the operation, the luminance residual samples are scaled / descaled. The scaling parameters used to calculate the co-located luminance The prediction may be based on a predicted sample. Y one brightness prediction sample Given a value of the filter, the scaling factor of the corresponding residual sample is calculated by the following steps: It is calculated via

[0103] In the same example, the scaling factor (e.g., the luminance residual scaling factor Scale Y )teeth , assigned luminance prediction samples as described in the box immediately following this paragraph. Calculated based on segments. Scale Y =scaleForward[Idx Y ] where Y is the luminance residual value for which the scaling factor is being calculated, and Scale Y teeth , is the scaling factor, and scaleForward[i] (i=0...15) is There are 16 predefined LUT tables, Idx Y is the brightness prediction sample domain The segment index of the segment assigned to the value. scaleForward[i] (i=0...15) is pre-computed as follows: do. scaleForward[i]=(OrgCW< <SCALE_FP_PREC) / SignaledCW[i] where OrgCW and SignaledCW[i] are the original domain and mapping. is the number of codewords in the ith segment in the coded domain, and SCALE _FP_PREC is the precision of the scaling factor.

[0104] In the same example, the brightness scaling factor Scale Y Given this, the next paragraph You can apply the luma residual sample scaling method as described in the Cut.

[0105] The motivation behind this example is that the forward mapping in the current LMCS is The advantage of this method is that it is based on a linear model. Both the original luminance samples and the luminance predicted samples are The same fragment (i.e., two pivot points InputPivot[i] and InputPi vot[i+1]) of the original luminance sample. The two forward mapping functions for the luminance samples and the predicted luminance samples are identical. So, Y'res=FwdMap(Yorg)-FwdMap(Ypred)=FwdMa p(Yorg-Ypred) == FwdMap(Yres). By applying the mapping, the corresponding decoder-side reconstruction operation can be expressed as Yrecon=Ypred+InvMap(Y'res).

[0106] In other words, the situation where both the original luma sample and the luma predicted sample are located in the same fragment. So, the luminance mapping method in LMCS can be implemented as shown in Figure 8. As implemented in the implementation, this is achieved by a single residual scaling operation in the decoding process. It can be achieved. Encoder side:

number

number

[0107] Such a conclusion is based on the fact that both the original luminance samples and the luminance predicted samples are aligned at two pivot points I The same PivotPivot[i] and InputPivot[i+1] are defined by This possible implementation of the example is based on the assumption that the The state is that the original luminance sample and the luminance predicted sample are located in different segments of the piecewise linear model. Even if the existing luminance mapping operation in VVC is simplified and / or can still be used as an approximation. Experimental results are based on such simplifications and / or This shows that the coding performance is hardly affected by the use of approximation.

[0108] Again, this example shows both the original and predicted luma sample values. are located in the same segment of a piecewise linear mode. The forward / inverse mapping functions applied to the luma samples and the predicted luma samples are the same. Therefore, we simply depend on the luma prediction samples to determine the corresponding residual scaling coefficients. It is safe to calculate.

[0109] However, if the CU prediction sample is not accurate enough (e.g., from the reference sample), Samples far away from the target are usually predicted less accurately. , the predicted samples and original samples are located in different segments of the piecewise linear model. In this case, the scaling coefficients derived based on the predicted sample values are , the residual samples in the original (i.e., unmapped) domain and the mapping The signal is then used to reflect the original mapping relationship between the residual samples in the filtered domain. It may not be reliable.

[0110] Figure 9 shows the results obtained by using only the predicted samples to derive the scaling coefficients. 9 is a diagram showing the residual mapping error. In FIG. 9, the filled triangle points represent the division The filled circular dots represent the pivot control points of different segments of the linear function. , which represent the original and predicted sample values. org and Y pred is the former The original samples and predicted samples in the (i.e., unmapped) domain This is a sample. org and Y' pred are Y org and Y pre d are the mapped samples of Y resand Y' res The existing The original domain and the mapping area when the existing sample-based intensity mapping method is applied. Y' is the corresponding residual in the filtered domain. resScale was proposed 10 is a mapped residual sample derived based on a luma residual scaling scheme. As shown in Figure 9, the original and predicted samples are in the same segment of the piecewise linear model. Since the scaling coefficients derived based on the predicted samples are not in the mapped domain (i.e., Y' res ) is a scaled residual ( That is, Y' resScale ) may not be accurate enough to generate

[0111] In the second example, both the original and predicted luminance sample values are piecewise linear models. There is no need to assume that the two are located in the same segment of the code.

[0112] In this second example, the luminance prediction samples are used to improve the accuracy of the residual scaling coefficients. Instead of deriving the scaling coefficients directly from the segment of the piecewise linear model in which the rule is located, , the scaling factor is the scaling factor of N adjacent segments (N is a positive integer). It is calculated as the average of

[0113] In this second example, we use a scaling factor for multiple luminance prediction sample segments. The step of determining a scaling factor for the luma prediction samples comprises: assigning the brightness prediction sample segments to one of the brightness prediction sample segments; The scaling factor of the measured sample is applied to the adjacent samples of the assigned luma prediction sample segment. The scaling factor is calculated as the average of the scaling factors of several luminance prediction sample segments. and

[0114] More specifically, in one possible implementation of this second example, the scaling factor is: Based on the assigned luminance prediction sample segment, as explained in the steps below. For example, at the decoder side, the luma prediction sample Pred Y and the luminance residual

number

number

number

number

number

number

[0115] In a second possible implementation of this second example, which is otherwise identical to the implementation above, The Kaling coefficients are assigned as described in the box immediately following this paragraph. The luminance prediction can be calculated based on the sample segment. 1) Pred on the original domain Y The corresponding segment index of the piecewise linear model to which it belongs KusIdx Y Find or get. 2) The luminance residual scaling factor is calculated as follows:

number

[0116] The above two possible implementations of this second example are based on the N segments allocated. The only difference is the selection of the luminance prediction sample domain value segments.

[0117] In one chroma sample reconstruction procedure, the luminance residual is calculated at the input position as shown in Figure 10. The luma prediction sample values are obtained to decode both the luma sample and the chroma residual sample (1 001), and then the luma prediction samples associated with the luma residual samples are obtained (10 02), and then the chroma prediction sample associated with the chroma residual sample is obtained (100 3) Using the luminance prediction sample, the first scaling factor of the luminance residual sample and A second scaling factor for the chroma residual samples is derived (1004), and the first scaling factor is calculated. Scale the luminance residual samples (1005) using the factor, and then The factor is used to scale the chroma residual samples (1006) and the reconstructed luma samples The sample is calculated by adding the chroma prediction sample and the scaled chroma residual sample. The reconstructed chroma samples are calculated (1007) by the chroma prediction samples and the scaled The resulting chroma residual sample is calculated by adding (1008) the resulting chroma residual sample.

[0118] The chroma sample reconstruction procedure uses the luma residual and chroma samples to achieve a more uniform design. The aim is to harmonize the scaling methods for the degree residuals.

[0119] According to one or more embodiments of the chroma sample reconstruction procedure, the luma predicted sample values are , is the average of all luma prediction samples in the coding unit (CU) containing the input position. In these embodiments, the chroma scaling derivation method calculates a scaling factor for the luma residual. More specifically, one scaling factor is used for each luma residual sample. Instead of deriving the luminance coefficients separately, we use a single luminance coefficient that is calculated based on the average of the luminance prediction samples. A shared scaling factor is used to scale luma residual samples across CUs. can be.

[0120] According to another embodiment of the chroma sample reconstruction procedure, the luma predicted sample value is calculated based on the input position. All brightness values in predefined sub-blocks subdivided from the coding unit (CU) containing In this embodiment, both the luma and chroma residuals are scaled. A sub-block based method can be proposed to derive the ring coefficients. Specifically, one CU is first divided into a number of M×N sub-blocks. For a sub-block, all or part of the luma prediction samples are used, and the sub-block The corresponding scaler is used to scale both the luma and chroma residuals of the Compared with the first method, the second method derives the ring coefficients outside the sub-block. The luma prediction samples with low correlation are excluded from the calculation of the scaling coefficients of the sub-block. Therefore, the spatial accuracy of the estimated scaling coefficients can be improved. The method also includes: after the luminance prediction of the sub-block is completed, that is, after the luminance prediction of the entire CU is completed, The luma residual and chroma residual in one sub-block are calculated immediately without waiting for the complete generation of the sample. Considering that residual scaling may be initiated, the delay of luma and chroma residual reconstruction can be reduced.

[0121] According to a third embodiment of the chroma sample reconstruction procedure, the luma predicted sample domain values are In this embodiment, the luma residual scaling is The scaling method is extended to scale the chroma residuals, resulting in a different scaling for each chroma residual sample. The scaling factor is derived based on the co-located luma prediction sample values. .

[0122] In the above embodiment of the chroma sample reconstruction procedure, to scale the chroma residuals, , it is proposed to use the same LUT as used to calculate the luminance scaling factors. In one example, the CU-level scaling factor for the chroma residual, Scale C To derive The following can be followed: 1)avg Y Luminance prediction samples in CU (expressed in the original domain), denoted as Calculate the average of 2)avg Y The corresponding segment index Idx of the piecewise linear model to which it belongs Y See Put on or get. 3) Scale C The value of is calculated as follows: Scale C =scaleForward[Idx Y ] where scaleForward[i] (i=0...15) is one predefined The 16 LUTs are calculated as follows: scaleForward[i]=(OrgCW< <SCALE_FP_PREC) / SignaledCW[i] where OrgCW and SignaledCW[i] are the original domain and is the number of codewords in the i-th segment in the mapped domain, and S CALE_FP_PREC is the precision of the scaling factors.

[0123] The above example derives the scaling coefficients for the chroma residual for each sub-block of the current CU. In this case, the first step above can be easily extended to gY is calculated as the average of the luminance prediction samples in the original domain of the subblock. , Step 2 and Step 3 remain the same.

[0124] In the second chroma sample reconstruction procedure, as shown in Figure 11, skipping some predefined intermediate luminance prediction stages during the luminance prediction process for A plurality of luminance prediction samples are obtained (1101) by Use the sample to derive the scaling factor for the chroma residual samples within a CU (11 02), and scale the chroma residual samples in the CU using the scaling factor (1 103), the reconstructed chroma samples are calculated by the chroma prediction samples and scaling within the CU. The chroma residual sample is calculated by adding the chroma residual samples (1104).

[0125] According to one or more embodiments of the second saturation sample reconstruction procedure, a predefined The mid-luminance prediction stage uses decoder-side motion vector derivation (DMVR), bidirectional optical flow One or more dual-prediction methods: BDOF, BDOF, and Combined Inter-Intra Prediction (CIIP) In these embodiments, to address the delay issue, a DMVR, a prediction module, BDOF / PROF, Inter prediction derived before CIIP intra / inter combined processing The measured samples are used to derive a scaling factor for the chroma residual.

[0126] Figure 12 shows how DMVR, BDOF, and CIIP affect luminance prediction for chroma scaling. This embodiment of the second saturation sample reconstruction procedure is not applied to generate the samples 1 is a flowchart showing a workflow of an LMCS decoding process in an example of , DMVR 1203, BDOF 1204 and / or CIIP luma intra prediction Instead of waiting for the initial L0 and L1 luminance prediction 1201 to finish completely, and 1202, as soon as the predicted samples 1221 and 1222 are available. Next, the chroma residual scaling process 1208 can begin.

[0127] In Figure 12, the DMVR 1203, BDOF 1204, and / or CIIP 1 Before 205, combine the initial L0 and L1 prediction samples 1221 and 1222. In order to do this, one additional averaging operation 1211 is required in addition to the original averaging operation 1206. be.

[0128] To reduce complexity, in the second example of this embodiment of the second chroma sample reconstruction procedure: The initial L0 prediction sample is always used to derive the scaling factor for the chroma residual. It is possible.

[0129] FIG. 13 shows how an initial single predicted signal generates luma predicted samples for chroma scaling. In a second example of this embodiment of a second chroma sample reconstruction procedure, 13 is a flowchart showing the workflow of the LMCS decoding process. No additional averaging operation is required. The initial L0 predicted sample 1321 is 1303, BDOF 1304, and / or CIIP 1305. Used to derive the Kaling coefficients.

[0130] In a third example of this embodiment of the second chroma sample reconstruction procedure, chroma residual scaling One initial prediction signal (L0 or L1) is used as the luma prediction sample to derive the coefficients. In one possible implementation of this example, the initial predicted signal ( L0 or L1) whose reference picture is lower than the current picture The ones with the point of count (POC) distance are selected to derive the saturation residual scaling coefficients. It is selected.

[0131] In another embodiment of the second chroma sample reconstruction procedure, determining a chroma residual scaling factor While enabling CIIP to generate inter prediction samples, It is suggested to disable only DMVR and BDOF / PROF. In this method, the inter-prediction samples derived before DMVR and BDOF / PROF are First, they are averaged, then they are combined with the intra prediction samples of CIIP, and finally , the combined prediction samples are used to determine the chroma residual scaling factor. Used as.

[0132] In yet another embodiment of the second chroma sample reconstruction procedure, the chroma residual scaling factor DMVR and CIIP are used to generate the predicted samples used to determine It is proposed to disable only BDOF / PROF while retaining.

[0133] In yet another embodiment of the second chroma sample reconstruction procedure, the chroma residual scaling factor When deriving the luma prediction samples used to determine Therefore, it is proposed to retain BDOF / PROF and CIIP.

[0134] Furthermore, the method in the above embodiment of the second chroma sample reconstruction procedure may further include: These are shown because they are designed to reduce the delay of differential scaling.

[0043] It is noted that the method can also be used to reduce the delay of luma prediction residual scaling. For example, see the section "Luminance Mapping Based on Prediction-Dependent Residual Scaling" All methods are also applicable to the PDRS method described in section 1.

[0135] According to the existing DMVR design, the DMVR motion fine-tuning is used to reduce the computational complexity. The prediction samples used are 2-tap bilinear interpolation instead of the default 8-tap interpolation. After the corrected motion is determined, the final predicted sample for the current CU is generated. A default 8-tap filter is applied to generate the DMV. Generated by a bilinear filter to reduce the chroma residual decoding delay caused by R The calculated luma predicted samples (L0 and L1 predicted samples if the current CU is bi-predicted) It is proposed to use the mean of ( ) to determine the scaling factor for the saturation residual.

[0136] According to one chroma residual sample reconstruction procedure, one or more The luma prediction sample values are bilinearly filtered by the decoder-side motion vector derivation (DMVR). and one or more selected luminance prediction sample values are selected from the output of Another luma or luma signals with the same bit depth as the original coding bit depth of the input video The predicted sample values are adjusted (1402) to the same bit depth as the original coding bit depth of the input video. Recover one or more chroma residual samples using luma prediction sample values with depth. A scaling factor for the signal is derived (1403), and the scaling factor is used to Scale (1404) one or more chroma residual samples to obtain one or more chroma residual samples. The residual sample is divided into one or more scaled chroma residual samples and their corresponding The saturation prediction samples are reconstructed by adding the saturation prediction samples (1405).

[0137] In one or more embodiments of the chroma residual sample reconstruction procedure, the bilinear The step of selecting one or more luminance prediction sample values from the output of the filter includes: Selecting the L0 and L1 luminance prediction samples from the output of the bilinear filter in R Includes flops.

[0138] FIG. 15 illustrates LMCS decoding in one such embodiment of the chroma residual sample reconstruction procedure. This is a flowchart showing the processing workflow. The L0 and L1 prediction samples 1521 and 1522 from the output of the filter 1512 components are Used in chroma residual scaling 1507 to decode one or more chroma residual samples. is fed to the average 1511 to derive the chroma residual scaling input 1523 to be used. will be done.

[0139] In these embodiments, there is a bit code depth issue. To save internal storage size, the image is generated by the bilinear filter of the DMVR. The intermediate L0 and L1 prediction samples are of 10-bit precision, which is equivalent to 14 bits. The representation bit depth is different from that of the usual bi-predictive immediate prediction samples. The intermediate prediction samples output from the near filter are used to estimate the chroma residual samples due to their different accuracy. It cannot be directly applied to determine the Kaling coefficient.

[0140] To address this issue, we first consider the DMVR intermediate bit depth as a normal motion compensated interpolation. Match the intermediate bit depth used, i.e., the bit depth from 10 bits to 14 bits. It is then proposed to increase the number of bits to generate a normal bi-predictive signal. We reuse the existing averaging process to determine the chroma residual scaling factor. Prediction samples can be generated.

[0141] In one example of these embodiments, one or more selected luminance prediction sample values are input. Another one or more luminance predictors with the same bit depth as the original coding bit depth of the input video. The output of the DMVR bilinear filter is adjusted to the measured sample value by left shifting. Increasing the internal bit depth of the L0 and L1 luma prediction samples from and by averaging the 14-bit shifted L0 and L1 luminance prediction sample values. By taking the 14-bit average luminance prediction sample value and shifting it right, the 14-bit average Changing the internal bit depth of the luma prediction sample values to the original coding bit depth of the input video and converting the 14-bit average luminance predicted sample value by:

[0142] More specifically, in this example, the saturation scaling factor is listed in the box immediately following this paragraph: It is determined by the steps described. 1) Internal bit depth matching. The image is generated by a bilinear filter as shown below. The internal bit depth of the generated L0 and L1 prediction samples is increased from 10 bits to 14 bits. Increase to.

number

number

number

number

number

number

number

[0143] In another embodiment of the chroma residual sample reconstruction procedure, the output of the bilinear filter of the DMVR is select one or more luminance prediction sample values from the input, and one or more luminance prediction sample values having the same bit depth as the luminance value. The step of adjusting the selected luminance prediction sample value of the DMVR includes: Select one luminance prediction sample from the L0 and L1 luminance prediction samples from the output of The step of shifting the internal bit depth of one luminance prediction value selected by the shift is One selected luma prediction sample is converted to its original coding bit depth. and adjusting the brightness of the input video to a bit depth equal to the original encoding bit depth of the input video. and using the adjusted luma prediction samples as prediction samples.

[0144] FIG. 16 shows another embodiment of the chroma residual sample reconstruction procedure, in which the LMC A flowchart showing the workflow of the S decoding process. The L0 prediction samples 1621 from the output of the filter 1612 are used to predict one or more chromatic This is used in chroma residual scaling 1607 to decode the chroma residual samples. In this case, the initial single predicted sample (i.e., the L0 predicted sample) is directly used to calculate the chroma. It is proposed to derive a scaling factor for the residual.

[0145] In one such alternative embodiment of the chroma residual sample reconstruction procedure, the current CU Assuming that is bi-predicted, the chroma scaling factors are listed in the box immediately following this paragraph. As described above, the luminance samples output from the bilinear filter are converted to the original code of the input video. The bit depth is determined by shifting the bit depth to the encoding bit depth. If the bit depth is 10 or less,

number

number

[0146] Finally, instead of generating luma prediction samples, we use reference samples (i.e., external reference pixels). Scaling the chroma residuals directly using the integer-positioned samples taken from the texture In one or more embodiments, it is proposed to determine the coefficients of the chroma residual scale. To determine the ring coefficient, the average of the reference samples at L0 and L1 is used. In another embodiment, to calculate the chroma residual scaling factor, It can only be proposed to the reference sample of the direction (e.g., list L0).

[0147] According to the second chroma residual sample reconstruction procedure, as shown in FIG. 17, one or more Luminance reference sample values are selected (1701) from a reference picture, and one or more selected The obtained luminance reference sample values are converted to luminance sample values (1702), and the converted luminance samples are The scaling factor is derived using the pull (1703), and the scaling factor is used to calculate 1 Scale (1704) one or more chroma residual samples and The residual sample is divided into one or more scaled chroma residual samples and their corresponding The saturation prediction samples are reconstructed by adding the saturation prediction samples (1705).

[0148] In one or more embodiments of the second chroma residual sample reconstruction procedure, Select one or more luminance reference sample values from the The step of converting sample values to luma sample values is performed by converting L0 and L1 reference pictures to L1. The steps of obtaining both 0 and L1 luma reference sample values and converting the luma samples are and averaging the luminance reference sample values of L0 and L1 as the L value.

[0149] In another embodiment of the second chroma residual sample reconstruction procedure, one or a plurality of luminance reference samples are selected from the reference picture, and one or a plurality of selected luminance reference samples are converted to luminance sample values. The step of selecting one luminance reference sample value from the L0 and L1 luminance reference sample values from the L0 and L1 reference pictures, and the step of using the one selected luminance reference sample value as the converted luminance sample value, are included.

[0150] According to the existing LMCS design, the reconstructed luminance samples adjacent to the 64x64 region where the current CU is located are used to calculate the chroma residual scaling coefficient of the CU within the region. Further, one clipping operation, i.e., Clip1(), is applied to clip the reconstructed luminance adjacent samples to the dynamic range of the internal bit depth ([0, (1<<bitDepth)-1]) before the average is calculated.

[0151] Specifically, the method first fetches the 64 left adjacent luminance samples and the 64 top adjacent luminance samples of the corresponding 64x64 region to which the current CU belongs. Then, the average of the left and top adjacent samples, i.e., avgY, is calculated, and the segment index Y of avgY within the LMCS piecewise linear model is found. Finally, the 1dx chroma residual C Scal eInv = cScaleInv[Y 1dx is derived.

[0152] Specifically, in the current VVC draft, how to derive the corresponding average luminance is and the clipping operation Clip 1() is shown in a prominent font size. It is applied so that The following ordered steps are applied to derive the variable varScale: The variable invAvgLuma is derived as follows: - The array recLuma[i] (i=0(2*sizeY-1)) and the variable cnt are It is derived as follows: The variable cnt is set equal to 0. If -availL is equal to TRUE, the array recLuma[i](i=0..si zeY-1 is currPic[xCuCb-1][Min(yCuCb+i,pic_ height_in_luma_samples-1)](i=0..sizeY-1) and cnt is set equal to sizeY. -If availT is equal to TRUE, the array recLuma[cnt+i](i=0 ..sizeY-1) is currPic[Min(xCuCb+i,pic_widt h_in_luma_samples-1)][yCuCb-1](i=0..size Y-1) and cnt is set equal to (cnt+sizeY). The variable invAvgLuma is derived as follows: If -cnt is greater than 0, the following applies:

number

[0153] However, in the reconstruction process, the predicted samples are added to the residual samples of one CU. Afterwards, the resulting sample values are already clipped to the dynamic range of the internal bit depth. This means that all neighboring reconstructed luma samples around the current 64x64 region are , which means that they are guaranteed to be within the range of the internal bit depth. The average of the clips, i.e., avgY, cannot exceed this range. Clip1() calculates the corresponding chroma residual scaling factor. To further reduce the complexity and memory requirements of the LMCS design, adjacent Remove the clipping operation from calculating the average of the reconstructed luma samples, and It is proposed to derive a degree residual scaling factor.

[0154] FIG. 18 is a flow diagram showing the steps of the unclipping chroma residual scaling factor derivation procedure. In FIG. 18, a first predetermined area adjacent to a second predetermined area in which a CU is located is shown. A plurality of reconstructed luma samples from the region are selected during decoding of the CU (1801); The average of the multiple reconstructed luminance samples is calculated (1802), and the multiple reconstructed luminance samples are averaged. The average of the intensity samples is used to derive the chroma residual scaling factor for decoding the CU. Used directly without clipping (1803).

[0155] In one or more embodiments of the non-clipping chroma residual scaling factor derivation procedure, The average of the multiple reconstructed luminance samples is the arithmetic mean of the multiple reconstructed luminance samples.

[0156] In one or more embodiments of the non-clipping chroma residual scaling factor derivation procedure, When deriving the chroma residual scaling coefficients for decoding a CU, multiple reconstructed brightness The step of directly using the mean of the degree samples without clipping is the predefined division line. identifying an average segment index in the shape model; Deriving Chromium Residual Scaling Coefficients for Decoding CU Based on the Gradient of Shape Model The method includes the steps of:

[0157] In one or more embodiments of the non-clipping chroma residual scaling factor derivation procedure, generating luma prediction samples and luma residual samples within a first predetermined region; adding the luma residual samples to the luma prediction samples; and clipping the first image to a dynamic range of the encoded bit depth. A plurality of reconstructed luminance samples within a predetermined region of the image are generated.

[0158] In one or more embodiments of the non-clipping chroma residual scaling factor derivation procedure, The multiple reconstructed luma samples are then converted into multiple forward-mapped inter-luma reconstruction samples. It is.

[0159] In one or more embodiments of the unclipping chroma residual scaling factor derivation procedure, The second predetermined area is a 64×64 area in which the CU is located.

[0160] In one example, as shown in FIG. 19, the first predetermined area is a true area of the second predetermined area 1904. This may include the upper adjacent samples in the upper 1x64 region 1902. In addition, the first predetermined region is a 64x1 region immediately to the left of the second predetermined region 1904 and may include the left neighboring samples within 1903.

[0161] According to the existing LMCS design, the reconstructed samples in both the original domain and the mapped domain are used for CUs encoded in different modes. Correspondingly, a plurality of LMCS transforms are involved in the current encoding / decoding process to convert the predicted and reconstructed luminance samples between the two domains. Specifically, in the intra mode, CIIP mode, and IBC mode, the reference samples from the adjacent reconstructed domains of one current CU used to generate intra prediction samples are maintained within the mapped domain. In contrast, in the CIIP mode and all inter modes, the motion compensated prediction samples generated from the temporal reference pictures are in the original domain. Since the luminance reconstruction operation is performed in the mapped domain, those inter prediction samples of the luminance component need to be converted to the mapped domain before being added to the residual samples. On the other hand, for both intra and inter modes, the inverse mapping is always applied to the reconstructed luminance samples that are converted from the mapped domain to the original domain.

[0162]

[0163]

[0164] <000开1949>

[0165] [[ID=-1]] [[ID=-2]]

[0166] [[ID=-3]] [[ID=-4]] [[ID=-5]] [[ID=-6]] [[ID=-7]] [[ID=-8]] [[ID=-9]] [[ID=-10]] [[ID=-ll]] [[ID=-12]] [[ID=-13]] [[ID=-14]] [[ID=-15]] [[ID=-16]] [[ID=-17]] [[ID=-l8]] [[ID=-19]] [[ID=-20]] [[ID=-21]] [[ID=-22]] [[ID=-23]] [[ID=-24]] [[ID=-25]] [[ID=-26]] [[ID=-27]] [[ID=-28]] [[ID=-29]] [[ID=-30]] [[ID=-31]] [[ID=-32]] [[ID=-33]] [[ID=-34]] [[ID=-35]] [[ID=-36]] [[ID=-37]] [[ID=-38]] [[ID=-39]] [[ID=-40]] [[ID=-41]] [[ID=-42]] [[ID=-43]] [[ID=-44]] [[ID=-45]] [[ID=-46]] [[ID=-47]] [[ID=-48]] [[ID=-49]] [[ID=-50]] [[ID=-51]] [[ID=-52]] [[ID=-53]] [[ID=-54]] [[ID=-55]] [[ID=-56]] [[ID=-57]] [[ID=-58]] [[ID=-59]] [[ID=-60]] [[ID=-61]]

[0167] [[ID=-62]] [[ID=-63]] [[ID=-64]] [[ID=-65]] [[ID=-66]] [[ID=-67]] [[ID=-68]] [[ID=-69]] [[ID=-70]] [[ID=-71]] [[ID=-72]] [[ID=-73]] [[ID=-74]] [[ID=-75]] [[ID=-76]] [[ID=-77]] [[ID=-78]] [[ID=-79]] [[ID=-80]] [[ID=-81]] [[ID=-82]] [[ID=-83]] [[ID=-84]] [[ID=-85]] [[ID=-86]] [[ID=-87]] [[ID=-88]] [[ID=-89]] [[ID=-90]] [[ID=-91]] [[ID=-92]] [[ID=-93]] [[ID=-94]] [[ID=-95]] [[ID=-96]] [[ID=-97]] [[ID=-98]] [[ID=-99]] [[ID=-100]] [[ID=-101]] [[ID=-102]] [[ID=-103]] [[ID=-104]] [[ID=-105]] [[ID=-106]] [[ID=-107]]

[0168] [[ID=-108]] [[ID=-109]] [[ID=-110]] [[ID=-111]] [[ID=-112]]

[0169] [[ID=-1l3]] [[ID=-114]] [[ID=-115]] [[ID=-1l6]] [[ID=-117]] [[ID=-118]] [[ID=-119]] [[ID=-120]] [[ID=-121]] [[ID=-122]] [[ID=-123]]

[0170] [[ID=-124]] [[ID=-125]] [[ID=-126]] [[ID=-127]] [[ID=-128]] [[ID=-129]] [[ID=-130]] [[ID=-l31]] [[ID=-132]] [[ID=-133]] [[ID=-134]] [[ID=-135]] Meanwhile, for both intra and inter modes, the reconstructed luma samples are The same clipping operation is performed on the reconstructed luma samples after they have been transformed back to the original domain. also applies to

[0164] However, based on the existing LMCS design, the samples obtained from the forward LMCS Bitstream control that ensures that the dynamic range of the internal bit depth is always within the dynamic range of the internal bit depth. There is one problem. This is because the inter-CU mapped luminance prediction samples are This means that the dynamic range cannot be exceeded. The existing clipping operation applied to the mapped luma prediction samples of the code is redundant. As an example, the order of inter prediction samples in inter mode and CIIP mode is It may be proposed to remove the clipping operation after the orientation change. When converting the mapped luminance samples from the mapped domain to the original domain, the inverse L It may be proposed to remove the clipping operation from the MCS mapping process.

[0165] More specifically, in the current VVC draft, the inverse mapping process for luma samples is as follows: The clipping operation Clip 1() in formula (1242) is explained as follows: Applied as shown in size. 8.8.2.2 Inverse Mapping of Luminance Samples The input to this process is the luma sample lumaSample. The output of this process is the modified luma sample invLumaSample. The value of invLumaSample is derived as follows: -slice_lmcs_enab for slices containing luma samples If led_flag is equal to 1, the following ordered steps are applied: 1. The variable idxYInv takes lumaSample as input and outputs idxYInv. As input, the piecewise function index of the luminance samples, as specified in clause 8.8.2.3, is used. The ID of the .NET process is derived by calling the .NET process ID. 2. The variable invSample is derived as follows:

number

number

[0166] Additionally, the current VVC draft also includes weighted samples for joint merging and intra prediction. The pull prediction process is explained below and is shown here in prominent font size. As you can see, the clipping operation Clip 1() has been applied. 8.5.6.7 Weighted Sample Prediction Processing for Join Merge and Intra Prediction The inputs to this process are: - the top left sample of the current luma coding block relative to the top left luma sample of the current picture Luminance position (xCb, yCb) to specify the pull - the width of the current coding block, cbWidth, - current coding block height cbHeight - Two (cbWidth) x (cbHeight) arrays predSamplesIn ter and predSamplesIntra - Variable cIdx that specifies the color component index The output of this process is a (cbWidth)×(cbHeight) array of predicted sample values predSamplesComb The variable scallFact is derived as follows [Number] The adjacent luminance positions (xNbA, yNbA) and (xNbB, yNbB) are respectively , (xCb - 1, yCb - 1+(cbHeight << scallFactY)) and (xCb - 1+(cbWidth << scallFactX), yCb - 1), are equally Set When X is replaced by either A or B, the variables availableX and i sIntraCodedNeighbourX are derived as follows - In the derivation process of adjacent block availability specified in Clause 6.4.4, the position (xCurr, yCurr) set equal to (xCb, yCb ), the adjacent position (xNbX, yNbX) set equal to (xNbY, yNbY), the checkPredModeY set equal to FALSE, and the cIdx set equal to 0 are called as inputs , and the output is assigned to availableX - The variable isIntraCodedNeighbourX is derived as follows - When availableX is equal to TRUE and CuPredMode[0][xNbX [yNbX] is equal to MODE_INTRA, isIntraCodedNei ghbourX is set equal to TRUE - Otherwise, isIntraCodedNeighbourX is set equal to FALSE - Otherwise, isIntraCodedNeighbourX is set equal to FALSE and equal The weight w is derived as follows: -isIntraCodedNeighbourA and isIntraCodedNei If both ghbourB are equal to TRUE, then w is set equal to 3. -If not, isIntraCodedNeighbourA and isIntra If both CodedNeighborA and CodedNeighborB are equal to FALSE, then w is set equal to 1. will be done. Otherwise, w is set equal to 2. cIdx is equal to 0 and slice_lmcs_enabled_flag is equal to 1 If not, have x=0..cbWidth-1 and y=0..cbHeight-1 predSamplesInter[x][y] is modified as follows:

number

number

[0167] Furthermore, the current VVC draft does not support picture reconstruction using a mapping process of luminance samples. The construction is explained below, and the clipping operation Clip 1() The standards are applied as shown in the table. 8.7.5.2 Picture reconstruction by luminance sample mapping The inputs to this process are: - The position of the top left sample of the current block relative to the top left sample of the current picture (x Curr, yCurr), -nCurrSw variable that specifies the block width -nCurrSh variable that specifies the block height - (nCurrSw) x (nCurrS) specifies the luma prediction sample for the current block h) Array predSamples - (nCurrSw) x (nCurrS) specifies the luminance residual sample of the current block h) Array resSamples The output of this process is the reconstructed luma picture sample array recSamples. be. Mapped predicted luminance samples predMapSamples (nCurrSw )×(nCurrSh) array is derived as follows: - predMapSamples[i][j] if one of the following conditions is true: ] is pred for i=0..nCurrSw-1, j=0..nCurrSh-1. Set equal to Samples[i][j]. -CuPredMode[0][xCurr][yCurr] is MODE_INTRA is equal to. -CuPredMode[0][xCurr][yCurr] is MODE_IBC and equal. -CuPredMode[0][xCurr][yCurr] is MODE_PLT and equal. -CuPredMode[0][xCurr][yCurr] is MODE_INTER and ciip_flag[xCurr][yCurr] is equal to 1. - Otherwise (CuPredMode[0][xCurr][yCurr] is MO DE_INTER and ciip_flag[xCurr][yCurr] equals 0 (required), the following applies:

number

number

[0168] These redundant clipping operations increase the computational complexity and overhead in existing LMCS designs. This leads to extra on-chip memory requirements. To reduce this, it is proposed to remove these redundant clipping operations.

[0169] According to the non-clipping chroma sample decoding procedure, the chroma scaling is Intermodal or combined under the Luminance Mapping with Coding (LMCS) framework Coding units (CUs) coded using inter-intra prediction (CIIP) mode During the decoding of , the reconstructed samples of the luma component are (2001) and multiple reconstructed samples of the luminance component are mapped to the domain. By transforming from the original domain to the original, multiple transformed samples of the luminance component are (2002) and the saturation scale for decoding the saturation samples of the CU. When deriving the ring coefficients, multiple transformed luminance component samples in the original domain are used. but without clipping, as used in (2003).

[0170] In one or more embodiments of the non-clipping chroma sample decoding procedure, the So, if a CU is coded by inter-mode, the mapped domain is The step of obtaining a plurality of reconstructed samples of the luminance component in the original domain is Calculating (2101) a plurality of inter-predicted samples of a luminance component in a mapping to obtain multiple transformed inter-predicted samples of the luma component in the filtered domain. To achieve this, multiple inter-predicted samples of the luma component are mapped from the original domain to the a step (2102) of transforming, without clipping, the A plurality of transformed inter-predicted samples of the luma component in the mapped domain is added to multiple residual samples of the luminance component in the mapped domain, and the result is As a result, multiple reconstructed samples of the luminance component in the mapped domain are obtained. and (2103)

[0171] In one or more other embodiments of the non-clipping chroma sample decoding procedure, the As shown, when a CU is coded in CIIP mode, the mapped domain The step of obtaining a plurality of reconstructed samples of the luminance component in the original domain Calculating (2201) a plurality of inter-predicted samples of a luminance component in a multi-pixel image; Taking multiple transformed inter-predicted samples of the luma component in the mapped domain. To obtain the inter-predicted samples of the luma component, multiple inter-predicted samples are mapped from the original domain. a step of transforming (2202) the mapped image into the domain without clipping; calculating a plurality of intra prediction samples of the luma component in the determined domain (22 03) and the addition of multiple transformed inter-predicted samples and multiple intra-predicted samples. Derive predicted samples of the luminance component in the mapped domain by weighted averaging. Step (2204) and The derived predicted samples of the luminance component in the mapped domain are then mapped The resulting mapping is A step (2) in which a plurality of reconstructed samples of the luminance component in the reconstructed domain are obtained. 205) and

[0172] With the current LMCS enabled, forward and reverse luminance mapping is 11-bit This is done using one LUT table defined with a precision of 1000 s. For example, Taking the mapping example, the current forward mapping scaling factor is defined as follows: will be done. ScaleCoeff[i]=(lmcsCW[i]*(1<<11)+(1<<(L og 2(OrgCW)-1)))>>(Log 2(OrgCW)) where lmcsCW is the length of one segment of the mapped luminance domain. OrgCW is one of the original luminance domains, where 1<<(BitDepth-4) is the length of the segment.

[0173] However, this precision of 11 bits is achieved with an internal coding bit depth of less than 16 bits. It turns out that this is only sufficient for L when the internal coding bit depth is 16 bits. The value of og2 (OrgCW) is 12. In such a case, the 11-bit precision improvement is This is not sufficient to support scaling factor derivation. If Ream conformance is applied, i.e., the segmentation in the mapped luminance domain Even when the total length of the ment is less than or equal to (1<<BitDepth)-1, the forward luminance The predicted luminance sample value after mapping may exceed the dynamic range of the internal coding bit depth.

[0174] Based on such considerations, two solutions are proposed.

[0175] In one solution, it is proposed to always apply a clipping operation to the mapped luminance predictions in the inter mode and CIIP mode. Moreover, the current bit stream compatibility can be removed.

[0176] Under this solution, the weighted sample prediction processing for merge and intra prediction is as follows. Compared with the specification of the same procedure in the current VVC draft, the clipping operation Clip 1 () is always applied to Equation (1028 a) so that the mapped luminance prediction sample contains the clipping value. 8.5.6.7 Weighted sample prediction processing for merge and intra prediction The input to this process is as follows. When cIdx is equal to 0 and slice_lmcs_enabled_flag is equal to 1, predSamplesInter[x][y] with x = 0..cbWidth-1 and y = 0..cbHeight-1 is modified as follows. When x = 0..cbWidth-1 and y = 0..cbHeight-1, predSamplesComb[x][y] is derived as follows.

Equation

Equation

[0177] Even with this solution, picture reconstruction using the luminance sample mapping process is Compared to the specification of the same procedure in the current VVC draft, the mapped brightness A clipping operation Clip1() is added so that the predicted sample contains the clipping value. will be added. 8.7.5.2 Picture reconstruction by luminance sample mapping The inputs to this process are: - The position of the top left sample of the current block relative to the top left sample of the current picture (x Curr, yCurr), -nCurrSw variable that specifies the block width -nCurrSh variable that specifies the block height - (nCurrSw) x (nCurrS) specifies the luma prediction sample for the current block h) Array predSamples - (nCurrSw) x (nCurrS) specifies the luminance residual sample of the current block h) Array resSamples The output of this process is the reconstructed luma picture sample array recSamples. be. Mapped predicted luminance samples predMapSamples (nCurrSw )×(nCurrSh) array is derived as follows: - predMapSamples[i][j] if one of the following conditions is true: ] is pred for i=0..nCurrSw-1, j=0..nCurrSh-1. Set equal to Samples[i][j]. -CuPredMode[0][xCurr][yCurr] is MODE_INTRA is equal to. -CuPredMode[0][xCurr][yCurr] is MODE_IBC and equal. -CuPredMode[0][xCurr][yCurr] is MODE_PLT and equal. -CuPredMode[0][xCurr][yCurr] is MODE_INTER and ciip_flag[xCurr][yCurr] is equal to 1. - Otherwise (CuPredMode[0][xCurr][yCurr] is MO DE_INTER and ciip_flag[xCurr][yCurr] equals 0 (required), the following applies:

number

number

[0178] In the second solution, the precision of the LMCS scaling coefficient derivation is increased to 11 bits (M bits and It is proposed to increase the

[0179] Under this second solution, the forward luminance mapping becomes: ScaleCoeff[i]=(lmcsCW[i]*(1< <M)+(1<<(Lo g 2(OrgCW)-1)))>>(Log 2(OrgCW)) idxY=predSamples[i][j]>>Log2(OrgCW) predMapSamples[i][j]=LmcsPivot[idxY]+ (ScaleCoeff[idxY]*(predSamples[i][j]-In putPivot[idxY])+(1<<(M-1)))>>M(1219) with i=0..nCurrSw-1,j=0..nCurrSh-1

[0180] In one embodiment of the second solution, scaling both the forward and inverse luminance mapping It is proposed to increase the accuracy of the coefficient derivation. It is proposed to only increase the accuracy of the scaling factor derivation for the degree mapping.

[0181] If either of the above two solutions is applied, Intermode and CIIP mode It also safely removes any current clipping operation applied to the mapped luma prediction samples of the can be removed.

[0182] According to a first aspect of the present disclosure, as shown in FIG. 23, Mapping of the luminance component of a CU coded in center mode or CIIP mode A plurality of prediction samples in the selected domain are obtained (2301), and the luminance component of the CU is , where multiple residual samples in the mapped domain are received from the bitstream. (2302) A plurality of predicted samples in the mapped domain are The resulting luminance component is added to the residual samples in the domain of interest. A number of reconstructed samples in the filtered domain are obtained (2303), and multiple samples of the luminance component are obtained. The number of reconstructed samples is determined based on a number of predefined inverse mapping scaling factors. , is converted from the mapped domain to the original domain (2304).

[0183] In one or more embodiments of the first aspect of the present disclosure, as illustrated in FIG. U is coded by inter-mode and is the mapped domain of the luminance component of CU. Obtaining multiple prediction samples in the CU is done by obtaining the luminance of the CU from the temporal reference picture of the CU. Deriving a plurality of inter-prediction samples in the original domain of the components (2401). and then a predefined coding bit depth and a predefined forward mapping Based on multiple predefined forward mapping scaling factors within the accuracy Transforms multiple inter-predicted samples of a component from the original domain to the mapped domain. and converting (2402).

[0184] In one or more other embodiments of the first aspect of the present disclosure, as illustrated in FIG. , CU is coded by CIIP mode, and the luminance component of CU is mapped to the domain Obtaining multiple prediction samples in the CU from the CU's temporal reference picture Deriving a plurality of inter-prediction samples in the original domain of the luma component (250 1) with a predefined coding bit depth and a predefined forward mapping precision. based on a number of predefined forward mapping scaling factors in Transform multiple inter-prediction samples from the original domain to the mapped domain. (2502) and the luminance component of the CU is mapped to multiple images in the domain. 2503, and converting the converted inter-prediction samples. as a weighted average of multiple intra- and intra-prediction samples to the mapped domain. and deriving (2504) a predicted sample of the luma component of the CU in the

[0185] In one example, as shown in FIG. 26, a predefined encoding bit depth and a predefined Multiple predefined forward mapping scalings within the given forward mapping accuracy Based on the coefficients, multiple inter-predicted samples of the luma component are mapped from the original domain. Transforming into the scaled domain is done using multiple predefined forward mapping scales. The mapping coefficients are used to map multiple inter-predicted samples of the luma component from the original domain. Transforming the encoded data into a predefined encoding domain (2601) and and based on a predefined forward mapping precision, a clipping operation is required. and determining whether a clipping operation is required (2602). In response, a plurality of inter-predicted samples of the luminance component in the mapped domain are Clipping to a predefined coding bit depth (2603) and In response to determining that no operation is required, a cross-prediction of the plurality of inter-predicted samples of the luma component is performed. and bypassing ripping (2604).

[0186] Determining whether a clipping operation is necessary in one or more cases is the predefined coding bit depth is greater than the predefined forward mapping precision. determining that a clipping operation is necessary if the clipping operation is not necessary.

[0187] Determining whether a clipping operation is necessary in one or more cases is the predefined coding bit depth is smaller than the predefined forward mapping precision. determining that a clipping operation is not necessary if the

[0188] In one or more examples, determining whether a clipping operation is required includes: Regardless of the predefined coding bit depth and predefined forward mapping precision , including determining that a clipping operation is required.

[0189] In one or more examples, determining whether a clipping operation is necessary involves a predetermined Regardless of the predefined coding bit depth and predefined forward mapping precision, This includes determining that no flipping operation is required.

[0190] In one or more examples, the predefined forward mapping precision is 15 bits.

[0191] In one or more examples, the predefined forward mapping precision is 11 bits.

[0192] FIG. 27 illustrates an apparatus for video encoding according to some implementations of the present disclosure. The device 2700 is a block diagram of a mobile phone, a tablet computer, a digital broadcasting The terminal may be a transmitting terminal, a tablet device, a personal digital assistant, or the like.

[0193] As shown in FIG. 27, the device 2700 includes the following components: a processing component 2702, a memory 2704, a power component 2706, a multimedia component 2708, and a power supply component 2709. 2708 , audio components 2710, input / output (I / O) interfaces 2712, sensor components 2714, and communication components 2716.

[0194] The processing component 2702 typically handles display, call, data communication, camera operation, and recording operation. The processing component 2702 controls the overall operation of the device 2700, including operations related to the , one or more instructions for executing instructions to complete all or part of the steps of the above method. The processing component 2702 may include one or more processors 2720. To facilitate interaction between the processing component 2702 and other components, one or For example, the processing component 2702 may include a multimedia To facilitate interaction between the audio component 2708 and the processing component 2702, a multimedia The device may include a media module.

[0195] The memory 2704 stores different types of data to support the operation of the device 2700. Examples of such data include any data that is executed on the device 2700. instructions for applications or methods, contact data, phone book data, messages , photos, videos, etc. Memory 2704 may store any type of temporary or non-temporary storage. The memory 2704 may include a static RAM, a memory card, a storage medium, or a combination thereof. Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), erasable programmable read-only memory (EPROM), program Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), Magnetic Memory , flash memory, magnetic disk, or compact disk. The storage medium may be, for example, a hard disk drive (HDD), a solid state drive ( SSD), flash memory, hybrid drive or solid state hybrid Hard Drive (SSHD), Read-Only Memory (ROM), Compact Disc Reader Dedicated memory (CD-ROM), magnetic tape, floppy disk, etc. That's fine.

[0196] A power supply component 2706 provides power to the different components of the device 2700. Element 2706 includes a power management system, one or more power sources, and a power supply for device 2700. The system may include other components related to the generation, management, and distribution of power.

[0197] The multimedia component 2708 provides an output interface between the device 2700 and the user. In some examples, the screen may be a liquid crystal display (LCD) or and a touch panel (TP). If the screen includes a touch panel, the screen The touch panel may be implemented as a touch screen that receives input signals from a user. The panel uses one or more touch sensors to detect touches, slides, and gestures on the touch panel. The touch sensor may include one or more touch sensors. It not only senses the boundaries of a touch or slide action, but also the duration and pressure associated with the touch or slide action. In some examples, the multimedia component 2708 may detect force. The device 2700 may include a front camera and / or a rear camera. or when in an operating mode such as video mode, the front and / or rear camera The camera can receive external multimedia data.

[0198] The audio component 2710 is configured to output and / or input audio signals. For example, the audio component 2710 includes a microphone (MIC). The device 2700 may be in various operating modes, such as a call mode, a recording mode, and a voice recognition mode. When the microphone is connected to the external audio device, the microphone is configured to receive an external audio signal. The audio signal may be further stored in memory 2704 or transmitted to communication component 2706. 16. In some examples, the audio component 2710 may be It further includes a speaker for outputting an audio signal.

[0199] The I / O interface 2712 connects the processing component 2702 to the peripheral interface module. The peripheral interface module provides an interface between the The buttons may be keyboards, click wheels, buttons, etc. This may include, but is not limited to, the Home button, Volume buttons, Start button, and Lock button. Not limited to:

[0200] The sensor component 2714 may include a sensor, a sensor array, or a sensor module for providing status assessments in different aspects of the device 2700. For example, the sensor component 2714 may include a plurality of sensors. The on / off state and relative position of the components can be detected. The display and keypad of the device 2700. The sensor component 2714 also changes in the position of device 2700 or components of device 2700, changes in the user's touch on device 2700, Detecting presence, orientation or acceleration / deceleration of device 2700, and temperature changes of device 2700 The sensor component 2714 can detect the presence of nearby objects without physical contact. The sensor component 2714 may include a proximity sensor configured to capture an image. The device may further include an optical sensor, such as a CMOS or CCD image sensor, used in the application. In some examples, the sensor component 2714 may include an accelerometer, a gyroscope, or the like. It may further include a cope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0201] The communication component 2716 provides wired or wireless communication between the apparatus 2700 and other devices. The device 2700 is configured to facilitate WiFi, 4G, or a combination thereof. In one example, a wireless network can be accessed based on a communication standard such as The communication component 2716 receives broadcast signals from an external broadcast management system via a broadcast channel. or receive broadcast-related information. In one example, the communications component 2716 facilitates short-range communications. The device may further include a near field communication (NFC) module for performing the above-described operations. The module supports Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Implementations based on Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies This may be done.

[0202] In one example, device 2700 may be an application specific integrated circuit (ASIC), a digital signal processor, or DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLD), Field Programmable Gate Array (FPGA), Controller, Micro a microcontroller, microprocessor, or other electronic device that performs the above method. It may be implemented by one or more of these.

[0203] In one or more examples, the functions described may be implemented in hardware, software, firmware, or other similar applications. If implemented in software, the method may be implemented in software, hardware, or any combination thereof. In this case, the functionality may be stored on a computer-readable medium or may be implemented as one or more instructions or The code is transmitted via a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium may be a tangible medium such as a data storage medium. or in some cases according to a communication protocol, for example. communication media, including any medium that facilitates the transfer of a computer program from one place to another Thus, a computer-readable medium generally includes: (1) a non- (2) a tangible computer-readable storage medium that is transitory; or (3) a communication medium such as a signal or carrier wave. The data storage medium may correspond to a data transmission medium. to retrieve instructions, code, and / or data structures for implementing one or more Can be accessed by multiple computers or one or more processors The computer program product may be any available medium. It may include a readable medium.

[0204] Furthermore, the above method can be implemented using the apparatus 2700. application-specific integrated circuits, programmable logic arrays, and other hardware devices The hardware implementation may include a dedicated hardware implementation of the methods described herein. The device may be configured to implement one or more of the following methods: Examples include devices and systems such as various electronic and computing systems. One or more examples described herein may broadly include inter-module systems. and communicate via a module or as part of an application specific integrated circuit. Two or more specific interconnected hardware components with associated control and data signals that can The functionality can be implemented using software modules or devices. The device or system shown is a The term "module" and "sub-module" may refer to a specific implementation. module), "circuit", "sub-circuit" , "circuitry", "sub-circuitry", The terms "unit" or "sub-unit" are used to refer to Stores code or instructions that can be executed by one or more processors The modules referred to herein may include memory (shared, dedicated, or group). A module contains one or more circuits, with or without stored code or instructions. A module or circuit may contain one or more components connected together. It is possible.

[0205] Other embodiments of the present invention may be realized from the discussion of this specification and practice of the invention disclosed herein. This application is based on the general principles of the present invention and will be apparent to those skilled in the art. any deviation from the present disclosure that comes within known or customary practice in the art. It is intended to cover any modifications, uses, or adaptations of the present invention. It is intended that the following claims be interpreted as illustrative only, and that the true scope and spirit of the invention will be determined by the appended claims. This is indicated by the range of requirements.

[0206] The present invention is not limited in scope to the precise examples described above and shown in the accompanying drawings. It will be understood that various modifications and changes can be made without departing from the spirit and scope of the present invention. It is intended that the scope of the invention be limited only by the appended claims.

Claims

1. Dividing a picture into one or more coding units (CUs); Obtaining a plurality of prediction samples in a mapped domain of a luma component of a current CU coded by a joint inter-intra prediction (CIIP) mode under a luma mapping with chroma scaling (LMCS) framework; Obtaining a plurality of residual samples in the mapped domain of the luma component of the CU; adding the prediction samples in the mapped domain to the residual samples in the mapped domain to obtain reconstructed samples in the mapped domain of the luma component of the current CU; transforming the reconstructed samples of the luma component from the mapped domain to an original domain based on a plurality of predefined inverse mapping scaling factors; obtaining prediction information of the current CU based on the reconstructed samples in the original domain to form a video bitstream; Including, Obtaining the plurality of prediction samples in the mapped domain of the luma component of the current CU includes: Deriving a plurality of inter-predicted samples in the original domain of the luma component of the current CU from a temporal reference picture of the current CU; transforming the inter-predicted samples of the luma component from the original domain to the mapped domain based on a predefined coding bit depth and a predefined number of forward mapping scaling factors within a predefined forward mapping precision; Calculating a plurality of intra-prediction samples in the mapped domain of the luma component of the current CU; deriving the predicted sample of the luma component of the current CU in the mapped domain as a weighted average of the transformed inter predicted samples and the intra predicted samples; 1. A method for video encoding, comprising:

2. Transforming the inter-predicted samples of the luma component from the original domain to the mapped domain based on the predefined coding bit depth and the predefined forward mapping scaling factors within the predefined forward mapping precision, includes: transforming the inter-predicted samples of the luma component from the original domain to the mapped domain using the predefined forward mapping scaling factors; clipping the inter-predicted samples in the mapped domain of the luma component to a dynamic range of the predefined coding bit depth; The method of claim 1 , comprising:

3. The method of claim 1 , wherein the predefined forward mapping precision is 11 bits.

4. 1. A computing device comprising: one or more processors; a non-transitory storage device coupled to the one or more processors; a plurality of programs stored on the non-transitory storage device that, when executed by the one or more processors, cause the computing device to perform the method of any one of claims 1 to 3; a computing device,

5. 4. A non-transitory computer-readable storage medium storing a plurality of programs that are executed by a computing device having one or more processors, the plurality of programs, when executed by the one or more processors, cause the computing device to perform the method of any one of claims 1 to 3 to generate a bitstream, and store the bitstream in the non-transitory computer-readable storage medium.

6. A method for storing a bitstream, comprising: Executing the method for video coding according to any one of claims 1 to 3 to generate the bitstream; storing the bitstream; A method comprising:

Images