Method for video encoding, computing device, non-transitory computer-readable storage medium, computer program, and method for storing a bitstream

Abstract

To provide methods and devices for rectifying a forward mapping coding bit length issue introduced by LMCS.SOLUTION: In one method, a plurality of prediction samples, in a mapped domain, of luma component of a coding unit (CU) that is coded by an inter mode or combined inter and intra prediction (CIIP) mode under luma mapping with chroma scaling (LMCS) framework are obtained; a plurality of residual samples, in the mapped domain, of the luma component of the CU are received from a bitstream; the plurality of prediction samples in the mapped domain are added to the plurality of residual samples in the mapped domain, so that a plurality of reconstructed samples of the luma component in the mapped domain are obtained; and the plurality of reconstructed samples of the luma component are converted from the mapped domain into an original domain based on a pre-defined plurality of inverse mapping scaling factors.SELECTED DRAWING: Figure 23

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is a joint venture of U.S. Provisional Patent Application No. 63 / 043,569, filed June 24, 2020. The entire disclosure of the aforementioned application is hereby incorporated in its entirety for all purposes. The body is incorporated herein by reference.

[0002] The present invention relates generally to video encoding and compression. More particularly, the present invention relates to a method for encoding and compressing video signals. Prediction dependent residual scaling for the scaling units Perform video encoding using 3D residual scaling (PDRS) The present invention relates to a system and method for [Background technology]

[0003] This section provides background information related to the present disclosure. The information should not necessarily be construed as prior art.

[0004] Use any of a variety of video encoding techniques to compress the video data The video encoding may be performed according to one or more video encoding standards. Some exemplary video coding standards are Versatile Video Coding (VersaVo) and tile video coding (VVC), joint search test model (joint exploration model,JEM) coding, high-efficiency video coding (H.26 5 / HEVC), Advanced Video Coding (H.264 / AVC), and Video Extraction Includes Part Group (MPEG) encoding.

[0005] Video encoding generally utilizes prediction methods (e.g., inter prediction, intra prediction, etc.) that exploit the redundancy inherent in a video image or sequence. One of the objectives of video encoding technology is to compress video data into a form that uses a lower bitrate while avoiding or minimizing degradation of video quality. For example, inter prediction, intra prediction, etc.). The purpose of video encoding technology One is to compress the video data into a form using a lower bitrate while avoiding or minimizing degradation of video quality.

SUMMARY OF THE INVENTION

PROBLEMS TO BE SOLVED BY THE INVENTION

[0006] This section provides a general overview of the present disclosure and is not an exhaustive disclosure of its full scope or all of its features.

MEANS FOR SOLVING THE PROBLEMS

[0007] According to a first aspect of the present application, a plurality of prediction samples in the mapped domain of the luminance component of an encoded unit (CU) encoded by a combined inter and intra prediction (CIIP) mode under the framework of luma mapping with chroma scaling (LMCS) are obtained, a plurality of residual samples in the mapped domain of the luminance component of the CU are obtained, the plurality of prediction samples in the mapped domain are added to the plurality of residual samples in the mapped domain to obtain a plurality of reconstructed samples in the mapped domain of the luminance component, and the plurality of reconstructed samples of the luminance component are converted from the mapped domain to the original domain based on a plurality of predefined inverse mapping scaling coefficients. , Knot is obtained According to a second aspect of the present application, a method for video decoding is provided. This method includes obtaining a plurality of prediction samples in a mapped domain of a luminance component of a coded unit (CU) encoded by a combined inter-intra prediction (CIIP) mode under a framework of luminance mapping with chroma scaling (LMCS), obtaining a plurality of residual samples in the mapped domain of the luminance component of the CU, adding the plurality of prediction samples in the mapped domain to the plurality of residual samples in the mapped domain to obtain a plurality of reconstructed samples in the mapped domain of the luminance component, and converting the plurality of reconstructed samples of the luminance component from the mapped domain to the original domain based on a plurality of predefined inverse mapping scaling coefficients. In one embodiment, obtaining a plurality of prediction samples in the mapped domain of the luminance component of the CU includes deriving a plurality of inter prediction samples in the original domain of the luminance component of the CU from a temporal reference picture of the CU, and converting the plurality of inter prediction samples of the luminance component from the original domain to the mapped domain based on a plurality of predefined forward mapping scaling coefficients within a predefined coding bit depth and a predefined forward mapping accuracy. In one embodiment, converting the plurality of inter prediction samples of the luminance component from the original domain to the mapped domain based on a plurality of predefined forward mapping scaling coefficients within a predefined coding bit depth and a predefined forward mapping accuracy includes converting the plurality of inter prediction samples of the luminance component from the original domain to the mapped domain without a clipping operation using the plurality of predefined forward mapping scaling coefficients. In one embodiment, the predefined forward mapping accuracy is 11 bits.

[0008] ​​​​ According to the 3 aspect of the present application, a computing device includes one or more processors, a memory, and a plurality of programs stored in the memory. When the programs are executed by the one or more processors, the computing device is caused to perform the operations as described above in the first aspect Or the second aspect of the present application.

[0009] According to the 4 aspect of the present application, a non-transitory computer-readable storage medium stores a plurality of programs that are executed by a computing device having one or more processors. When the programs are executed by the one or more processors, the computing device is caused to perform the operations as described above in the first aspect Or the second aspect of the present application.

[0010] Hereinafter, a set of exemplary non-limiting embodiments of the present disclosure will be described in conjunction with the accompanying drawings . Modifications of structure, method, or function may be made by those skilled in the art based on the examples presented herein and such modifications are all included within the scope of the present disclosure. Where there are no contradictions , the teachings of different embodiments can be combined with each other, but it is not necessary to combine them.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Fig. 24

Fig. 25

Fig. 26

Fig. 27

Best Mode for Carrying Out the Invention

[0012] The terms used in the present disclosure are not for limiting the present disclosure, but for illustrating specific examples. Thus, the singular forms "a", "an" and "the" used in the present disclosure and the appended claims The term "the" also refers to the plural form unless the context clearly indicates otherwise. As used in this specification, the term "and / or" shall be understood to mean any and all possible combinations of one or more of the associated listed items.

[0013] In this specification, terms such as "first", "second", "third", etc. may be used to describe various information, but it should be understood that the information should not be limited by these terms. These terms are only used to distinguish one category of information from another. For example, without departing from the scope of the present disclosure, the first information may be referred to as the second information, and vice versa. When used in this specification, the term "if" may, depending on the context, be understood to mean "when", "upon", or "in response to".

[0014] Throughout this specification, references to one or more "embodiments", "an embodiment", "another embodiment", etc. mean that one or more specific features, structures, or characteristics described in connection with the embodiment are included in at least one embodiment of the present disclosure. Thus, the appearances of the phrases "in one embodiment", "in an embodiment", "in another embodiment", etc. at various places throughout this specification do not necessarily all refer to the same embodiment. Furthermore, the specific features, structures, or characteristics in one or more embodiments can be combined in any suitable manner.

[0015] The first version of the HEVC standard was completed in October 2013, which provides about 50% bitrate savings compared to the previous-generation video coding standard H.264 / MPEG AVC, or equivalent perceptual quality. The HEVC standard provides significant coding improvements over its predecessor, but there is evidence that excellent coding efficiency can be achieved by using additional coding tools on top of HEVC. Based on this, both VCEG and MPEG started exploring new coding technologies for future video coding standardization. To initiate important research on advanced technologies that have the potential to significantly improve coding efficiency, a Joint Video Exploration Team (JVET) was formed by ITU-T VECG and ISO / IEC MPEG in October 2015. A reference software called the Joint Exploration Model (JEM) was maintained by JVET by integrating some additional coding tools on top of the HEVC Test Model (HM). In October 2017, a Call for Proposals (CfP) regarding video compression with capabilities beyond HEVC was jointly solicited by ITU-T and ISO / IEC. In April 2018, 23 CfP responses were received and evaluated in 10 JVET meetings, demonstrating an increase in compression efficiency of about 40% over HEVC. Based on such evaluation results, JVET newly launched a development project for the Versatile Video Coding (VVC), a next-generation video coding standard. In the same month, a reference software called the VVC Test Model (VTM) was established to demonstrate the reference implementation of the VVC standard. (Joint Video Exploration Team,JVET) was formed

[0016] ​​​​​​​​​​​​​​

[0017] The prediction methods used in video encoding typically perform spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to reduce or remove the redundancy inherent in video data and are generally associated with block-based video encoding. Similar to HEVC, VVC is also constructed based on a block-based hybrid video encoding framework. In block-based video encoding, the input video signal is processed block by block. For each block, spatial prediction and / or temporal prediction can be performed. In newer video encoding standards such as the current V

[0018] VC design, the block can be further divided based on a multi-type tree structure that includes not only quad-trees but also binary trees and / or ternary trees. This allows for better adaptation to changing local characteristics. Spatial prediction (also known as "intra prediction") uses pixels from samples of already encoded adjacent blocks (referred to as reference samples) within the same video picture / slice to predict the current block. Spatial prediction reduces the spatial redundancy inherent in the video signal. During the decoding process, the video bitstream is first entropy decoded in the entropy decoding unit. The encoding mode and prediction information are sent to either the spatial prediction unit (when intra-encoded) or the temporal prediction unit (when inter-encoded) to form the predicted block. The residual transform coefficients are used to reconstruct the residual block.

[0019]

[0020] ​​​​​​​​​Therefore, it is sent to the inverse quantization unit and the inverse transform unit. Then, the prediction block and the residual difference block are added. The reconstructed block may further pass through in-loop filtering before being stored in the reference picture store . Thereafter, the reconstructed video in the reference picture store is sent to drive the display device and is used to predict future video blocks.

[0021] In newer video coding standards such as the current VVC design, the luminance mapping with chroma scaling (LMCS) coding tool can be applied before in-loop filtering . LMCS aims to adjust the dynamic range of the input signal to improve coding efficiency . However, in the current design of LMCS, the mapping accuracy of the inter-prediction samples of the luminance component may exceed the dynamic range of the internal coding depth.

[0022] Conceptually, many video coding standards, including those described in the background art section above, are similar. For example, substantially all video coding standards use block-based processing and share similar video coding block diagrams to achieve video compression.

[0023] Figure 1 shows a block diagram of an exemplary block-based hybrid video encoder 100 that can be used with many video coding standards. In encoder 100, a video frame is divided into a plurality of video blocks for processing. For each given video block, a prediction is formed based on either an inter-prediction method or an intra-prediction method.

[0024] a prediction is formed based on either an inter-prediction method or an intra-prediction method. For each given video block, a prediction is formed based on either an inter-prediction method or an intra-prediction method. In inter prediction, one or more predictors are formed by motion estimation and motion compensation based on pixels from previously reconstructed frames. In intra prediction, a predictor is formed based on the reconstructed pixels within the current frame. By mode decision, the best predictor can be selected to predict the current block. The prediction residual representing the difference between the current video block and its predictor is sent to the conversion circuit 102. The conversion coefficients are then sent from the conversion circuit 102 to the quantization circuit 104 for entropy reduction. The quantized coefficients are then supplied to the entropy encoding circuit 106 to generate a compressed video bit stream. As shown in FIG. 1, video block

[0025] splitting information, motion vectors, reference picture indices, and any prediction-related information 110 from the inter prediction circuit and / or the intra prediction circuit 112 are also supplied via the entropy encoding circuit 106 and stored in the compressed video bit stream 11 4. In the encoder 100, decoder-related circuits are also required to reconstruct pixels for prediction purposes. First, the prediction residual is reconstructed by the inverse quantization circuit 116 and the inverse conversion circuit 118. This reconstructed prediction residual is combined with the block predictor 120 to generate the unfiltered reconstructed pixels of the current video block. Temporal prediction (also called "inter prediction" or "motion compensation prediction") uses the reconstructed pixels from previously encoded video pictures to predict the current video block. block.

[0026] In the encoder 100, decoder-related circuits are also required to reconstruct pixels for prediction purposes. First, the prediction residual is reconstructed by the inverse quantization circuit 116 and the inverse conversion circuit 118. This reconstructed prediction residual is combined with the block predictor 120 to generate the unfiltered reconstructed pixels of the current video block.

[0027] Temporal prediction (also called "inter prediction" or "motion compensation prediction") uses the reconstructed pixels from previously encoded video pictures to predict the current video block. Temporal prediction reduces the temporal redundancy inherent in a video signal. The number is usually one or more that indicate the amount and direction of movement between the current CU and its time reference. It is signaled by multiple motion vectors (MVs) and multiple reference pictures. If supported, one reference picture index is additionally transmitted, which is Used to identify which reference picture in the reference picture store the temporal prediction signal comes from. It is used.

[0028] After spatial and / or temporal prediction is performed, the intra / inter prediction in the encoder 100 The mode decision circuit 121 determines the best prediction mode, for example based on a rate-distortion optimization method. The block predictor 120 is then subtracted from the current video block. The resulting prediction residuals are then de-correlated using a transform circuit 102 and a quantizer circuit 104. The resulting quantized residual coefficients are inversely quantized by the inverse quantization circuit 116 and then inversely transformed. The inverse transformation is performed by path 118 to form a reconstructed residual, which is then added to the predicted block. The reconstructed CU is stored in the picture buffer 117. before being put into the reference picture store and used to encode future video blocks. Then, the reconstructed CU is applied with a deblocking filter and a sample adaptive offset (sample e adaptive offset,SAO), and / or adaptive in-loop filter Further loops such as adaptive in-loop filter (ALF) Intra-filtering 115 can be applied to the output video bitstream 114. To form the coding mode (inter or intra), prediction mode information, motion information , and all quantization residual coefficients are further compressed and packed and sent to the entropy encoding unit 106 to form a bitstream.

[0029] For example, deblocking filters are available in current versions of AVC, HEVC, and VVC. In HEVC, an additional in-loop filter called SAO (sample adaptive offset) is defined to further improve the encoding efficiency. In the current version of the VVC standard, yet another in-loop filter called ALF (adaptive loop filter) is being actively studied and is likely to be included in the final standard.

[0030] These in-loop filter operations are optional. Performing these operations helps improve the encoding efficiency and visual quality. They may also be turned off as a decision provided by the encoder 100 to reduce the computational complexity.

[0031] Intra prediction is usually based on the non-filtered reconstructed pixels, while inter prediction is based on the filtered reconstructed pixels. Note that these filter options are turned on by the encoder 100.

[0032] Figure 2 is a block diagram showing an exemplary video decoder 200 that can be used with many video encoding standards. This decoder 200 is similar to the reconstruction-related section present in the encoder 100 of FIG. 1. In the decoder 200 (FIG. 2), the input video bitstream 201 is used to derive the quantized coefficient levels and prediction-related information, and the ent It is first decoded through the tropy decoder 202. Then, the quantized coefficient levels are processed through inverse quantization 204 and inverse transformation 206 to obtain the reconstructed prediction residual. The block predictor mechanism implemented in the intra / inter mode selector 212 executes either intra prediction 208 or motion compensation 210 based on the decoded prediction information. It is configured to do so. The set of unfiltered reconstructed pixels is obtained by adding the reconstructed prediction residual from the inverse transformation 206 and the prediction output generated by the block predictor mechanism using the adder 21 4. The reconstructed block can further pass through the in-loop filter 209 before being stored in the picture buffer 213 that functions as a reference picture store.

[0033] The reconstructed video in the picture buffer 213 can then be sent to drive a display device and can be used to predict future video blocks. When the in-loop filter 209 is turned on, a filtering operation is performed on these reconstructed pixels to derive the final reconstructed video output 222. In a video coding standard such as HEVC, a block may be divided based on a quadtree. In a newer video coding standard such as the current VVC, more splitting methods are adopted, and one coding tree unit (CTU) can be divided into CUs, which can be adapted to various local characteristics based on quadtree, binary tree, or ternary tree. The separation of CUs, prediction units (PUs), and transform units (TUs) is present in most coding modes of the current VVC

[0034] ​ It does not exist in the coding tree unit (CTU). Each coding unit (CU) is always used as the basic unit for both prediction and transformation without further splitting. However, in some specific coding modes such as the intra-sub-partition coding mode, each CU can still contain multiple transform units (TUs). In the multi-type tree structure, one CTU is first split by a quadtree structure. Next, each quadtree leaf node can be further split by a binary tree structure and a ternary tree structure. However, in some specific coding modes such as the intra-sub-partition coding mode, each CU can still contain multiple transform units (TUs). In the multi-type tree structure, one CTU is first split by a quadtree structure. Next, each quadtree leaf node can be further split by a binary tree structure and a ternary tree structure. It does not exist in the coding tree unit (CTU). Each coding unit (CU) is always used as the basic unit for both prediction and transformation without further splitting. However, in some specific coding modes such as the intra-sub-partition coding mode, each CU can still contain multiple transform units (TUs). In the multi-type tree structure, one CTU is first split by a quadtree structure. Next, each quadtree leaf node can be further split by a binary tree structure and a ternary tree structure. It does not exist in the coding tree unit (CTU). Each coding unit (CU) is always used as the basic unit for both prediction and transformation without further splitting. However, in some specific coding modes such as the intra-sub-partition coding mode, each CU can still contain multiple transform units (TUs). In the multi-type tree structure, one CTU is first split by a quadtree structure. Next, each quadtree leaf node can be further split by a binary tree structure and a ternary tree structure. It does not exist in the coding tree unit (CTU). Each coding unit (CU) is always used as the basic unit for both prediction and transformation without further splitting. However, in some specific coding modes such as the intra-sub-partition coding mode, each CU can still contain multiple transform units (TUs). In the multi-type tree structure, one CTU is first split by a quadtree structure. Next, each quadtree leaf node can be further split by a binary tree structure and a ternary tree structure.

[0035] Figure 3 shows five splitting types currently adopted in the current Versatile Video Coding (VVC), namely, quadtree splitting 301, horizontal binary splitting 302, vertical binary splitting 303, horizontal ternary splitting 304, and vertical value splitting 305. In the situation where the multi-type tree structure is utilized, one CTU is first split by a quadtree structure. Next, each quadtree leaf node can be further split by a binary tree structure and a ternary tree structure. Figure 3 shows five splitting types currently adopted in the current Versatile Video Coding (VVC), namely, quadtree splitting 301, horizontal binary splitting 302, vertical binary splitting 303, horizontal ternary splitting 304, and vertical value splitting 305. In the situation where the multi-type tree structure is utilized, one CTU is first split by a quadtree structure. Next, each quadtree leaf node can be further split by a binary tree structure and a ternary tree structure.

[0036] One or more of the exemplary block splittings 301, 302, 303, 304, or 305 in Figure 3 can be used to perform spatial prediction and / or temporal prediction using the configuration shown in Figure 1. Spatial prediction (or "intra prediction") predicts the current video block using samples from the samples of the encoded adjacent blocks (referred to as reference samples) within the same video picture / slice. Spatial prediction reduces the spatial redundancy inherent in the video signal. One or more of the exemplary block splittings 301, 302, 303, 304, or 305 in Figure 3 can be used to perform spatial prediction and / or temporal prediction using the configuration shown in Figure 1. Spatial prediction (or "intra prediction") predicts the current video block using samples from the samples of the encoded adjacent blocks (referred to as reference samples) within the same video picture / slice. Spatial prediction reduces the spatial redundancy inherent in the video signal. One or more of the exemplary block splittings 301, 302, 303, 304, or 305 in Figure 3 can be used to perform spatial prediction and / or temporal prediction using the configuration shown in Figure 1. Spatial prediction (or "intra prediction") predicts the current video block using samples from the samples of the encoded adjacent blocks (referred to as reference samples) within the same video picture / slice. Spatial prediction reduces the spatial redundancy inherent in the video signal. One or more of the exemplary block splittings 301, 302, 303, 304, or 305 in Figure 3 can be used to perform spatial prediction and / or temporal prediction using the configuration shown in Figure 1. Spatial prediction (or "intra prediction") predicts the current video block using samples from the samples of the encoded adjacent blocks (referred to as reference samples) within the same video picture / slice. Spatial prediction reduces the spatial redundancy inherent in the video signal. One or more of the exemplary block splittings 301, 302, 303, 304, or 305 in Figure 3 can be used to perform spatial prediction and / or temporal prediction using the configuration shown in Figure 1. Spatial prediction (or "intra prediction") predicts the current video block using samples from the samples of the encoded adjacent blocks (referred to as reference samples) within the same video picture / slice. Spatial prediction reduces the spatial redundancy inherent in the video signal. One or more of the exemplary block splittings 301, 302, 303, 304, or 305 in Figure 3 can be used to perform spatial prediction and / or temporal prediction using the configuration shown in Figure 1. Spatial prediction (or "intra prediction") predicts the current video block using samples from the samples of the encoded adjacent blocks (referred to as reference samples) within the same video picture / slice. Spatial prediction reduces the spatial redundancy inherent in the video signal.

[0037] In new video coding standards such as the current VVC, a new coding tool, luminance mapping with chroma subsampling (LMCS), is added. Loop filters (e.g., In new video coding standards such as the current VVC, a new coding tool, luminance mapping with chroma subsampling (LMCS), is added. Loop filters (e.g., , deblocking filter, SAO, and ALF) LMCS will be added as a tool for simplification.

[0038] In general, LMCS has two main modules. First, adaptive piecewise linear model and secondly, a luminance-dependent chroma residual scale It's a ring.

[0039] FIG. 4 shows the modified decoding process using LMCS. In FIG. 4, a particular block is Decoding module executed in the pinged domain, entropy decoding 401, inverse Quantization 402, inverse transform 403, luma intra prediction 404, and luma sample reconstruction 40 5 (i.e., the luminance prediction sample Y' pred and the luminance residual sample Y' res Add and and the reconstructed luminance sample Y' recon (generating certain other blocks) represents a decryption module that runs in the original (i.e., unmapped) domain. 4, the motion compensation prediction 409, the chroma intra prediction 412, and the chroma sample reconstruction 413 (i.e. That is, saturation prediction sample C pred and chroma residual sample C res and are added together to reconstruct Saturation sample C recon ), and in-loop filtering 407 (generating A further group of blocks includes , a forward mapping 410 and a reverse (or inverse) mapping 406 of the luma samples; and new operational models introduced by LMCS, including chroma residual scaling 411. Also, as shown in FIG. 4, a decoded picture buffer (DPB) 40 All reference pictures stored in 8 (for luminance) and 415 (for chrominance) are in the original domain. They are mainly in the main.

[0040] The in-loop mapping of LMCS aims to adjust the dynamic range of the input signal to improve the coding efficiency. The in-loop mapping of luminance samples in existing LMCS designs is constructed based on two mapping functions, one forward mapping function FwdMap, and one corresponding inverse mapping function InvMap. The forward mapping function uses a piecewise linear model with 16 equally sized segments to signal from the encoder to the decoder. The inverse mapping function can be directly derived from the forward mapping function and thus does not need to be signaled. The parameters of the luminance mapping model are signaled at the slice level. To indicate whether the luminance mapping model should be signaled for the current slice, an existence flag is signaled first. If the luminance mapping model exists for the current slice, the corresponding piecewise linear model parameters are further signaled. Based on the piecewise linear model, the dynamic range of the input signal is divided into 16 segments of equal size in the original domain, and each segment is mapped to a corresponding segment. For a given segment in the original domain, its corresponding segment in the mapped domain can have the same or different sizes. The size of each segment in the mapped domain is the number of codewords of that segment (i.e.,

[0041] ​​​​​​​​​​​​​, is indicated by the mapped sample values). For each segment in the original domain the number of codewords in the corresponding segment in the mapped domain Based on this, linear mapping parameters can be derived. For example, when the input is 10-bit depth, each of the 16 segments in the original domain has 64 pixel values and each segment in the mapped domain also has 64 codewords assigned to it If so, it represents a simple one-to-one mapping (i.e., a mapping that does not change each sample value). The number of signaled codewords for each segment in the mapped domain is used to calculate the scaling factor and adjust the mapping function accordingly for that segment. Further, at the slice level, another LMCS control flag is signaled to enable / disable the LMCS of the slice.

[0042] For each segment, the corresponding piecewise linear model is defined as described in the box immediately following this paragraph. For the i-th segment (i = 0...15), the corresponding piecewise linear model is defined by two input pivot points InputPivot[i] and InputPivot[i + 1], and also by two output (mapped) pivot points MappedPivot[i] and Ma ppedPivot[i + 1]. Further, assuming a 10-bit input video the values of InputPivot[i] and MappedPivot[i] ( i = 0...15) are calculated as follows. 1. Set the variable OrgCW = 64. ​​When \(i = 0:16\), \(InputPivot[i]=i*OrgCW\) When \(i = 0:16\), \(MappedPivot[i]\) is calculated as follows. \(MappedPivot[0]=0;\) For the case of \((i = 0;i<16;i++)\) \(MappedPivot[i + 1]=MappedPivot[i]+Signale\) \(dCW[i]\) Here, \(SignaledCW[i]\) is the signaled number of the codeword for the \(i\)-th segment.

[0043] As shown in Figure 4, it is necessary to operate in two different domains during LMCS processing. For each CU encoded through the inter prediction mode ("inter CU"), its motion compensation prediction is executed in the original domain. However, the reconstruction of the luminance component (i.e., the addition of the luminance prediction samples and the luminance residual samples) is executed in the mapped domain. Therefore, the motion compensated luminance prediction \(Y\) is mapped to the value \(Y'\) pred in the mapped domain before \(Y'\) pred is used for pixel reconstruction 405. Before that, the forward mapping function 410, i.e., \(Y'\) pred \(=FwdMap(Y\) pred ) is used to map the value \(Y\) in the original domain to the value \(Y'\) in the mapped domain. On the other hand, considering that for each CU encoded through the intra prediction mode ("intra CU"), the intra prediction 404 is executed in the mapped domain before \(Y'\) pred is used for pixel reconstruction 405 (shown in Figure 4), the mapping of the prediction samples is not necessary. Finally, the reconstructed luminance sample \(Y'\) pred reco n After generating n , the inverse mapping function 406 is applied to reconstruct the luminance sample Y ’ recon back to the value Y of the original domain recon before proceeding to the luminance DPB 408, i.e., Y = InvMap(Y’ recon )). Different from the forward mapping 410 of the prediction samples that need to be applied only to the inter CU, the inverse mapping 406 of the reconstructed samples needs to be applied to both the inter CU and the intra CU recon ). To summarize, on the decoder side, the in-loop luminance mapping of the current LMCS is performed such that the luminance prediction sample Y is first converted to the domain where it is mapped as needed. Y’pred = FwdMap(Ypred). Next, the mapped prediction sample is added to the decoded luminance residual to form the reconstructed luminance sample in the mapped domain. Y’recon = Y’pred + Y’rez. Finally, the inverse mapping is applied to convert the reconstructed luminance sample Y’recon back to the original domain . Yrecon = InvMap(Y’recon). On the encoder side, since the luminance residual is encoded in the mapped domain, it is generated as the difference between the original sample of the mapped luminance and the predicted sample of the mapped luminance. Y’rez = FwdMap(Yorg) - FwdMap(Ypred).

[0044] Luminance-dependent chrominance residual scaling, the second step of LMCS, is the quantization accuracy between the luminance signal and its corresponding chrominance signal when the in-loop mapping is applied to the luminance signal pred . Y’pred = FwdMap(Ypred). Next, the mapped prediction sample is added to the decoded luminance residual to form the reconstructed luminance sample in the mapped domain . Y’recon = Y’pred + Y’rez. Finally, the inverse mapping is applied to convert the reconstructed luminance sample Y’recon back to the original domain . . Yrecon = InvMap(Y’recon). On the encoder side, since the luminance residual is encoded in the mapped domain, it is generated as the difference between the original sample of the mapped luminance and the predicted sample of the mapped luminance . Y’rez = FwdMap(Yorg) - FwdMap(Ypred). . . Y’res = FwdMap(Yorg) - FwdMap(Ypred).

[0045] Luminance-dependent chrominance residual scaling, the second step of LMCS, is the quantization accuracy between the luminance signal and its corresponding chrominance signal when the in-loop mapping is applied to the luminance signal​ is designed to compensate for the interaction. Whether the chroma residual scaling is effective or not is also notified in the slice header. If the luminance mapping is enabled and the parallel tree splitting of the luminance component and the chroma component is disabled for the current slice an additional flag is signaled to indicate whether the luminance-dependent chroma residual scaling is applied. If the luminance mapping is not used or the parallel tree splitting is enabled for the current slice, the luminance-dependent chroma residual scaling is always disabled. Furthermore for a CU containing four or fewer chroma samples, the chroma residual scaling is always disabled .

[0046] For both intra CUs and inter CUs, the scaling parameters used to scale the chroma residuals depend on the average of the corresponding mapped luminance prediction samples . The scaling parameters are derived as described in the box immediately following this paragraph . avg’ Y is expressed as the average of the luminance prediction samples in the mapped domain . The scaling parameter C ScaleInv is calculated according to the following steps 1. Find the segment index Y Y in the piecewise linear model to which avg’ belongs in the mapped domain 1dx . Here, Y 1dx has integer values in the range from 0 to 15 . 2. C ScaleInv = cScaleInv[Y 1dx , where cScaleIn v[i], i = 0...15 are 16 pre-calculated look-up tables (LUTs​ ) is true. Intra prediction is performed in the mapped domain of LMCS, so for intra, combined inter-intra prediction (CIIP), or when the CU is encoded as an intra-block-copy (IBC) mode, avg’ Y is calculated as the average of the luminance prediction samples. Otherwise, avg’ Y is calculated as the average of the forward-mapped inter-prediction luminance samples.

[0047] Figure 4 also shows the calculation of the average of the luminance prediction samples for luminance-dependent chroma residual scaling. For an inter CU, the pred forward-mapped luminance prediction Y’ is supplied to the chroma residual scaling 411 together with the scaled chroma resScale residual C to derive the chroma residual C res which is supplied to the chroma reconstruction 413 together with the chroma recon prediction C to derive the reconstructed chroma value C pred . For an intra CU, the intra prediction 404 generates a Y’ which is already in the mapped domain and is supplied to the chroma residual pred scaling 411 in the same way as for an inter CU. Unlike the luminance mapping which is performed on a sample-by-sample basis, C

[0048] is fixed for the entire chroma CU. Given C ScaleInv , chroma residual scaling is applied as described in the box immediately after this paragraph. ScaleInv When C is given, chroma residual scaling is applied as described in the box immediately Encoder side:

Number

[0049] In the current new video coding standards such as VVC, new coding tools have been introduced. Some examples of the new coding tools are Bidirectional Optical Flow (BDOF), Decoder-side Motion Vector Refinement (DMVR), Combined Inter and Intra Prediction (CIIP), Affine Mode, and Prediction Refinement with Optical Flow (PROF) for Affine Mode.

[0050] In the current VVC, Bidirectional Optical Flow (BDOF) is applied to correct the prediction samples of the bidirectional prediction coding block.

[0051] Figure 5 is an explanatory diagram of BDOF processing. BDOF is a fine adjustment of the motion regarding the samples executed on the block-based motion compensation prediction when bidirectional prediction is used. The motion fine adjustment [Number] of each 4×4 sub-block 501

Number

[0052] Specifically, as described in the box immediately following this paragraph, the motion fine-tuning

Number

Number

Number

Number

Number

[0053] The value in the box immediately above

Number

Number

Number

Number

Number

Number

Number

[0054] the value in the box immediately above

Number

Number

Number

Number

Number

Number

[0055] The above-mentioned movement fine adjustment [Number] for deriving the value of As described in the box, based on the motion fine-tuning derived according to Equation (1), the final dual prediction samples of the CU are calculated by interpolating the L0 / L1 prediction samples along the motion trajectory as shown in the box immediately following this paragraph based on the optical flow model. [Number] Here, [Number] and [Number] are the right shift value and offset value applied to combine the L0 and L1 prediction signals for dual prediction, respectively [Number] and [Number] are equal to.

[0056] Based on the above-mentioned bit-depth control method, it is guaranteed that the maximum bit depth of the intermediate parameters of the entire BDOF process does not exceed 32 bits, and the maximum input to multiplication is within 15 bits. That is, a 15-bit multiplier is sufficient for BDOF implementation. 。

[0057] DMVR can be further corrected by using bilateral matching prediction. The dual prediction used for merge blocks that can have two initially signaled MVs. It is a technology.

[0058] Specifically, in DMVR, bilateral matching is used to find the best match between two blocks along the motion trajectory of the current CU within two different reference pictures. Thereby, it is used to derive the motion information of the current CU. The cost function used for the matching process is the row subsampled SAD (Sum of Absolute Differences). After the matching process is performed, the corrected MV is used for motion compensation in the prediction stage. The motion vector prediction of the subsequent picture's temporal motion vector and the uncorrected MV are used for motion vector prediction between the motion vector of the current CU and the motion vectors of its spatial neighbors. Under the assumption of continuous motion trajectories, the motion vectors MV0 and MV1 pointing to the two reference blocks are proportional to the temporal distances between the current picture and the two reference pictures, i.e., TD0 and TD1. As a special case, when the current picture is temporally between the two reference pictures and the temporal distances from the current picture to the two reference pictures are the same, the bilateral matching becomes a mirror-based bidirectional MV.

[0059]

[0060] ​​​​​​​​In the current VVC, an inter-prediction method and an intra-prediction method are used in a hybrid video coding method, and each PU can only select inter-prediction or intra-prediction to utilize the correlation in either the temporal domain or the spatial domain, but not both. However, as pointed out in previous literature, the residual signals generated by the inter-prediction block and the intra-prediction block may exhibit very different characteristics from each other. Therefore, if the two types of predictions can be efficiently combined, another more accurate prediction can be expected to reduce the energy of the prediction residual and improve the coding efficiency. Furthermore, in natural video content, the motion of moving objects may become complex. For example, there may be a region that contains both old content (e.g., objects included in previously encoded pictures) and new content (e.g., objects excluded in previously encoded pictures). In such a scenario, neither inter-prediction nor intra-prediction can provide an accurate prediction of the current block. To further improve the prediction efficiency, a combined inter-intra prediction (CIIP) that combines the intra-prediction and inter-prediction of one CU encoded in the merge mode is adopted in the VVC standard. Specifically, for each merge CU, one additional flag is signaled to indicate whether CIIP is enabled for the current CU. When the flag is equal to 1, CIIP applies only the planar mode to generate the intra-prediction samples of the luminance and chrominance components. Furthermore, equal weights (i.e., 0.5 In the current VVC, an inter-prediction method and an intra-prediction method are used in a hybrid video coding method, and each PU can only select inter-prediction or intra-prediction to utilize the correlation in either the temporal domain or the spatial domain, but not both. However, as pointed out in previous literature, the residual signals generated by the inter-prediction block and the intra-prediction block may exhibit very different characteristics from each other. Therefore, if the two types of predictions can be efficiently combined, another more accurate prediction can be expected to reduce the energy of the prediction residual and improve the coding efficiency. Furthermore, in natural video content, the motion of moving objects may become complex. For example, there may be a region that contains both old content (e.g., objects included in previously encoded pictures) and new content (e.g., objects excluded in previously encoded pictures). In such a scenario, neither inter-prediction nor intra-prediction can provide an accurate prediction of the current block. To further improve the prediction efficiency, a combined inter-intra prediction (CIIP) that combines the intra-prediction and inter-prediction of one CU encoded in the merge mode is adopted in the VVC standard. Specifically, for each merge CU, one additional flag is signaled to indicate whether CIIP is enabled for the current CU. When the flag is equal to 1, CIIP applies only the planar mode to generate the intra-prediction samples of the luminance and chrominance components. Furthermore, equal weights (i.e., 0.5 In the current VVC, an inter-prediction method and an intra-prediction method are used in a hybrid video coding method, and each PU can only select inter-prediction or intra-prediction to utilize the correlation in either the temporal domain or the spatial domain, but not both. However, as pointed out in previous literature, the residual signals generated by the inter-prediction block and the intra-prediction block may exhibit very different characteristics from each other. Therefore, if the two types of predictions can be efficiently combined, another more accurate prediction can be expected to reduce the energy of the prediction residual and improve the coding efficiency. Furthermore, in natural video content, the motion of moving objects may become complex. For example, there may be a region that contains both old content (e.g., objects included in previously encoded pictures) and new content (e.g., objects excluded in previously encoded pictures). In such a scenario, neither inter-prediction nor intra-prediction can provide an accurate prediction of the current block. To further improve the prediction efficiency, a combined inter-intra prediction (CIIP) that combines the intra-prediction and inter-prediction of one CU encoded in the merge mode is adopted in the VVC standard. Specifically, for each merge CU, one additional flag is signaled to indicate whether CIIP is enabled for the current CU. When the flag is equal to 1, CIIP applies only the planar mode to generate the intra-prediction samples of the luminance and chrominance components. Furthermore, equal weights (i.e., 0.5 In the current VVC, an inter-prediction method and an intra-prediction method are used in a hybrid video coding method, and each PU can only select inter-prediction or intra-prediction to utilize the correlation in either the temporal domain or the spatial domain, but not both. However, as pointed out in previous literature, the residual signals generated by the inter-prediction block and the intra-prediction block may exhibit very different characteristics from each other. Therefore, if the two types of predictions can be efficiently combined, another more accurate prediction can be expected to reduce the energy of the prediction residual and improve the coding efficiency. Furthermore, in natural video content, the motion of moving objects may become complex. For example, there may be a region that contains both old content (e.g., objects included in previously encoded pictures) and new content (e.g., objects excluded in previously encoded pictures). In such a scenario, neither inter-prediction nor intra-prediction can provide an accurate prediction of the current block. To further improve the prediction efficiency, a combined inter-intra prediction (CIIP) that combines the intra-prediction and inter-prediction of one CU encoded in the merge mode is adopted in the VVC standard. Specifically, for each merge CU, one additional flag is signaled to indicate whether CIIP is enabled for the current CU. When the flag is equal to 1, CIIP applies only the planar mode to generate the intra-prediction samples of the luminance and chrominance components. Furthermore, equal weights (i.e., 0.5 In the current VVC, an inter-prediction method and an intra-prediction method are used in a hybrid video coding method, and each PU can only select inter-prediction or intra-prediction to utilize the correlation in either the temporal domain or the spatial domain, but not both. However, as pointed out in previous literature, the residual signals generated by the inter-prediction block and the intra-prediction block may exhibit very different characteristics from each other. Therefore, if the two types of predictions can be efficiently combined, another more accurate prediction can be expected to reduce the energy of the prediction residual and improve the coding efficiency. Furthermore, in natural video content, the motion of moving objects may become complex. For example, there may be a region that contains both old content (e.g., objects included in previously encoded pictures) and new content (e.g., objects excluded in previously encoded pictures). In such a scenario, neither inter-prediction nor intra-prediction can provide an accurate prediction of the current block. To further improve the prediction efficiency, a combined inter-intra prediction (CIIP) that combines the intra-prediction and inter-prediction of one CU encoded in the merge mode is adopted in the VVC standard. Specifically, for each merge CU, one additional flag is signaled to indicate whether CIIP is enabled for the current CU. When the flag is equal to 1, CIIP applies only the planar mode to generate the intra-prediction samples of the luminance and chrominance components. Furthermore, equal weights (i.e., 0.5 In the current VVC, an inter-prediction method and an intra-prediction method are used in a hybrid video coding method, and each PU can only select inter-prediction or intra-prediction to utilize the correlation in either the temporal domain or the spatial domain, but not both. However, as pointed out in previous literature, the residual signals generated by the inter-prediction block and the intra-prediction block may exhibit very different characteristics from each other. Therefore, if the two types of predictions can be efficiently combined, another more accurate prediction can be expected to reduce the energy of the prediction residual and improve the coding efficiency. Furthermore, in natural video content, the motion of moving objects may become complex. For example, there may be a region that contains both old content (e.g., objects included in previously encoded pictures) and new content (e.g., objects excluded in previously encoded pictures). In such a scenario, neither inter-prediction nor intra-prediction can provide an accurate prediction of the current block. To further improve the prediction efficiency, a combined inter-intra prediction (CIIP) that combines the intra-prediction and inter-prediction of one CU encoded in the merge mode is adopted in the VVC standard. Specifically, for each merge CU, one additional flag is signaled to indicate whether CIIP is enabled for the current CU. When the flag is equal to 1, CIIP applies only the planar mode to generate the intra-prediction samples of the luminance and chrominance components. Furthermore, equal weights (i.e., 0.5 In the current VVC, an inter-prediction method and an intra-prediction method are used in a hybrid video coding method, and each PU can only select inter-prediction or intra-prediction to utilize the correlation in either the temporal domain or the spatial domain, but not both. However, as pointed out in previous literature, the residual signals generated by the inter-prediction block and the intra-prediction block may exhibit very different characteristics from each other. Therefore, if the two types of predictions can be efficiently combined, another more accurate prediction can be expected to reduce the energy of the prediction residual and improve the coding efficiency. Furthermore, in natural video content, the motion of moving objects may become complex. For example, there may be a region that contains both old content (e.g., objects included in previously encoded pictures) and new content (e.g., objects excluded in previously encoded pictures). In such a scenario, neither inter-prediction nor intra-prediction can provide an accurate prediction of the current block.

[0061] To further improve the prediction efficiency, a combined inter-intra prediction (CIIP) that combines the intra-prediction and inter-prediction of one CU encoded in the merge mode is adopted in the VVC standard. Specifically, for each merge CU, one additional flag is signaled to indicate whether CIIP is enabled for the current CU. When the flag is equal to 1, CIIP applies only the planar mode to generate the intra-prediction samples of the luminance and chrominance components. Furthermore, equal weights (i.e., 0.5 To further improve the prediction efficiency, a combined inter-intra prediction (CIIP) that combines the intra-prediction and inter-prediction of one CU encoded in the merge mode is adopted in the VVC standard. Specifically, for each merge CU, one additional flag is signaled to indicate whether CIIP is enabled for the current CU. When the flag is equal to 1, CIIP applies only the planar mode to generate the intra-prediction samples of the luminance and chrominance components. Furthermore, equal weights (i.e., 0.5 To further improve the prediction efficiency, a combined inter-intra prediction (CIIP) that combines the intra-prediction and inter-prediction of one CU encoded in the merge mode is adopted in the VVC standard. Specifically, for each merge CU, one additional flag is signaled to indicate whether CIIP is enabled for the current CU. When the flag is equal to 1, CIIP applies only the planar mode to generate the intra-prediction samples of the luminance and chrominance components. Furthermore, equal weights (i.e., 0.5 To further improve the prediction efficiency, a combined inter-intra prediction (CIIP) that combines the intra-prediction and inter-prediction of one CU encoded in the merge mode is adopted in the VVC standard. Specifically, for each merge CU, one additional flag is signaled to indicate whether CIIP is enabled for the current CU. When the flag is equal to 1, CIIP applies only the planar mode to generate the intra-prediction samples of the luminance and chrominance components. Furthermore, equal weights (i.e., 0.5 To further improve the prediction efficiency, a combined inter-intra prediction (CIIP) that combines the intra-prediction and inter-prediction of one CU encoded in the merge mode is adopted in the VVC standard. Specifically, for each merge CU, one additional flag is signaled to indicate whether CIIP is enabled for the current CU. When the flag is equal to 1, CIIP applies only the planar mode to generate the intra-prediction samples of the luminance and chrominance components. Furthermore, equal weights (i.e., 0.5 To further improve the prediction efficiency, a combined inter-intra prediction (CIIP) that combines the intra-prediction and inter-prediction of one CU encoded in the merge mode is adopted in the VVC standard. Specifically, for each merge CU, one additional flag is signaled to indicate whether CIIP is enabled for the current CU. When the flag is equal to 1, CIIP applies only the planar mode to generate the intra-prediction samples of the luminance and chrominance components. Furthermore, equal weights (i.e., 0.5 ) is applied as the final prediction sample of CIIP CU by averaging the inter-prediction sample and the intra- prediction sample.

[0062] VVC also supports the affine mode for motion compensation prediction. In HEVC, only the translational motion model is applied to motion compensation prediction. In the real world, there are many types of motions such as zoom-in / zoom-out, rotation, perspective motion, and other irregular motions. In VVC, by signaling one flag for each inter-coded block, affine motion compensation prediction is applied to indicate whether the translational motion or the affine motion model is applied to inter-prediction. In the current VVC design, two affine modes including the 4-parameter affine mode and the 6-parameter affine mode are supported for one affine-coded block.

[0063] The 4-parameter affine model has the following parameters. Two parameters for each translational motion in the horizontal and vertical directions, one parameter for the zoom motion, and one parameter for the rotational motion in both directions. The horizontal zoom parameter is equal to the vertical zoom parameter. The horizontal rotation parameter is equal to the vertical rotation parameter. In order to achieve better adaptation of the motion vector and the affine parameters in VVC, those affine parameters are transformed into two MVs (also called control point motion vectors (CPMVs)) located at the upper left corner and the upper right corner of the current block.

[0064] Motion of control pointsvector Based on this, the motion vector (v x , v y ) of one affine-encoded block is calculated as described in the box immediately following this paragraph.

Number

[0065] The six-parameter affine mode has the following parameters. Two parameters for each translational motion in the horizontal and vertical directions, one parameter for zoom motion, one parameter for rotational motion in the horizontal direction, one parameter for zoom motion, one parameter for rotational motion in the vertical direction, and one parameter for zoom motion. The six-parameter affine motion model is encoded with three MVs in three CPMVs.

[0066] The three control points of one six-parameter affine block are located at the upper left, upper right, and lower left corners of the block. The motion of the upper left control point is related to translational motion, the motion of the upper right control point is related to horizontal rotation and zoom motion, and the motion of the lower left control point is related to vertical rotation and zoom motion. Compared with the four-parameter affine motion mode affine motion model, the horizontal rotation and zoom motions of the six-parameter may not be the same as their vertical counterparts.

[0067] Assume that (V0, V1, V2) are the MVs of the upper left, upper right, and lower left corners of the current block. Then, the motion vectors of each sub-block (v , v x , v y ) are derived using the three MVs at the control points as described in the box immediately following this paragraph.

Number

[0068] To improve the affine motion compensation accuracy, predictive fine-tuning (P ROF) using optical flow is currently being studied in the current VVC, which corrects sub-block-based affine motion compensation based on an optical flow model. Specifically, after performing sub-block-based affine motion compensation, the luminance prediction samples of one affine block are corrected by one sample fine-tuning value derived based on the optical flow equation. Specifically, the operation of PROF can be summarized into the following four steps. Based on the optical flow equation, the luminance prediction samples of one affine block are corrected by one sample fine-tuning value derived based on the optical flow equation. Specifically, the operation of PROF can be summarized into the following four steps. After performing sub-block-based affine motion compensation, the luminance prediction samples of one affine block are corrected by one sample fine-tuning value derived based on the optical flow equation. Based on the optical flow equation, the luminance prediction samples of one affine block are corrected by one sample fine-tuning value derived based on the optical flow equation. In detail, the operation of PROF can be summarized into the following four steps.

[0069] In step 1, sub-block-based affine motion compensation is performed, and the sub-block prediction is generated using the derived sub-block MV for the four-parameter affine model in the above equation (6) and the six-parameter affine model in the above equation (7). For the four-parameter affine model in the above equation (6) and the six-parameter affine model in the above equation (7), the derived sub-block MV is used to generate the sub-block prediction. For the four-parameter affine model in the above equation (6) and the six-parameter affine model in the above equation (7), the derived sub-block MV is used to generate the sub-block prediction. For the four-parameter affine model in the above equation (6) and the six-parameter affine model in the above equation (7), the derived sub-block MV is used to generate the sub-block prediction.

Number

[0070] In step 2, the spatial gradients of each prediction sample

Number

Number

Number

[0071] Furthermore, in step 2, in order to calculate the gradient, one additional row / column of the prediction samples needs to be generated on each side of one sub-block. To reduce the memory bandwidth and complexity, the samples on the extended boundary are copied from the nearest integer pixel position within the reference picture to avoid additional interpolation processing. In step 3, the luminance prediction fine-tuning value is calculated as described in the box immediately following this paragraph.

[0072] Here, is calculated for the sample position

Number

Number

Number

Number

Number

[0073] Also, in the current PROF design, after adding the prediction fine-tuning to the original prediction samples, as described in the box immediately following this paragraph, a single clipping operation is performed as the fourth step to clip the values of the corrected prediction samples within 15 bits. The values of the corrected prediction samples are clipped within 15 bits as described in the box immediately following this paragraph. As a fourth step, a single clipping operation is performed to clip the values of the corrected prediction samples within 15 bits. is performed.

Number

Number

Number

Number

[0074] Since the affine model parameters and the pixel positions with respect to the sub - block centers do not change for each sub - block,

Number

Number

Number

Number

Number

Number

[0075] Based on the above - mentioned affine sub - block MV derivation formulas (6) and (7), the MV difference​

Number

Number

Number

Number

Number

Number

Number

Number

Number

Number

[0076] According to the current LMCS design, the chroma residual samples are their corresponding luminance prediction samples Newer coding tools are inter-CU-effective. When set, chroma residual samples are scaled via the LMCS in this inter CU. The luma prediction samples used to estimate the luminance are the cross-applications of these newer coding tools. The last one to be acquired is the

[0077] Figure 6 shows the LM with DMVR, BDOF, and CIIP all enabled. A flowchart showing the workflow of chroma residual scaling in CS. The outputs from the 0 prediction 601 and the luma L1 prediction 602 are fed to the DMVR 603 and the BD The inter-predicted value 621 is sequentially supplied to the OF 604, and the obtained inter-predicted value 621 is a intra-predicted value. The average luminance prediction 606 is fed together with the luminance intra prediction 622 from 605 to the average 606. 23, which is fed into chroma residual scaling 607 along with chroma residual 608; As a result, chroma residual scaling 607, chroma prediction 610, and chroma reconstruction 609 work together. The resulting output can be generated using the

[0078] The current LMCS design presents three challenges to the video decoding process. First, Mapping between domains (different domain mappings) requires additional computational complexity and on-chip Second, the derivation of the luma and chroma scaling coefficients requires a large amount of memory for different luma and chroma images. The fact that we use predicted values introduces additional complications. Third, the LMCS and newer The interaction between the encoding tool and the LMCS adds delays to the decoding process, i.e. Introduce the problem.

[0079] First, the current LMCS design does not allow for the reconstructed samples of the original domain and the mapping Both of the domains obtained are used in various decoding modules. As a result, these samples often need to be converted from one domain to another between different decoding modules, which can lead to both higher computational complexity and more on-chip memory.

[0080] Specifically, in the case of the intra mode, CIIP mode, and IBC mode, the mapped domain reference samples from the adjacent reconstructed domain of one existing CU are used to generate prediction samples. However, in the case of the inter mode, motion compensation prediction is performed using the original domain reconstruction samples of the temporal reference picture as references. The reconstructed samples stored in the DPB are also in the original domain. As shown in FIG. 4, such a mixed representation of the reconstructed samples under different prediction modes incurs additional forward and inverse luminance mapping operations.

[0081] For example, in the case of an inter CU, the luminance reconstruction operation (i.e., adding the prediction sample and the residual sample together) is performed in the mapped domain, so the inter prediction luminance samples generated in the original domain need to be converted to the mapped domain before they are used for luminance sample reconstruction. In another example, for both the intra CU and the inter CU, inverse (or reverse direction) mapping is always applied, and after the reconstructed luminance samples are converted from the mapped domain to the original domain, they are stored in the DPB. Such a design not only increases the computational complexity due to the additional forward / reverse mapping operations, but also requires maintaining multiple versions of the reconstructed samples.​ requires more on-chip memory.

[0082] Based on the above description, in some LMCS designs, luminance mapping and luminance-dependent chroma residual scaling are performed to encode the luminance component and the chroma component, respectively. In an actual hardware implementation, the forward and inverse (or reverse) mapping functions FwdMap and InvMap can be implemented using a look-up table (LUT) or calculated on-the-fly. If a LUT-based solution is used, the possible output elements from the functions FwdMap, InvMap, and cScaleInv can be pre-calculated and stored in advance as a LUT, and then used for the luminance mapping and chroma residual scaling operations of all CUs within the current slice. Assuming the input video is 10-bit, each LUT of FwdMap and InvMap has 2 = 1024 elements, and each element of the LUT has 10 bits. Therefore, the total memory capacity of the LUTs for forward and inverse luminance mapping is equal to 2 * 102 10 4 * 10 = 20480 bits = 2560 bytes. On the other hand, to derive the chroma scaling parameter C a 16-entry LUT table cScaleInv needs to be maintained in the encoder and decoder, and each chroma scaling parameter is stored in 32 bits. Correspondingly, the memory size used to store the LUT cScaleInv is equal to 16 * 32 = 512 bits = 64 bytes. The difference between 2560 and 64 is due to the forward and inverse (reverse) mapping operations. ScaleInv derived from the encoder and decoder requires a 16-entry LUT table cScaleInv to be maintained, and each chroma scaling parameter is stored in 32 bits. Correspondingly, the memory size used to store the LUT cScaleInv is equal to 16 * 32 = 512 bits = 64 bytes. The difference between 2560 and 64 is due to the forward and inverse (reverse) mapping operations. is equal to 16 * 32 = 512 bits = 64 bytes. The difference between 2560 and 64 is due to the forward and inverse (reverse) mapping operations. equal to 2560 and 64 is due to the forward and inverse (reverse) mapping operations. Shows the scale of the additional on-chip memory required.

[0083] Furthermore, in new video coding standards such as the current VVC, both intra prediction and deblocking filters use the reconstructed samples of the adjacent blocks described above. Therefore, one extra row of reconstructed samples at the width of the current picture / slice needs to be maintained in a buffer, which is also known as the "line buffer" in video coding. The reconstructed samples in the line buffer are at least used as a reference for the intra prediction and deblocking operations of the CUs located in the first row within one CTU. According to the existing LMCS design, the intra prediction and deblocking filters use reconstructed samples from different domains. Therefore, additional on-chip memory is required to store both the original domain reconstructed samples and the mapped domain reconstructed samples, which may approximately double the size of the line buffer.

[0084] In addition to increasing the line buffer size, another implementation option to avoid doubling the line buffer size is to perform the domain mapping operation on the fly. However, this comes at the cost of an unacceptable increase in computational complexity.

[0085] Therefore, the current design of LMCS requires additional computational complexity and on-chip memory for the necessary mapping between different domains.

[0086] Second, in the proposed adaptive luminance residual scaling, both the luminance component and the chrominance component ​​It has a scaling operation on these prediction residuals. Luminance in the current design of LMCS and both the chroma scaling coefficient derivation methods derive the corresponding scaling coefficients using luminance prediction sample values, but there are differences between their corresponding operations.

[0087] In the case of luminance residual scaling, the scaling coefficient is derived for each sample by enabling each luminance residual sample to have its own scaling coefficient. However, in the case of chroma residual scaling, the scaling coefficient is fixed for the entire CU i.e., all chroma residual samples within the CU share the same scaling coefficient calculated based on the average of the mapped luminance prediction samples.

[0088] Also, two different LUTs are used to calculate the scaling coefficients for luminance residuals and chroma residuals. Specifically, the input to the luminance LUT is the mapping model segment index of the original luminance prediction sample value, and the input to the chroma LUT is the mapping model segment index of the average value of the mapped luminance prediction samples. In some examples, it is possible to scale both luminance residuals and chroma residuals using one LUT without the need to map the luminance prediction samples to the mapped domain.

[0089] Such differences bring extra complexity to the encoding process, and a harmonized approach to luminance and chroma scaling coefficient derivation is desirable. Therefore, several methods can be proposed to harmonize the scaling methods for luminance residuals and chroma residuals in order to achieve one unified design. ​

[0090] Thirdly, as described above regarding "luminance-dependent chroma residual scaling", for the current LMCS design, chroma residual samples are scaled based on their corresponding luminance prediction samples. This means that the chroma residual samples of one LMCS CU cannot be reconstructed until all the luminance prediction samples of the CU are fully generated. Also, as described above, in order to improve the efficiency of inter prediction, DMVR, BDOF, and CIIP can be applied. As shown in FIG. 6, for the chroma residual scaling of the current design of LMCS, new coding tools such as all three modules of DMVR, BDOF, and CIIP can be sequentially invoked to generate the luminance prediction samples used to determine the scaling factors of the chroma residual. Considering the high computational complexity of the three modules, waiting for their successful completion before performing the chroma residual scaling of LMCS can cause significant delays for the decoding of chroma samples. In the case of an affine CU, since each affine CU can perform PROF processing followed by LMCS, the PROF processing may also have a delay problem, which may also cause a delay problem in the decoding of chroma samples. Furthermore, in the current design of LMCS, unnecessary clipping operations are performed during the derivation of chroma residual scaling factors, further increasing the computational complexity and the additional requirements for on-chip memory. This disclosure solves or alleviates these problems presented by the current design of LMCS. For the decoding of chroma samples, it may cause a significant delay. In the case of an affine CU, since each affine CU can perform PROF processing followed by LMCS, the PROF processing may also have a delay problem, which may also cause a delay problem in the decoding of chroma samples. For the affine CU case, since each affine CU can perform the PROF process and subsequently execute the LMCS, the PROF process may also have a delay issue, which may also cause a delay problem in the decoding of chroma samples. For the affine CU case, since each affine CU can perform the PROF process and subsequently execute the LMCS, the PROF process may also have a delay issue, which may also cause a delay problem in the decoding of chroma samples. This disclosure solves or alleviates these problems presented by the current design of LMCS.

[0091] Furthermore, in the current design of LMCS, unnecessary clipping operations are performed during the derivation of chroma residual scaling factors, further increasing the computational complexity and the additional requirements for on-chip memory. Furthermore, in the current design of LMCS, unnecessary clipping operations are performed during the derivation of chroma residual scaling factors, further increasing the computational complexity and the additional requirements for on-chip memory. Furthermore, in the current design of LMCS, unnecessary clipping operations are performed during the derivation of chroma residual scaling factors, further increasing the computational complexity and the additional requirements for on-chip memory.

[0092] This disclosure solves or alleviates these problems presented by the current design of LMCS. For the purpose of, more specifically, the present disclosure discusses a method capable of reducing the complexity of LMCS for hardware codec implementation while maintaining coding gain. Instead of using the existing LMCS framework that converts prediction / reconstruction samples by a mapping operation, a new method called Prediction-Dependent Residual Scaling (PDRS) is proposed, which directly scales the prediction residuals without sample mapping. The proposed method can achieve similar effects and coding efficiency as LMCS, but the implementation complexity is much lower.

[0093] In the PDRS procedure, as shown in FIG. 7, luminance prediction samples for decoding luminance residual samples are obtained (701), scaling factors are derived using the luminance prediction samples (702), the luminance residual samples are scaled using the scaling factors (703), and the reconstructed luminance samples are calculated by adding the luminance prediction samples and the scaled luminance residual samples (704). In some examples, an adaptive luminance residual scaling method is proposed to reduce the implementation complexity of LMCS. Specifically, different from the existing LMCS method that directly converts the predicted / reconstructed luminance samples to the mapped domain before calculating the luminance prediction residuals, in the proposed method of the PDRS procedure, the luminance prediction residual samples are derived in the same way as the normal prediction process in the original domain without a mapping operation, and then a scaling operation on the luminance prediction residuals is performed. The scaling of the luminance prediction residuals is based on the corresponding luminance prediction samples. In the PDRS procedure, as shown in FIG. 7, luminance prediction samples for decoding luminance residual samples are obtained (701), scaling factors are derived using the luminance prediction samples (702), the luminance residual samples are scaled using the scaling factors (703), and the reconstructed luminance samples are calculated by adding the luminance prediction samples and the scaled luminance residual samples (704). The proposed method can achieve similar effects and coding efficiency as LMCS, but the implementation complexity is much lower. Instead of using the existing LMCS framework that converts prediction / reconstruction samples by a mapping operation, a new method called Prediction-Dependent Residual Scaling (PDRS) is proposed, which directly scales the prediction residuals without sample mapping.

[0094] In the PDRS procedure, as shown in FIG. 7, luminance prediction samples for decoding luminance residual samples are obtained (701), scaling factors are derived using the luminance prediction samples (702), the luminance residual samples are scaled using the scaling factors (703), and the reconstructed luminance samples are calculated by adding the luminance prediction samples and the scaled luminance residual samples (704). In the PDRS procedure, as shown in FIG. 7, luminance prediction samples for decoding luminance residual samples are obtained (701), scaling factors are derived using the luminance prediction samples (702), the luminance residual samples are scaled using the scaling factors (703), and the reconstructed luminance samples are calculated by adding the luminance prediction samples and the scaled luminance residual samples (704). In the PDRS procedure, as shown in FIG. 7, luminance prediction samples for decoding luminance residual samples are obtained (701), scaling factors are derived using the luminance prediction samples (702), the luminance residual samples are scaled using the scaling factors (703), and the reconstructed luminance samples are calculated by adding the luminance prediction samples and the scaled luminance residual samples (704). In the PDRS procedure, as shown in FIG. 7, luminance prediction samples for decoding luminance residual samples are obtained (701), scaling factors are derived using the luminance prediction samples (702), the luminance residual samples are scaled using the scaling factors (703), and the reconstructed luminance samples are calculated by adding the luminance prediction samples and the scaled luminance residual samples (704). In the PDRS procedure, as shown in FIG. 7, luminance prediction samples for decoding luminance residual samples are obtained (701), scaling factors are derived using the luminance prediction samples (702), the luminance residual samples are scaled using the scaling factors (703), and the reconstructed luminance samples are calculated by adding the luminance prediction samples and the scaled luminance residual samples (704).

[0095] In some examples, an adaptive luminance residual scaling method is proposed to reduce the implementation complexity of LMCS. Specifically, different from the existing LMCS method that directly converts the predicted / reconstructed luminance samples to the mapped domain before calculating the luminance prediction residuals, in the proposed method of the PDRS procedure, the luminance prediction residual samples are derived in the same way as the normal prediction process in the original domain without a mapping operation, and then a scaling operation on the luminance prediction residuals is performed. In some examples, an adaptive luminance residual scaling method is proposed to reduce the implementation complexity of LMCS. Specifically, different from the existing LMCS method that directly converts the predicted / reconstructed luminance samples to the mapped domain before calculating the luminance prediction residuals, in the proposed method of the PDRS procedure, the luminance prediction residual samples are derived in the same way as the normal prediction process in the original domain without a mapping operation, and then a scaling operation on the luminance prediction residuals is performed. In some examples, an adaptive luminance residual scaling method is proposed to reduce the implementation complexity of LMCS. Specifically, different from the existing LMCS method that directly converts the predicted / reconstructed luminance samples to the mapped domain before calculating the luminance prediction residuals, in the proposed method of the PDRS procedure, the luminance prediction residual samples are derived in the same way as the normal prediction process in the original domain without a mapping operation, and then a scaling operation on the luminance prediction residuals is performed. It depends on the value and the piecewise linear model. As a result, the forward and inverse luminance mapping operations in the current LMCS design can be completely removed, and all prediction samples and reconstruction samples involved in the decoding process are maintained in the original sampled domain. Based on the above features, the proposed method is called prediction-dependent residual scaling. Furthermore, in order to improve the delay of chroma residual scaling derivation, several methods can be proposed to completely or partially exclude the DMVR, BDOF, and CIIP operations from the generation of the luminance prediction samples used to calculate the scaling parameters of the chroma residual samples.

[0096] Figure 8 is a flowchart showing the workflow of the decoding process when the PDRS procedure is applied in the LMCS process. This shows the elimination of the need for mapping between different domains. Here, except for the residual decoding modules (e.g., entropy decoding 801, inverse quantization 802, and inverse transform 803), all other decoding modules (intra prediction 80 4, 809, 812, and inter prediction 816, reconstruction 806 and 813, and all in-loop filters 807 and 814 included) operate in the original domain. Specifically, to reconstruct the luminance samples, the method proposed in the PDRS procedure only needs to inverse scale the prediction residual samples Y to the original amplitude level and then add them to the luminance prediction samples Y res pred

[0097] By the PDRS procedure, the forward and inverse luminance sample mapping operations in the existing LMCS design are completely removed. This not only saves / reduces the computational complexity but also LMC ​​​​​​Reduce the potential memory capacity size to save S parameters. For example, when the LUT-based solution is used to perform luminance mapping, the previous memory capacity used to store two mapping LUTs FwdMap[] and InvMap[] (about 2560 bytes) is no longer required in the proposed method. Further, unlike the existing luminance mapping method that requires storing reconstructed luminance samples in both the original domain and the mapped domain, the proposed method of the PDRS procedure generates and maintains all prediction samples and reconstruction samples only in the original domain. Correspondingly, compared with the existing luminance mapping, the proposed method in the PDRS procedure can efficiently reduce the size of the line buffer used to store the samples reconstructed for intra prediction and deblocking by half.

[0098] According to one or more embodiments of the PDRS procedure, the luminance prediction samples and luminance residual samples are from the same position within one of the luminance prediction block and its associated residual block. That is.

[0099] According to one or more embodiments of the PDRS procedure, the step of deriving the scaling factor using the luminance prediction samples includes dividing the entire range of possible luminance prediction sample values into a plurality of luminance prediction sample segments, calculating one scaling factor for each of the plurality of luminance prediction sample segments based on a predefined section linear model, and determining the scaling factor of the luminance prediction samples based on the scaling factors of the plurality of luminance prediction sample segments.

[0100] In one example, a luminance prediction is performed based on a scaling factor for a plurality of luminance prediction sample segments. The step of determining a scaling factor for the sample comprises: A step of assigning the brightness prediction sample to one of the sample segments and Scaling factor for the luminance prediction sample segment assigned the scaling factor of the and calculating:

[0101] In this example, the plurality of luma prediction sample segments includes 16 segments in 16 predefined LUT tables scaleForward, and a predefined LUT table for calculating one scaling factor for each of the plurality of luma prediction sample segments. section The linear model contains 16 values corresponding to the 16 segments in the predefined LUT table scaleForward.

[0102] To maintain the accuracy of the operation, the luminance residual samples are scaled / descaled. The scaling parameters used to compute the The prediction may be determined based on a predicted sample. Y One brightness prediction sample The scaling factor of the corresponding residual sample is calculated by the following steps: It is calculated via.

[0103] In the same example, the scaling factor (e.g., the luminance residual scaling factor Scale Y )teeth , the assigned luminance prediction samples, as described in the box immediately following this paragraph. Calculated based on segments. Scale Y =scaleForward[Idx Y ] Here, Y is the luminance residual value for which the scaling coefficient is calculated, and Scale Y is , the scaling coefficient, and scaleForward[i] (i = 0...15) is the pre - defined 16 LUT tables, and Idx Y is the segment index of the segment assigned to the luminance prediction sample domain value. scaleForward[i] (i = 0...15) is pre - calculated as follows . scaleForward[i]=(OrgCW << SCALE_FP_PREC) / SignaledCW[i] Here, OrgCW and SignaledCW[i] are the number of codewords of the i - th segment in the original domain and the mapped domain, respectively, and SCALE _FP_PREC is the accuracy of the scaling coefficient.

[0104] In the same example, when the luminance scaling coefficient Scale Y is given, the luminance residual sample scaling method can be applied as described in the box immediately following this paragraph .

[0105] The motivation behind this example is that the forward mapping in the current LMCS is based on a single - segment linear model. When both the original luminance sample and the luminance prediction sample are located in the same segment (i.e., the same segment defined by two pivot points InputPivot[i] and InputPi vot[i + 1]), the two forward mapping functions for the original luminance sample and the predicted luminance sample are exactly the same. Corresponding to this Then, Y’res = FwdMap(Yorg) - FwdMap(Ypred) = FwdMa p(Yorg - Ypred) == FwdMap(Yres). By applying the inverse mapping to both sides of this equation, the corresponding decoder-side reconstruction operation can be expressed as follows and can be done. Yrecon = Ypred + InvMap(Y’res).

[0106] In other words, in the situation where both the original luminance sample and the luminance prediction sample are located in the same segment, the luminance mapping method in LMCS can be achieved, for example, as shown in FIG. 8, by one residual scaling operation in the decoding process as implemented in this possible implementation form. Encoder side:

Number

Number

Number

[0107] Such a conclusion is derived based on the assumption that both the original luminance sample and the luminance prediction sample are located in the same segment defined by two pivot points InputPivot[i] and InputPivot[i + 1]. However, this possible implementation form of this example can still be used as a simplification and / or approximation of the existing luminance mapping operation in VVC even when the original luminance sample and the luminance prediction sample are located in different segments of the piecewise linear model. The experimental results show that such a simplification and / or approximation has little impact on the coding performance.

[0108] ​​​​​​​​Although it is repetitive, this example is based on the assumption that both the original luminance sample value and the predicted luminance sample value are located in the same segment of the piecewise linear mode. In this case, the forward / inverse mapping functions applied to the original luminance sample and the predicted luminance sample are the same. Therefore, it is safe to calculate the corresponding residual scaling coefficient simply depending on the luminance prediction sample.

[0109] However, when the prediction sample of the CU is not accurate enough (for example, samples far from the reference sample are usually not predicted very accurately, in the case of an intra-predicted CU ), the prediction sample and the original sample often are located in different segments of the piecewise linear model. In this case, the scaling coefficient derived based on the prediction sample value may not be reliable when reflecting the original mapping relationship between the residual sample in the original (i.e., unmapped) domain and the residual sample in the mapped domain.

[0110] FIG. 9 is a diagram showing the residual mapping error caused by using only the prediction sample to derive the scaling coefficient. In FIG. 9, the filled triangular points represent the pivot control points of different segments in the piecewise linear function, and the filled circular points represent the original sample value and the predicted sample value. Y and Y are the original sample and the predicted sample in the original (i.e., unmapped) domain. Y' org and Y' pred are the original sample and the predicted sample in the original (i.e., unmapped) domain. Y' and Y' org and Y' pred are Y org and Y pre d is the mapped sample. Y res and Y' res are the original domain and the corresponding residuals in the mapped domain when the existing sample-based luminance mapping method in VVC is applied. Y' is the corresponding residual in the mapped domain. resScale Y' is the mapped residual sample derived based on the proposed luminance residual scaling method. As shown in FIG. 9, since the original sample and the predicted sample are not within the same segment of the piecewise linear model, the scaling coefficient derived based on the predicted sample may not be accurate enough to approximate the original residual within the mapped domain (i.e., Y' ). That is, Y' res ) to generate the scaled residual (i.e., Y' ). resScale ) may not be accurate enough.

[0111] In the second example, it is not necessary to assume that both the original luminance sample value and the predicted luminance sample value are located in the same segment of the piecewise linear model.

[0112] In this second example, to improve the accuracy of the residual scaling coefficient, instead of directly deriving the scaling coefficient from the segment of the piecewise linear model where the luminance prediction sample is located, the scaling coefficient is calculated as the average of the scaling coefficients of N (N is a positive integer) adjacent segments.

[0113] In this second example, the step of determining the scaling coefficient of the luminance prediction sample based on the scaling coefficients of multiple luminance prediction sample segments includes the step of assigning the luminance prediction sample to one of the multiple luminance prediction sample segments, and the luminance prediction sample to one of the segments of the multiple ​​​The scaling factor of the measurement sample is calculated as the average of the scaling factors of several luminance prediction sample segments adjacent to the assigned luminance prediction sample segment. This includes the step of calculating.

[0114] More specifically, in one possible implementation of this second example, the scaling factor can be calculated based on the assigned luminance prediction sample segment as described in the following steps. For example, on the decoder side, when the luminance prediction sample Pred and the luminance residual Y are given, the scaling factor applied to the inverse scaling

Number

Number

Number

Number

Number

Number

[0115] In a second possible implementation of this second example, which is identical to the above implementation in other respects, the scaling coefficient can be calculated based on the assigned luminance prediction sample segment as described in the box immediately following this paragraph. 1) Find or obtain the corresponding segment index Idx Y of the piece - wise linear model to which Pred belongs in the original domain Y 2) The luminance residual scaling coefficient is calculated as follows.

Equation

[0116] These two possible implementations of this second example differ only in the selection of N luminance prediction sample domain values segments based on the assigned segments.

[0117] ​​​​In one chroma sample reconstruction procedure, as shown in FIG. 10, at the input position, the luminance residual A luminance prediction sample value is obtained to decode both the sample and the chroma residual sample (1 001), then a luminance prediction sample associated with the luminance residual sample is obtained (10 02), then a chroma prediction sample associated with the chroma residual sample is obtained (100 3), the luminance prediction sample is used to derive a first scaling coefficient of the luminance residual sample and a second scaling coefficient of the chroma residual sample (1004), the first scaling coefficient is used to scale the luminance residual sample (1005), the second scaling coefficient is used to scale the chroma residual sample (1006), the reconstructed luminance sa mple is calculated by adding the chroma prediction sample and the scaled chroma residual sample (1007), and the reconstructed chroma sample is calculated by adding the chroma prediction sample and the scaled chroma residual sample (1008). The chroma sample reconstruction procedure aims to harmonize the scaling methods of the luminance residual and the chroma residual so as to achieve a more unified design.

[0118] According to one or more embodiments of the chroma sample reconstruction procedure, the luminance prediction sample value is the average of all the luminance prediction samples within the coding unit (CU) including the input position.

[0119] In these embodiments, the chroma scaling derivation method is used to calculate the scaling coefficient of the luminance residual, and more specifically, instead of separately deriving one scaling coefficient for each luminance residual sample, one calculated based on the average of the luminance prediction samples ​​​​​The total scaling factor is used to scale the luminance residual samples of the entire CU. It is used.

[0120] According to another embodiment of the chroma sample reconstruction procedure, the luminance prediction sample value is the average of all luminance prediction samples within a predefined sub-block subdivided from the coding unit (CU) containing the input position. In this embodiment, a sub-block-based method can be proposed to derive the scaling factors for both the luminance residual and the chroma residual. Specifically, one CU is first equally divided into a plurality of M×N sub-blocks. For each sub-block, all or partial luminance prediction samples are used to derive the corresponding scaling factor used to scale the residuals of both the luminance and chroma of the sub-block. Compared with the first method, in the second method, the low-correlation luminance prediction samples outside the sub-block are excluded from the calculation of the sub-block scaling factor, so the spatial accuracy of the estimated scaling factor can be improved. On the other hand, considering that in the second method, the scaling of the luminance residual and chroma residual within one sub-block can be started immediately after the luminance prediction of the sub-block is completed, that is, without waiting for the complete generation of the luminance prediction samples of the entire CU, the delay in luminance and chroma residual reconstruction can be reduced. According to a third embodiment of the chroma sample reconstruction procedure, the luminance prediction sample domain value includes the luminance prediction samples arranged at the same location. In this embodiment, the luminance residual scaling method is extended to scale the chroma residual, and different scaling factors for each chroma residual sample are used. This can reduce the delay in luminance and chroma residual reconstruction. It can be reduced.

[0121] According to a third embodiment of the chroma sample reconstruction procedure, the luminance prediction sample domain value includes the luminance prediction samples arranged at the same location. In this embodiment, the luminance residual scaling method is extended to scale the chroma residual, and different scaling factors for each chroma residual sample The ringing coefficient is derived based on the luminance prediction sample values arranged at the same location .

[0122] In the above embodiment of the chroma sample reconstruction procedure, in order to perform scaling of the chroma residual , it is proposed to use the same LUT as that used for the calculation of the luminance scaling coefficient . In one example, for deriving the scaling coefficient Scale C at the CU level of the chroma residual, the following can be followed. 1) Calculate the average of the luminance prediction samples (represented in the original domain) within the CU, denoted as avg Y . 2) Find or obtain the corresponding segment index Idx Y of the piecewise linear model to which avg Y belongs. 3) Calculate the value of Scale C as follows. Scale C = scaleForward[Idx Y Here, scaleForward[i] (i = 0...15) is one pre - defined 16 - entry LUT, which is calculated as follows. scaleForward[i]=(OrgCW << SCALE_FP_PREC) / SignaledCW[i] Here, OrgCW and SignaledCW[i] are the number of codewords of the i - th segment in the original domain and the mapped domain respectively, and S CALE_FP_PREC is the accuracy of the scaling coefficient.

[0123] In the above example, the scaling coefficient of the chroma residual is derived for each sub - block of the current CU ​​​​When it can be easily extended. In that case, in the above first step, av gY is calculated as the average of the luminance prediction samples in the original domain of the sub-block, but steps 2 and 3 remain the same.

[0124] In the second chroma sample reconstruction procedure, as shown in FIG. 11, during the luminance prediction process for the coding unit (CU), skip some pre-defined intermediate luminance prediction stages to obtain a plurality of luminance prediction samples (1101). Using the obtained plurality of luminance prediction samples to derive the scaling coefficients for the chroma residual samples within the CU (11 02), and using the scaling coefficients to scale the chroma residual samples within the CU (1 103). The reconstructed chroma samples are calculated by adding the chroma prediction samples and the scaled chroma residual samples within the CU (1104).

[0125] According to one or more embodiments of the second chroma sample reconstruction procedure, the pre-defined intermediate luminance prediction stages include one or more bidirectional prediction modules of decoder-side motion vector derivation (DMVR), bidirectional optical flow (BDOF), and combined inter-intra prediction (CIIP). In these embodiments, to solve the latency problem, use the inter-prediction samples derived before DMVR, BDOF / PROF, and CIIP intra / inter combination processing to derive the scaling coefficients of the chroma residuals. BDOF / PROF, CIIP intra / inter combination processing to derive the scaling coefficients of the chroma residuals.

[0126] FIG. 12 shows this embodiment of the second chroma sample reconstruction procedure where DMVR, BDOF, and CIIP are not applied to generate luminance prediction samples for chroma scaling ​​​A flowchart showing the workflow of LMCS decoding process in an example. Here , instead of waiting for the complete termination of DMVR 1203, BDOF 1204 and / or the luminance intra prediction of CIIP section 1205, as soon as the prediction samples 1221 and 1222 based on the initial L0 and L1 luminance predictions 1201 and 1202 become available, the chroma residual scaling process 1208 can be started.

[0127] In FIG. 12, before DMVR 1203, BDOF 1204, and / or CIIP 1 205, in addition to the original averaging operation 1206, one additional averaging operation 1211 is required to combine the initial L0 and L1 prediction samples 1221 and 1222

[0128] In a second example of this embodiment of the second chroma sample reconstruction procedure to reduce complexity , the initial L0 prediction sample is always used and the scaling factor for the chroma residual can be derived

[0129] FIG. 13 is a flowchart showing the workflow of the LMCS decoding process in a second example of this embodiment of the second chroma sample reconstruction procedure, where the initial single prediction signal is applied to generate the luminance prediction sample for chroma scaling . No additional averaging operation is required in addition to the original averaging operation 1306. The prediction sample 1321 of the initial L0 is used to derive the scaling factor of the chroma residual before DMVR 1303, BDOF 1304, and / or CIIP 1305.

[0130] In a third example of this embodiment of the second chroma sample reconstruction procedure, chroma residual scaling​​​​​​​ As a luminance prediction sample used to derive the coefficient, one initial prediction signal (L0 or L1) is adaptively selected. In one possible implementation of this example, among the initial prediction signals (L0 or L1), the one with a picture order count (POC) distance whose reference picture is smaller than the current picture is selected to derive the chroma residual scaling coefficient.

[0131] In another embodiment of the second chroma sample reconstruction procedure, it is proposed to disable only DMVR and BDOF / PROF while enabling CIIP to generate the inter-prediction sample used to determine the chroma residual scaling coefficient. Specifically, in this method, the inter-prediction samples derived before DMVR and BDOF / PROF are first averaged. Next, they are combined with the intra-prediction samples of CIIP. Finally, the combined prediction samples are used as the prediction samples for determining the chroma residual scaling coefficient.

[0132] In yet another embodiment of the second chroma sample reconstruction procedure, it is proposed to disable only BDOF / PROF while maintaining DMVR and CIIP to generate the prediction samples used to determine the chroma residual scaling coefficient.

[0133] In yet another embodiment of the second chroma sample reconstruction procedure, it is proposed to maintain BDOF / PROF and CIIP while disabling DMVR when deriving the luminance prediction samples used to determine the chroma residual scaling coefficient.

[0134] Furthermore, the method in the above embodiment of the second chroma sample reconstruction procedure is shown because it is designed to reduce the delay of chroma prediction residual scaling, but it is worth mentioning that those methods can also be used to reduce the delay of luminance prediction residual scaling. For example, all methods can also be applied to the PDRS method described in the section of "Luminance mapping based on prediction-dependent residual scaling".

[0135] According to the existing DMVR design, in order to reduce the computational complexity, the prediction samples used for DMVR motion fine-tuning are generated using a 2-tap bilinear filter instead of the default 8-tap interpolation. After the corrected motion is determined, the default 8-tap filter is applied to generate the final prediction sample of the current CU. Therefore, in order to reduce the chroma residual decoding delay caused by DMVR, it is proposed to determine the scaling coefficient of the chroma residual using the luminance prediction samples (the average of the L0 and L1 prediction samples if the current CU is bi-predicted) generated by the bilinear filter.

[0136] According to one chroma residual sample reconstruction procedure, as shown in FIG. 14, one or more luminance prediction sample values are selected (1401) from the output of the bilinear filter of decoder-side motion vector derivation (DMVR), and one or more selected luminance prediction sample values are adjusted (1402) to another one or more luminance prediction sample values having the same bit depth as the original encoded bit depth of the input video, and one or more chroma residual samples are recovered using the luminance prediction sample values having the same bit depth as the original encoded bit depth of the input video. Derive a scaling factor for scaling (1403), and use the scaling factor to scale one or more chroma residual samples (1404), and one or more chroma residual samples are reconstructed by adding them to their corresponding chroma prediction samples (1405). In one or more embodiments of the chroma residual sample reconstruction procedure, the step of selecting one or more luminance prediction sample values from the output of the bilinear filter of DMVR includes the step of selecting the luminance prediction samples of L0 and L1 from the output of the bilinear filter of DMVR. In one or more embodiments of the chroma residual sample reconstruction procedure, the step of selecting one or more luminance prediction sample values from the output of the bilinear filter of DMVR includes the step of selecting the luminance prediction samples of L0 and L1 from the output of the bilinear filter of DMVR. In one or more embodiments of the chroma residual sample reconstruction procedure, the step of selecting one or more luminance prediction sample values from the output of the bilinear filter of DMVR includes the step of selecting the luminance prediction samples of L0 and L1 from the output of the bilinear filter of DMVR.

[0137] In one or more embodiments of the chroma residual sample reconstruction procedure, the step of selecting one or more luminance prediction sample values from the output of the bilinear filter of DMVR includes the step of selecting the luminance prediction samples of L0 and L1 from the output of the bilinear filter of DMVR. In one or more embodiments of the chroma residual sample reconstruction procedure, the step of selecting one or more luminance prediction sample values from the output of the bilinear filter of DMVR includes the step of selecting the luminance prediction samples of L0 and L1 from the output of the bilinear filter of DMVR. In one or more embodiments of the chroma residual sample reconstruction procedure, the step of selecting one or more luminance prediction sample values from the output of the bilinear filter of DMVR includes the step of selecting the luminance prediction samples of L0 and L1 from the output of the bilinear filter of DMVR. In one or more embodiments of the chroma residual sample reconstruction procedure, the step of selecting one or more luminance prediction sample values from the output of the bilinear filter of DMVR includes the step of selecting the luminance prediction samples of L0 and L1 from the output of the bilinear filter of DMVR.

[0138] FIG. 15 is a flowchart showing the workflow of the LMCS decoding process in such an embodiment of the chroma residual sample reconstruction procedure. The L0 and L1 prediction samples 1521 and 1522 from the output of the bilinear filter component 1512 of DMVR 1503 are supplied to the average 1511 to derive the chroma residual scaling input 1523 to be used in the chroma residual scaling 1507 for decoding one or more chroma residual samples. FIG. 15 is a flowchart showing the workflow of the LMCS decoding process in such an embodiment of the chroma residual sample reconstruction procedure. The L0 and L1 prediction samples 1521 and 1522 from the output of the bilinear filter component 1512 of DMVR 1503 are supplied to the average 1511 to derive the chroma residual scaling input 1523 to be used in the chroma residual scaling 1507 for decoding one or more chroma residual samples. FIG. 15 is a flowchart showing the workflow of the LMCS decoding process in such an embodiment of the chroma residual sample reconstruction procedure. The L0 and L1 prediction samples 1521 and 1522 from the output of the bilinear filter component 1512 of DMVR 1503 are supplied to the average 1511 to derive the chroma residual scaling input 1523 to be used in the chroma residual scaling 1507 for decoding one or more chroma residual samples. FIG. 15 is a flowchart showing the workflow of the LMCS decoding process in such an embodiment of the chroma residual sample reconstruction procedure. The L0 and L1 prediction samples 1521 and 1522 from the output of the bilinear filter component 1512 of DMVR 1503 are supplied to the average 1511 to derive the chroma residual scaling input 1523 to be used in the chroma residual scaling 1507 for decoding one or more chroma residual samples. FIG. 15 is a flowchart showing the workflow of the LMCS decoding process in such an embodiment of the chroma residual sample reconstruction procedure. The L0 and L1 prediction samples 1521 and 1522 from the output of the bilinear filter component 1512 of DMVR 1503 are supplied to the average 1511 to derive the chroma residual scaling input 1523 to be used in the chroma residual scaling 1507 for decoding one or more chroma residual samples. FIG. 15 is a flowchart showing the workflow of the LMCS decoding process in such an embodiment of the chroma residual sample reconstruction procedure. The L0 and L1 prediction samples 1521 and 1522 from the output of the bilinear filter component 1512 of DMVR 1503 are supplied to the average 1511 to derive the chroma residual scaling input 1523 to be used in the chroma residual scaling 1507 for decoding one or more chroma residual samples.

[0139] In these embodiments, there is a problem with the bit code depth. To save the internal memory capacity size used by DMVR, the intermediate L0 and L1 prediction samples generated by the bilinear filter of DMVR are of 10-bit precision. This is different from the representation bit depth of the immediate prediction samples of the normal dual prediction equivalent to 14 bits. Therefore, the intermediate prediction samples output from the bilinear filter, due to their different precision, are used for chroma residual scaling. In these embodiments, there is a problem with the bit code depth. To save the internal memory capacity size used by DMVR, the intermediate L0 and L1 prediction samples generated by the bilinear filter of DMVR are of 10-bit precision. This is different from the representation bit depth of the immediate prediction samples of the normal dual prediction equivalent to 14 bits. Therefore, the intermediate prediction samples output from the bilinear filter, due to their different precision, are used for chroma residual scaling. In these embodiments, there is a problem with the bit code depth. To save the internal memory capacity size used by DMVR, the intermediate L0 and L1 prediction samples generated by the bilinear filter of DMVR are of 10-bit precision. This is different from the representation bit depth of the immediate prediction samples of the normal dual prediction equivalent to 14 bits. Therefore, the intermediate prediction samples output from the bilinear filter, due to their different precision, are used for chroma residual scaling. In these embodiments, there is a problem with the bit code depth. To save the internal memory capacity size used by DMVR, the intermediate L0 and L1 prediction samples generated by the bilinear filter of DMVR are of 10-bit precision. This is different from the representation bit depth of the immediate prediction samples of the normal dual prediction equivalent to 14 bits. Therefore, the intermediate prediction samples output from the bilinear filter, due to their different precision, are used for chroma residual scaling. In these embodiments, there is a problem with the bit code depth. To save the internal memory capacity size used by DMVR, the intermediate L0 and L1 prediction samples generated by the bilinear filter of DMVR are of 10-bit precision. This is different from the representation bit depth of the immediate prediction samples of the normal dual prediction equivalent to 14 bits. Therefore, the intermediate prediction samples output from the bilinear filter, due to their different precision, are used for chroma residual scaling. It cannot be directly applied to determine the Kaling factor.

[0140] To address this issue, we first consider the DMVR intermediate bit depth as a normal motion compensated interpolation. Match the intermediate bit depth used, i.e., change the bit depth from 10 bits to 14 bits. It is then proposed to increase the number of bits to 1, which is then applied to generate the normal bi-predictive signal. We reuse the existing averaging process to determine the chroma residual scaling factor. A predicted sample can be generated.

[0141] In one example of these embodiments, one or more selected luminance prediction sample values are input. Another luma prediction block or blocks having the same bit depth as the original coding bit depth of the input video. The output of the DMVR bilinear filter is adjusted to the measured sample value by left shifting. Increasing the internal bit depth of the L0 and L1 luma prediction samples from and by averaging the 14-bit shifted L0 and L1 luma prediction sample values. Obtain the 14-bit average brightness prediction sample value and shift right to obtain the 14-bit average Changing the internal bit depth of the luma prediction sample values to the original coding bit depth of the input video. and converting the 14-bit average luminance prediction sample value by:

[0142] More specifically, in this example, the saturation scaling factor is written in the box immediately following this paragraph: This is determined by the steps described. 1) Intra-bit-depth matching. The bilinear filter generates The internal bit depth of the generated L0 and L1 predicted samples has been increased from 10 bits to 14 bits. Increase to. [Number] Here, [Number] and [Number] is the predicted sample output from the bilinear filter, [Number] and [Number] is the scaled predicted sample after bit depth alignment. [Number] is a constant used to compensate for the shifted dynamic range of the predicted sample resulting from the following averaging operation is used to compensate for the shifted dynamic range of the predicted sample resulting from the following averaging operation. 2) Average of the scaled predicted samples of L0 and L1. Chroma residual scaling The final luminance sample used to determine the scaling factor is calculated by averaging two scaled luminance predicted samples as follows. [Number] Here, the bit depth is the encoded bit depth of the input video.

[0143] In other embodiments of the chroma residual sample reconstruction procedure, one or more luminance predicted sample values are selected from the output of the bilinear filter of the DMVR, and one or more other luminance predicted sample values having the same bit depth as the original encoded bit depth of the input video are added to one or more other luminance predicted sample values having the same bit depth as the original encoded bit depth of the input video. The step of adjusting the selected luminance prediction sample value is a bilinear filter of DMVR select one of the luminance prediction samples of L0 and L1 from the output of a step, and the internal bit depth of one luminance prediction value selected by the shift is the original coded bit depth of the input video changing to adjust one selected luminance prediction sample by changing to the original coded bit depth of the input video a step, and a step of using the luminance prediction sample adjusted as a luminance prediction sample having the same bit depth as the original coded bit depth of the input video, including.

[0144] Figure 16 is a flowchart showing the workflow of the LMC S decoding process in such another embodiment of the chroma residual sample reconstruction procedure. The L0 prediction sample 1621 from the output of the bilinear filter 1612 component of DMVR 1603 is used in the chroma residual scaling 1607 for decoding one or more chroma residual samples. In this embodiment it is proposed to directly use the initial single prediction sample (i.e., the L0 prediction sample) to derive the scaling factor of the chroma residual.

[0145] In an example of such another embodiment of the chroma residual sample reconstruction procedure, assuming that the current CU is bi-predicted, the chroma scaling factor is as described in the box immediately following this paragraph, and the luminance sample output from the bilinear filter is the original symbol of the input video judged by shifting to the coded bit depth. If the bit depth is 10 or less

Number

Number

[0146] Finally, instead of generating luminance prediction samples, reference samples (i.e., samples at integer positions taken from external reference pictures) are directly used to determine the scaling factor of the chroma residual. It is proposed to use the average of the reference samples at L0 and L1 to determine the chroma residual scaling factor. In one or more embodiments, it is proposed to use only the reference samples in one direction (e.g., list L0) to calculate the chroma residual scaling factor. According to the second chroma residual sample reconstruction procedure, as shown in FIG. 17, one or more luminance reference sample values are selected from the reference picture (1701), one or more selected luminance reference sample values are converted to luminance sample values (1702), the converted luminance samples are used to derive a scaling factor (1703), the scaling factor is used to scale one or more chroma residual samples (1704), and one or more chroma residual samples are reconstructed by adding one or more scaled chroma residual samples and their corresponding chroma prediction samples (1705). In one or more embodiments of the second chroma residual sample reconstruction procedure, the step of selecting one or more luminance reference sample values from the reference picture and converting one or more selected luminance reference sample values to luminance sample values includes obtaining both the luminance reference sample values of L0 and L1 from the L0 and L1 reference pictures and the converted luminance samples. It can be proposed for only the reference samples in one direction (e.g., list L0) to calculate the chroma residual scaling factor.

[0147] According to the second chroma residual sample reconstruction procedure, as shown in FIG. 17, one or more luminance reference sample values are selected from the reference picture (1701), one or more selected luminance reference sample values are converted to luminance sample values (1702), the converted luminance samples are used to derive a scaling factor (1703), the scaling factor is used to scale one or more chroma residual samples (1704), and one or more chroma residual samples are reconstructed by adding one or more scaled chroma residual samples and their corresponding chroma prediction samples (1705). In one or more embodiments of the second chroma residual sample reconstruction procedure, the step of selecting one or more luminance reference sample values from the reference picture and converting one or more selected luminance reference sample values to luminance sample values includes obtaining both the luminance reference sample values of L0 and L1 from the L0 and L1 reference pictures and the converted luminance samples. One or more selected luminance reference sample values are converted to luminance sample values (1702), the converted luminance samples are used to derive a scaling factor (1703), the scaling factor is used to scale one or more chroma residual samples (1704), and one or more chroma residual samples are reconstructed by adding one or more scaled chroma residual samples and their corresponding chroma prediction samples (1705). The converted luminance samples are used to derive a scaling factor (1703), the scaling factor is used to scale one or more chroma residual samples (1704), and one or more chroma residual samples are reconstructed by adding one or more scaled chroma residual samples and their corresponding chroma prediction samples (1705). The scaling factor is used to scale one or more chroma residual samples (1704), and one or more chroma residual samples are reconstructed by adding one or more scaled chroma residual samples and their corresponding chroma prediction samples (1705). One or more chroma residual samples are reconstructed by adding one or more scaled chroma residual samples and their corresponding chroma prediction samples (1705). One or more chroma residual samples are reconstructed by adding one or more scaled chroma residual samples and their corresponding chroma prediction samples (1705).

[0148] In one or more embodiments of the second chroma residual sample reconstruction procedure, the step of selecting one or more luminance reference sample values from the reference picture and converting one or more selected luminance reference sample values to luminance sample values includes obtaining both the luminance reference sample values of L0 and L1 from the L0 and L1 reference pictures and the converted luminance samples. One or more luminance reference sample values are selected from the reference picture, and the step of converting one or more selected luminance reference sample values to luminance sample values includes obtaining both the luminance reference sample values of L0 and L1 from the L0 and L1 reference pictures and the converted luminance samples. The step of converting one or more selected luminance reference sample values to luminance sample values includes obtaining both the luminance reference sample values of L0 and L1 from the L0 and L1 reference pictures and the converted luminance samples. The step of obtaining both the luminance reference sample values of L0 and L1 from the L0 and L1 reference pictures and the converted luminance samples. including averaging the luminance reference sample values of L0 and L1 as a luminance value.

[0149] In another embodiment of the second chroma residual sample reconstruction procedure, one or a plurality of luminance reference samples are selected from the reference picture, and one or a plurality of selected luminance reference samples are converted into luminance sample values. The step of converting includes selecting one luminance reference sample value from among the L0 and L1 luminance reference sample values from the L0 and L1 reference pictures, and using the one selected luminance reference sample value as the converted luminance sample value.

[0150] According to the existing LMCS design, the reconstructed luminance samples adjacent to the 64x64 region where the current CU is located are used to calculate the chroma residual scaling factor of the CUs within the region. Furthermore, one clipping operation, namely Clip1(), is applied to clip the reconstructed luminance adjacent samples to the dynamic range of the internal bit depth ([0, ( (1<<bitDepth)-1]) before the average is calculated.

[0151] Specifically, the method first fetches the 64 left adjacent luminance samples and the 64 upper adjacent luminance samples of the corresponding 64x64 region to which the current CU belongs. Then, the average of the left and upper adjacent samples, that is, avgY, is calculated, and the segment index Y of avgY within the LMCS piecewise linear model is found. Finally, the chroma residual C 1dx is derived as C = cScaleInv[Y Scal eInv 1dx

[0152] ​​Specifically, in the current VVC draft, how to derive the corresponding average luminance is as follows and is described as follows. The clipping operation Clip 1() is applied as shown in a prominent font size . To derive the variable varScale, the following sequential steps are applied The variable invAvgLuma is derived as follows - The array recLuma[i] (i = 0(2*sizeY - 1)), and the variable cnt are derived as follows - The variable cnt is set equal to 0 - If availL is equal to TRUE, the array recLuma[i] (i = 0..size Y - 1) is set equal to currPic[xCuCb - 1][Min(yCuCb + i, pic_ height_in_luma_samples - 1)] (i = 0..sizeY - 1) and cnt is set equal to sizeY - If availT is equal to TRUE, the array recLuma[cnt + i] (i = 0 ..sizeY - 1) is set equal to currPic[Min(xCuCb + i, pic_widt h_in_luma_samples - 1)][yCuCb - 1] (i = 0..size Y - 1) and cnt is set equal to (cnt + sizeY) - The variable invAvgLuma is derived as follows - If cnt is greater than 0, the following applies

Equation

[0153] However, in the reconstruction process, after adding the prediction sample to the residual sample of one CU the resulting sample value is already clipped to the dynamic range of the internal bit depth This means that all adjacent reconstructed luminance samples around the current 64x64 region are guaranteed to be within the range of the internal bit depth. Therefore, their average, that is, avgY cannot exceed this range either. As a result, the existing clipping (i.e., Clip 1()) does not need to calculate the corresponding chroma residual scaling coefficient To further reduce the complexity and memory requirements of the LMCS design, it is proposed to remove the clipping operation and derive the chroma residual scaling coefficient starting from when calculating the average of adjacent reconstructed luminance samples

[0154] FIG. 18 is a flowchart showing the steps of the non-clipping chroma residual scaling coefficient derivation procedure In FIG. 18, a plurality of reconstructed luminance samples from a first predetermined region adjacent to a second predetermined region where the CU is located are selected (1801) during the decoding of the CU the average of the plurality of reconstructed luminance samples is calculated (1802), and the average of the plurality of reconstructed luminance samples is directly used without clipping when deriving the chroma residual scaling coefficient for decoding the CU (1803).

[0155] In one or more embodiments of the non-clipping chroma residual scaling coefficient derivation procedure the average of the plurality of reconstructed luminance samples is the arithmetic mean of the plurality of reconstructed luminance samples.​​​​​

[0156] In one or more embodiments of the non-clipping chroma residual scaling coefficient derivation procedure, When deriving the chroma residual scaling coefficient for decoding the CU, the step of directly using the average of a plurality of reconstructed luminance samples without clipping is to identify the segment index of the average in a predefined segmented model, and to derive the chroma residual scaling coefficient for decoding the CU based on the gradient of the segment line model. This includes the step of identifying the segment index of the average in a predefined segmented model, and the step of deriving the chroma residual scaling coefficient for decoding the CU based on the gradient of the segment line model. This includes the steps of:

[0157] In one or more embodiments of the non-clipping chroma residual scaling coefficient derivation procedure, generating luminance prediction samples and luminance residual samples within a first predetermined region; adding the luminance residual samples to the luminance prediction samples; and clipping the added luminance samples to the dynamic range of the coded bit depth, thereby generating a plurality of reconstructed luminance samples within the first predetermined region. This includes the steps of:

[0158] In one or more embodiments of the non-clipping chroma residual scaling coefficient derivation procedure, the plurality of reconstructed luminance samples are a plurality of forward-mapped inter-luminance reconstructed samples. This includes the steps of:

[0159] In one or more embodiments of the non-clipping chroma residual scaling coefficient derivation procedure, the second predetermined region is a 64×64 region where the CU is located.

[0160] In one example, as shown in FIG. 19, the first predetermined region may include upper adjacent samples within a 1x64 region 1902 directly above the second predetermined region 1904. Alternatively, or in addition to this, In addition, the first predetermined region is a 64x1 region immediately to the left of the second predetermined region 1904 and may include left neighboring samples within 1903.

[0161] According to the existing LMCS design, the reconstructed samples in both the original domain and the mapped domain are used for CUs encoded in different modes. Correspondingly, multiple LMCS transforms are involved in the current encoding / decoding process to convert predicted and reconstructed luminance samples between the two domains. Specifically, in the intra mode, CIIP mode, and IBC mode, the reference samples from the adjacent reconstructed domains of one current CU used to generate intra prediction samples are maintained within the mapped domain. In contrast, in the CIIP mode and all inter modes, the motion compensated prediction samples generated from the temporal reference pictures are in the original domain. Since the luminance reconstruction operation is performed in the mapped domain, those inter prediction samples of the luminance component need to be converted to the mapped domain before being added to the residual samples. On the other hand, for both intra and inter modes, the inverse mapping is always applied to the reconstructed luminance samples that are converted from the mapped domain to the original domain. Furthermore, a clipping operation, i.e., Clip1(), is applied to clip the inter prediction samples to the dynamic range of the internal bit depth (i.e., within the range of [0, (1<<bitDepth)-1]) after they are converted to the mapped domain.

[0162] Specifically, in the intra mode, CIIP mode, and IBC mode, the reference samples from the adjacent reconstructed domains of one current CU used to generate intra prediction samples are maintained within the mapped domain. In contrast, in the CIIP mode and all inter modes, the motion compensated prediction samples generated from the temporal reference pictures are in the original domain. Since the luminance reconstruction operation is performed in the mapped domain, those inter prediction samples of the luminance component need to be converted to the mapped domain before being added to the residual samples. On the other hand, for both intra and inter modes, the inverse mapping is always applied to the reconstructed luminance samples that are converted from the mapped domain to the original domain. Specifically, in the intra mode, CIIP mode, and IBC mode, the reference samples from the adjacent reconstructed domains of one current CU used to generate intra prediction samples are maintained within the mapped domain. In contrast, in the CIIP mode and all inter modes, the motion compensated prediction samples generated from the temporal reference pictures are in the original domain. Since the luminance reconstruction operation is performed in the mapped domain, those inter prediction samples of the luminance component need to be converted to the mapped domain before being added to the residual samples. On the other hand, for both intra and inter modes, the inverse mapping is always applied to the reconstructed luminance samples that are converted from the mapped domain to the original domain. Specifically, in the intra mode, CIIP mode, and IBC mode, the reference samples from the adjacent reconstructed domains of one current CU used to generate intra prediction samples are maintained within the mapped domain. In contrast, in the CIIP mode and all inter modes, the motion compensated prediction samples generated from the temporal reference pictures are in the original domain. Since the luminance reconstruction operation is performed in the mapped domain, those inter prediction samples of the luminance component need to be converted to the mapped domain before being added to the residual samples. On the other hand, for both intra and inter modes, the inverse mapping is always applied to the reconstructed luminance samples that are converted from the mapped domain to the original domain. Specifically, in the intra mode, CIIP mode, and IBC mode, the reference samples from the adjacent reconstructed domains of one current CU used to generate intra prediction samples are maintained within the mapped domain. In contrast, in the CIIP mode and all inter modes, the motion compensated prediction samples generated from the temporal reference pictures are in the original domain. Since the luminance reconstruction operation is performed in the mapped domain, those inter prediction samples of the luminance component need to be converted to the mapped domain before being added to the residual samples. On the other hand, for both intra and inter modes, the inverse mapping is always applied to the reconstructed luminance samples that are converted from the mapped domain to the original domain. Specifically, in the intra mode, CIIP mode, and IBC mode, the reference samples from the adjacent reconstructed domains of one current CU used to generate intra prediction samples are maintained within the mapped domain. In contrast, in the CIIP mode and all inter modes, the motion compensated prediction samples generated from the temporal reference pictures are in the original domain. Since the luminance reconstruction operation is performed in the mapped domain, those inter prediction samples of the luminance component need to be converted to the mapped domain before being added to the residual samples. On the other hand, for both intra and inter modes, the inverse mapping is always applied to the reconstructed luminance samples that are converted from the mapped domain to the original domain. Specifically, in the intra mode, CIIP mode, and IBC mode, the reference samples from the adjacent reconstructed domains of one current CU used to generate intra prediction samples are maintained within the mapped domain. In contrast, in the CIIP mode and all inter modes, the motion compensated prediction samples generated from the temporal reference pictures are in the original domain. Since the luminance reconstruction operation is performed in the mapped domain, those inter prediction samples of the luminance component need to be converted to the mapped domain before being added to the residual samples. On the other hand, for both intra and inter modes, the inverse mapping is always applied to the reconstructed luminance samples that are converted from the mapped domain to the original domain. Specifically, in the intra mode, CIIP mode, and IBC mode, the reference samples from the adjacent reconstructed domains of one current CU used to generate intra prediction samples are maintained within the mapped domain. In contrast, in the CIIP mode and all inter modes, the motion compensated prediction samples generated from the temporal reference pictures are in the original domain. Since the luminance reconstruction operation is performed in the mapped domain, those inter prediction samples of the luminance component need to be converted to the mapped domain before being added to the residual samples. On the other hand, for both intra and inter modes, the inverse mapping is always applied to the reconstructed luminance samples that are converted from the mapped domain to the original domain. Specifically, in the intra mode, CIIP mode, and IBC mode, the reference samples from the adjacent reconstructed domains of one current CU used to generate intra prediction samples are maintained within the mapped domain. In contrast, in the CIIP mode and all inter modes, the motion compensated prediction samples generated from the temporal reference pictures are in the original domain. Since the luminance reconstruction operation is performed in the mapped domain, those inter prediction samples of the luminance component need to be converted to the mapped domain before being added to the residual samples. On the other hand, for both intra and inter modes, the inverse mapping is always applied to the reconstructed luminance samples that are converted from the mapped domain to the original domain. Specifically, in the intra mode, CIIP mode, and IBC mode, the reference samples from the adjacent reconstructed domains of one current CU used to generate intra prediction samples are maintained within the mapped domain. In contrast, in the CIIP mode and all inter modes, the motion compensated prediction samples generated from the temporal reference pictures are in the original domain. Since the luminance reconstruction operation is performed in the mapped domain, those inter prediction samples of the luminance component need to be converted to the mapped domain before being added to the residual samples. On the other hand, for both intra and inter modes, the inverse mapping is always applied to the reconstructed luminance samples that are converted from the mapped domain to the original domain.

[0163] Furthermore, a clipping operation, i.e., Clip1(), is applied to clip the inter prediction samples to the dynamic range of the internal bit depth (i.e., within the range of [0, (1<<bitDepth)-1]) after they are converted to the mapped domain. Furthermore, a clipping operation, i.e., Clip1(), is applied to clip the inter prediction samples to the dynamic range of the internal bit depth (i.e., within the range of [0, (1<<bitDepth)-1]) after they are converted to the mapped domain. Furthermore, a clipping operation, i.e., Clip1(), is applied to clip the inter prediction samples to the dynamic range of the internal bit depth (i.e., within the range of [0, (1<<bitDepth)-1]) after they are converted to the mapped domain. is used. On the other hand, for both intra and inter modes, after the reconstructed luminance sample pull is converted to the original domain, the same clipping operation is applied to the reconstructed luminance sample as well.

[0164] However, based on the existing LMCS design, there is a bitstream control to ensure that the samples obtained from the forward LMCS are always within the dynamic range of the internal bit depth. This means that the mapped luminance prediction samples of the inter CU cannot exceed such a dynamic range. Therefore, the existing clipping operation applied to the mapped luminance prediction samples in the inter mode is redundant. As an example, it can be proposed to remove the clipping operation after the forward conversion of the inter prediction samples in the inter mode and the CIIP mode. In another example, it can be proposed to remove the clipping operation from the inverse LMCS mapping process when converting the reconstructed luminance sample from the mapped domain to the original domain. More specifically, in the current VVC draft, the inverse mapping process of the luminance sample is described as follows, and the clipping operation Clip 1() in Equation (1242) is applied as shown in the prominent font size. 8.8.2.2 Inverse Mapping Process of Luminance Sample The input to this process is the luminance sample lumaSample. The output of this process is the modified luminance sample invLumaSample.

[0165] The value of invLumaSample is derived as follows. - slice_lmcs_enab of the slice containing the luminance sample lumaSample is applied as shown in the prominent font size. 8.8.2.2 Inverse Mapping Process of Luminance Sample The input to this process is the luminance sample lumaSample. The output of this process is the modified luminance sample invLumaSample. The value of invLumaSample is derived as follows. - slice_lmcs_enab of the slice containing the luminance sample lumaSample If led_flag is equal to 1, the following ordered steps are applied: 1. The variable idxYInv takes lumaSample as input and outputs idxYInv. As input, the piecewise function index of the luminance samples, as specified in clause 8.8.2.3, is used. The ID of the . 2. The variable invSample is derived as follows:

number

number

[0166] In addition, the current VVC draft also includes weighted sampling for joint merging and intra prediction. The pull prediction process is explained as follows, and is shown here in prominent font size: As you can see, a clipping operation Clip 1() has been applied to it. 8.5.6.7 Weighted Sample Prediction Processing for Join Merge and Intra Prediction The inputs to this process are: - for the top-left luma sample of the current picture, Luminance position (xCb, yCb) to specify the pull - the width of the current coding block, cbWidth, - The height of the current coding block, cbHeight - Two (cbWidth) x (cbHeight) arrays predSamplesIn ter and predSamplesIntra - The variable cIdx that specifies the color component index The output of this process is a (cbWidth)×(cbHeight) array of predicted sample values predSamplesComb The variable scallFact is derived as follows

Equation

Number

Number

[0167] Furthermore, in the current VVC draft, the picture reconstruction using the mapping process of the luminance samples is described as follows, and the clipping operation Clip1() is applied as shown in prominent fonts. 8.7.5.2 Picture Reconstruction by Mapping Process of Luminance Samples The input to this process is as follows. - The position (xCurr, yCurr) of the top-left sample of the current block with respect to the top-left sample of the current picture - The variable nCurrSw that defines the block width​​​ - Variable nCurrSh that specifies the block height - (nCurrSw) x (nCurrSh) array predSamples that specifies the luminance prediction samples of the current block h) - (nCurrSw) x (nCurrSh) array resSamples that specifies the luminance residual samples of the current block h) The output of this process is the reconstructed luminance picture sample array recSamples There is. The (nCurrSw ) × (nCurrSh) array of mapped predicted luminance samples predMapSamples is derived as follows. - If any of the following conditions is true, predMapSamples[i][j is set equal to pred Samples[i][j] for i = 0..nCurrSw-1, j = 0..nCurrSh-1. - CuPredMode[0][xCurr][yCurr] is MODE_INTRA is equal to. - CuPredMode[0][xCurr][yCurr] is equal to MODE_IBC is equal to. - CuPredMode[0][xCurr][yCurr] is equal to MODE_PLT is equal to. - CuPredMode[0][xCurr][yCurr] is equal to MODE_INTER and ciip_flag[xCurr][yCurr] is equal to 1. - Otherwise (CuPredMode[0][xCurr][yCurr] is equal to MO DE_INTER and ciip_flag[xCurr][yCurr] is equal to 0 is true), the following applies.

Equation

number

[0168] These redundant clipping operations increase the computational complexity and overhead of existing LMCS designs. This leads to extra on-chip memory requirements. This further reduces the LMCS design complexity and memory requirements. To reduce this, it is proposed to remove these redundant clipping operations.

[0169] According to the non-clipping chroma sample decoding procedure, the chroma scaling is performed as shown in FIG. Under the Luminance Mapping with Coding (LMCS) framework, inter-mode or combined Coding units (CUs) coded using inter-intra prediction (CIIP) mode During the decoding of, the reconstructed samples of the luma component are (2001), and multiple reconstructed samples of the luminance component are mapped to the domain. By transforming from the input to the original domain, multiple transformed samples of the luminance component are (2002) and the chroma scale for decoding the chroma samples of the CU is In deriving the ring coefficients, multiple transformed luminance samples in the original domain are used. but used in, without clipping (2003).

[0170] In one or more embodiments of the non-clipping chroma sample decoding procedure, the As such, when the CU is encoded in inter mode, the step of obtaining a plurality of reconstructed samples of the luminance component in the mapped domain includes the step (2101) of calculating a plurality of inter prediction samples of the luminance component in the original domain and the step of obtaining, without clipping, a plurality of converted inter prediction samples of the luminance component in the mapped domain by converting a plurality of inter prediction samples of the luminance component from the original domain to the mapped domain (2102), and the step of adding a plurality of converted inter prediction samples of the luminance component in the mapped domain to a plurality of residual samples of the luminance component in the mapped domain, resulting in obtaining a plurality of reconstructed samples of the luminance component in the mapped domain (2103). In one or more other embodiments of the non-clipping chroma sample decoding procedure, as shown in FIG. 22, when the CU is encoded in CIIP mode, the step of obtaining a plurality of reconstructed samples of the luminance component in the mapped domain includes the step

[0171] (2201) of calculating a plurality of inter prediction samples of the luminance component in the original domain and the step of obtaining, without clipping, a plurality of converted inter prediction samples of the luminance component in the mapped domain by converting a plurality of inter prediction samples of the luminance component from the original domain to the mapped domain (2202), the step (2203) of calculating a plurality of intra prediction samples of the luminance component in the mapped domain, and the addition of a plurality of converted inter prediction samples and a plurality of intra prediction samples, whereby a plurality of reconstructed samples of the luminance component in the mapped domain are obtained (2204). ​​​​Derive a predicted sample of the luminance component in the mapped domain by weighted average step (2204), and add the derived predicted sample of the luminance component in the mapped domain to a plurality of residual samples of the luminance component in the mapped domain, resulting in a plurality of reconstructed samples of the luminance component in the mapped domain, step (2 205), including

[0172] When the current LMCS is enabled, the forward and inverse luminance mappings are performed using one LUT table defined with an accuracy of 11 bits. For example, taking the forward luminance ma pping as an example, the current forward mapping scaling coefficient is defined as follows is defined. is defined. ScaleCoeff[i]=(lmcsCW[i]*(1<<11)+(1<<(L og 2(OrgCW)-1)))>>(Log 2(OrgCW)) where lmcsCW is the length of one segment of the mapped luminance domain and OrgCW is the length of one segment of the original luminance domain equal to 1<<(BitDepth-4) is the length of one segment of the original luminance domain

[0173] However, such an accuracy of 11 bits has been found to be sufficient only for internal coding bit depths less than 16 bits. When the internal coding bit depth is 16 bits, the value of L og 2(OrgCW) becomes 12. In such a case, the improvement in 11-bit accuracy is not sufficient to support the scaling coefficient derivation. Thus, when the current bitstream conformance is applied, i.e., the segments in the mapped luminance domain when the current bitstream conformance is applied, i.e., the segments in the mapped luminance domain Even when the total length of the mentos is less than or equal to (1<<BitDepth)-1, the forward luminance The predicted luminance sample value after mapping may exceed the dynamic range of the internal coding bit depth.

[0174] Based on such considerations, two solutions are proposed.

[0175] One solution is to always apply a clipping operation to the mapped luminance predictions in the inter mode and CIIP mode. Moreover, the current bit stream compatibility can be removed.

[0176] Under this solution, the weighted sample prediction processing for merge and intra prediction is as follows. Compared with the specification of the same procedure in the current VVC draft, the clipping operation Clip 1 () is always applied to Equation (1028 a) so that the mapped luminance prediction sample includes the clipping value. 8.5.6.7 Weighted sample prediction processing for merge and intra prediction The input to this process is as follows. When cIdx is equal to 0 and slice_lmcs_enabled_flag is equal to 1, predSamplesInter[x][y] with x = 0..cbWidth-1 and y = 0..cbHeight-1 is modified as follows.

Number

Number

[0177] Even under this solution, picture reconstruction using the mapping process of the luminance samples is as follows. Compared with the specification of the same procedure in the current VVC draft, a clipping operation Clip1() is added so that the mapped luminance prediction samples include the clipping values. 8.7.5.2 Picture Reconstruction by Luminance Sample Mapping Process The input to this process is as follows. - The position (xCurr, yCurr) of the top-left sample of the current block relative to the top-left sample of the current picture - The variable nCurrSw that defines the block width - The variable nCurrSh that specifies the block height - The (nCurrSw) x (nCurrSh) array predSamples that specifies the luminance prediction samples of the current block - The (nCurrSw) x (nCurrSh) array resSamples that specifies the luminance residual samples of the current block The output of this process is the reconstructed luminance picture sample array recSamples. The (nCurrSw) × (nCurrSh) array of the mapped predicted luminance samples predMapSamples is derived as follows. - If any of the following conditions is true, predMapSamples[i][j] is set equal to predSamples[i][j] for i = 0..nCurrSw - 1, j = 0..nCurrSh - 1. - CuPredMode[0][xCurr][yCurr] is equal to MODE_INTRA. ​​​​​​​​​​​-CuPredMode[0][xCurr][yCurr] is equal to MODE_IBC and equal. -CuPredMode[0][xCurr][yCurr] is equal to MODE_PLT and equal. -CuPredMode[0][xCurr][yCurr] is equal to MODE_INTER and ciip_flag[xCurr][yCurr] is equal to 1. - Otherwise (CuPredMode[0][xCurr][yCurr] is equal to MO DE_INTER and ciip_flag[xCurr][yCurr] is equal to 0) the following applies.

Number

Number

[0178] In a second solution, it is proposed to increase the accuracy of the derivation of the LMCS scaling factors beyond 11 bits (M bits and notation).

[0179] Under this second solution, the forward luminance mapping is as follows. ScaleCoeff[i]=(lmcsCW[i]*(1<<M)+(1<<(Lo g 2(OrgCW)-1)))>>(Log 2(OrgCW)) idxY=predSamples[i][j]>>Log2(OrgCW) ​predMapSamples[i][j]=LmcsPivot[idxY]+ (ScaleCoeff[idxY]*(predSamples[i][j]-In putPivot[idxY])+(1<<(M-1)))>>M(1219) with i=0..nCurrSw-1,j=0..nCurrSh-1

[0180] In one embodiment of the second solution, scaling of both forward and inverse luminance mapping is proposed to improve the accuracy of coefficient derivation. In another embodiment of the second solution, only the accuracy of scaling coefficient derivation of forward luminance mapping is proposed to be improved.

[0181] When either of the above two solutions is applied, the current clipping operation applied to the mapped luminance prediction samples in the inter mode and the CIIP mode can also be safely removed.

[0182] According to the first aspect of the present disclosure, as shown in FIG. 23, a plurality of prediction samples in the mapped domain of the luminance component of a CU encoded in the inter mode or the CIIP mode under the LMCS framework are obtained (2301), a plurality of residual samples in the mapped domain of the luminance component of the CU are received from the bitstream (2302), the plurality of prediction samples in the mapped domain are added to the plurality of residual samples in the mapped domain, and as a result, a plurality of reconstruction samples in the mapped domain of the luminance component are obtained (2303), and the plurality of reconstruction samples of the luminance component are based on a plurality of predefined inverse mapping scaling coefficients (2304). ​​​​​​is converted from the mapped domain to the original domain (2304).

[0183] In one or more embodiments of the first aspect of the present disclosure, as illustrated in FIG. 24, C U is encoded by an inter mode, and obtaining a plurality of prediction samples in the mapped domain of the luminance component of CU means deriving a plurality of inter prediction samples in the original domain of the luminance component of CU from the temporal reference picture of CU (2401) and then, based on a predefined encoding bit depth and a plurality of predefined forward mapping scaling factors within a predefined forward mapping accuracy, converting a plurality of inter prediction samples of the luminance component from the original domain to the mapped domain (2402).

[0184] In one or more other embodiments of the first aspect of the present disclosure, as illustrated in FIG. 25 CU is encoded by a CIIP mode, and obtaining a plurality of prediction samples in the mapped domain of the luminance component of CU means deriving a plurality of inter prediction samples in the original domain of the luminance component of CU from the temporal reference picture of CU (250 1), and based on a predefined encoding bit depth and a plurality of predefined forward mapping scaling factors within a predefined forward mapping accuracy converting a plurality of inter prediction samples of the luminance component from the original domain to the mapped domain (2502), calculating a plurality of intra prediction samples in the mapped domain of the luminance component of CU (2503), and ​​​​​​​​​Deriving, as a weighted average of a pull and a plurality of intra prediction samples, a prediction sample of the luminance component of the CU in the mapped domain (2504). includes.

[0185] In one example, as shown in FIG. 26, based on a predefined coding bit depth and a plurality of predefined forward mapping scaling factors within a predefined forward mapping accuracy, converting a plurality of inter prediction samples of the luminance component from the original domain to the mapped domain is to convert a plurality of inter prediction samples of the luminance component from the original domain to the mapped domain using the predefined plurality of forward mapping scaling factors (2601), and based on the predefined coding bit depth and the predefined forward mapping accuracy, determining whether a clipping operation is required (2602), and in response to the determination that a clipping operation is required, clipping a plurality of inter prediction samples of the luminance component in the mapped domain to the predefined coding bit depth (2603), and in response to the determination that a clipping operation is not required, bypassing the clipping of the plurality of inter prediction samples of the luminance component (2604). predefined plurality of forward mapping scaling factors from the original domain to the mapped domain. converting a plurality of inter prediction samples of the luminance component from the original domain to the mapped domain using the predefined plurality of forward mapping scaling factors (2601), and based on the predefined coding bit depth and the predefined forward mapping accuracy, determining whether a clipping operation is required (2602), and in response to the determination that a clipping operation is required clipping a plurality of inter prediction samples of the luminance component in the mapped domain to the predefined coding bit depth (2603), and in response to the determination that a clipping operation is not required bypassing the clipping of a plurality of inter prediction samples of the luminance component (2604).

[0186] In one or more cases, determining whether a clipping operation is required includes determining that a clipping operation is required when the predefined coding bit depth is greater than the predefined forward mapping accuracy.

[0187] In one or more cases, determining whether a clipping operation is required ​includes determining that a clipping operation is not necessary if a pre - defined encoded bit depth is less than a pre - defined forward mapping accuracy.

[0188] In one or more examples, determining whether a clipping operation is necessary regardless of a pre - defined encoded bit depth and a pre - defined forward mapping accuracy includes determining that a clipping operation is necessary.

[0189] In one or more examples, determining whether a clipping operation is necessary regardless of a pre - defined encoded bit depth and a pre - defined forward mapping accuracy includes determining that a clipping operation is not necessary.

[0190] In one or more examples, the pre - defined forward mapping accuracy is 15 bits.

[0191] In one or more examples, the pre - defined forward mapping accuracy is 11 bits.

[0192] FIG. 27 illustrates a block diagram of an apparatus for video encoding according to some implementations of the present disclosure. Apparatus 2700 may be a terminal such as a mobile phone, a tablet computer, a digital transmission terminal, a tablet device, or a personal digital assistant.

[0193] As shown in FIG. 27, apparatus 2700 may include one or more of the following components: processing component 2702, memory 2704, power supply component 2706, multimedia component 2708 , audio component 2710, input / output (I / O) interface 2712, sensor component 2714, and communication component 2716.

[0194] The processing component 2702 generally controls the overall operations of the device 2700, such as operations related to display, call, data communication, camera operation, and recording operation. The processing component 2702 can include one or more processors 2720 to execute instructions for completing all or part of the steps of the above method. Further, the processing component 2702 can include one or more modules to facilitate the interaction between the processing component 2702 and other components. For example, the processing component 2702 can include a multimedia module to facilitate the interaction between the multimedia component 2708 and the processing component 2702.

[0195] The memory 2704 is configured to store different types of data to support the operations of the device 2700. Examples of such data include instructions for any application or method operating on the device 2700, contact data, phone book data, messages, photos, videos, etc. The memory 2704 can include any type of temporary or non - temporary storage medium or a combination thereof, and the memory 2704 can be a static random access memory (SRAM), an electrically erasable programmable read - only memory (EEPROM), an erasable programmable read - only memory (EPROM), a programmable read - only memory (PROM), a read - only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or a compact disk. The non - temporary storage medium can be, for example, a hard disk drive (HDD), a solid - state drive (SSD), a flash memory, a hybrid drive, or a solid - state hybrid (EEPROM), an erasable programmable read - only memory (EPROM), a programmable read - only memory (PROM), a read - only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or a compact disk. The non - temporary storage medium can be, for example, a hard disk drive (HDD), a solid - state drive (SSD), a flash memory, a hybrid drive, or a solid - state hybrid (SSD), a flash memory, a hybrid drive, or a solid - state hybrid drive. drive. Solid State Hybrid Drive (SSHD), read-only memory (ROM), compact disc read-only memory (CD-ROM), magnetic tape, floppy (registered trademark) disk, etc. may also be used.

[0196] The power component 2706 supplies power to different components of the device 2700. The power component 2706 may include a power management system, one or more power supplies, and other components related to the generation, management, and distribution of power for the device 2700.

[0197] The multimedia component 2708 includes a screen that provides an output interface between the device 2700 and the user. In some examples, the screen may include a liquid crystal display (LCD) and a touch panel (TP). When the screen includes a touch panel, the screen may be implemented as a touch screen that receives input signals from the user. The touch panel may include one or more touch sensors for detecting touches, swipes, and gestures on the touch panel. The touch sensors can not only sense the boundaries of touch or swipe operations but also detect the duration and pressure associated with the touch or swipe operations. In some examples, the multimedia component 2708 may include a front camera and / or a rear camera. When the device 2700 is in an operation mode such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data.

[0198] The audio component 2710 is configured to output and / or input audio signals. ​It is composed of. For example, the audio component 2710 includes a microphone (MIC). When the device 2700 is in an operating mode such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal may be further stored in the memory 2704 or transmitted via the communication component 27 16. In some examples, the audio component 2710 further includes a speaker for outputting an audio signal.

[0199] The I / O interface 2712 provides an interface between the processing component 2702 and the peripheral interface module . The above peripheral interface module may be a keyboard, a click wheel, buttons, etc. These buttons can include, but are not limited to, home buttons, volume buttons, start buttons, and lock buttons.

[0200] The sensor component 2714 includes one or more sensors for providing state evaluation in different aspects of the device 2700. For example, the sensor component 2714 can detect the on / off state of the device 2700 and the relative position of components. For example, the components are the display and keypad of the device 2700. The sensor component 2714 can also detect position changes of the device 2700 or components of the device 2700, the presence or absence of user contact on the device 2700, the orientation or acceleration / deceleration of the device 2700, and temperature changes of the device 2700. The sensor component 2714 can include a proximity sensor configured to detect the presence of nearby objects without physical contact. The sensor component 2714 can include an imaging component.​​​​​​​​​ further includes an optical sensor such as a CMOS or CCD image sensor for use in applications In some examples, the sensor component 2714 can further include an acceleration sensor, gyro scope sensor, magnetic sensor, pressure sensor, or temperature sensor.

[0201] The communication component 2716 is configured to facilitate wired or wireless communication between the device 2700 and other devices. The device 2700 can access a wireless network based on communication standards such as WiFi, 4G, or a combination thereof. In one example, the communication component 2716 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one example, the communication component 2716 may further include a Near Field Communication (NFC) module for facilitating short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra-Wideband (UWB) technology, Bluetooth (R) (BT) technology, and other technologies. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra-Wideband (UWB) technology, Bluetooth (R) (BT) technology, and other technologies. be implemented based on.

[0202] In one example, the device 2700 may be implemented by one or more of an Application-Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic elements that execute the above methods. In one example, the device 2700 may be implemented by one or more of an Application-Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic elements that execute the above methods. (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic elements that execute the above methods. In one or more examples, the described functions may be implemented in hardware, software, firmware or a combination of one or more of the above electronic elements.

[0203] In one or more examples, the described functions may be implemented in hardware, software, firmware It can be implemented in software, or any combination thereof. In the case of software implementation, the function can be stored in a computer-readable medium or transmitted via a computer-readable medium as one or more instructions or codes and executed by a hardware-based processing unit. The computer-readable medium can include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium, or a communication medium including any medium that facilitates the transfer of a computer program from one place to another according to, for example, a communication protocol. In this way, the computer-readable medium can generally correspond to (1) a tangible computer-readable storage medium that is non-transitory, or (2) a communication medium such as a signal or a carrier wave. The data storage medium can be accessed by one or more computers or one or more processors to retrieve the instructions, codes, and / or data structures for implementing the implementation forms described in this application. Any available medium may be used. The computer program product can include a computer-readable medium. Furthermore, the above method can be implemented using the apparatus 2700. The present disclosure can include dedicated hardware implementations such as application-specific integrated circuits, programmable logic arrays, and other hardware devices. The hardware implementation can be configured to implement one or more of the methods described herein. Examples that can include various implementations of devices and systems can broadly include various electronic and computing systems. One or more of the examples described herein are for inter-module transfer. It can be implemented in software, or any combination thereof. In the case of software implementation, the function can be stored in a computer-readable medium or transmitted via a computer-readable medium as one or more instructions or codes and executed by a hardware-based processing unit. The computer-readable medium can include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium, or a communication medium including any medium that facilitates the transfer of a computer program from one place to another according to, for example, a communication protocol. In this way, the computer-readable medium can generally correspond to (1) a tangible computer-readable storage medium that is non-transitory, or (2) a communication medium such as a signal or a carrier wave. The data storage medium can be accessed by one or more computers or one or more processors to retrieve the instructions, codes, and / or data structures for implementing the implementation forms described in this application. Any available medium may be used. The computer program product can include a computer-readable medium. Furthermore, the above method can be implemented using the apparatus 2700. The present disclosure can include dedicated hardware implementations such as application-specific integrated circuits, programmable logic arrays, and other hardware devices. The hardware implementation can be configured to implement one or more of the methods described herein. Examples that can include various implementations of devices and systems can broadly include various electronic and computing systems. One or more of the examples described herein are for inter-module transfer. It can be implemented in software, or any combination thereof. In the case of software implementation, the function can be stored in a computer-readable medium or transmitted via a computer-readable medium as one or more instructions or

[0204] codes and executed by a hardware-based processing unit. The computer-readable medium can include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium, or a communication medium including any medium that facilitates the transfer of a computer program from one place to another according to, for example, a communication protocol. In this way, the computer-readable medium can generally correspond to (1) a tangible computer-readable storage medium that is non-transitory, or (2) a communication medium such as a signal or a carrier wave. The data storage medium can be accessed by one or more computers or one or more processors to retrieve the instructions, codes, and / or data structures for implementing the implementation forms described in this application. Any available medium may be used. The computer program product can include a computer-readable medium. Furthermore, the above method can be implemented using the apparatus 2700. The present disclosure can include dedicated hardware implementations such as application-specific integrated circuits, programmable logic arrays, and other hardware devices. The hardware implementation can be configured to implement one or more of the methods described herein. Examples that can include various implementations of devices and systems can broadly include various electronic and computing systems. One or more of the examples described herein are for inter-module transfer. transfer. and communicate via modules or as part of an application specific integrated circuit, two or more specific interconnected hardware modules or devices having related control and data signals modules or devices can be used to implement functions. Thus, the disclosed apparatus or system can include software, firmware, and hardware implementations. The terms "module", "sub-module", "circuit", "sub-circuit" "circuitry", "sub-circuitry", "unit", or "sub-unit" can include a memory (shared, dedicated, or grouped) that stores code or instructions executable by one or more processors. The modules referred to herein can include one or more circuits, whether or not they contain stored code or instructions. A module or circuit can include one or more connected components. "module", "sub-module", "circuit", "sub-circuit", "circuitry", "sub-circuitry", "unit", or "sub-unit" can include a memory (shared, dedicated, or grouped) that stores code or instructions executable by one or more processors. The modules referred to herein can include one or more circuits, whether or not they contain stored code or instructions. A module or circuit can include one or more connected components. "circuitry", "sub-circuitry", "unit", or "sub-unit" can include a memory (shared, dedicated, or grouped) that stores code or instructions executable by one or more processors. The modules referred to herein can include one or more circuits, whether or not they contain stored code or instructions. A module or circuit can include one or more connected components. A memory (shared, dedicated, or grouped) that stores code or instructions executable by one or more processors can be included. The modules referred to herein can include one or more circuits, whether or not they contain stored code or instructions. A module or circuit can include one or more connected components. A memory (shared, dedicated, or grouped) that stores code or instructions executable by one or more processors can be included. The modules referred to herein can include one or more circuits, whether or not they contain stored code or instructions. A module or circuit can include one or more connected components. A memory (shared, dedicated, or grouped) that stores code or instructions executable by one or more processors can be included. The modules referred to herein can include one or more circuits, whether or not they contain stored code or instructions. A module or circuit can include one or more connected components. A memory (shared, dedicated, or grouped) that stores code or instructions executable by one or more processors can be included. The modules referred to herein can include one or more circuits, whether or not they contain stored code or instructions. A module or circuit can include one or more connected components. A memory (shared, dedicated, or grouped) that stores code or instructions executable by one or more processors can be included. The modules referred to herein can include one or more circuits, whether or not they contain stored code or instructions. A module or circuit can include one or more connected components.

[0205] Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice in the art to which the invention pertains. The specification and examples are to be considered as illustrative only, and the true scope and spirit of the invention are indicated by the following claims. Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice in the art to which the invention pertains. The specification and examples are to be considered as illustrative only, and the true scope and spirit of the invention are indicated by the following claims. Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice in the art to which the invention pertains. The specification and examples are to be considered as illustrative only, and the true scope and spirit of the invention are indicated by the following claims. Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice in the art to which the invention pertains. The specification and examples are to be considered as illustrative only, and the true scope and spirit of the invention are indicated by the following claims. Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice in the art to which the invention pertains. The specification and examples are to be considered as illustrative only, and the true scope and spirit of the invention are indicated by the following claims. Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice in the art to which the invention pertains. The specification and examples are to be considered as illustrative only, and the true scope and spirit of the invention are indicated by the following claims.

[0206] The present invention is not limited to the exact examples described above and shown in the accompanying drawings, and it will be understood that various modifications and changes can be made without departing from its scope. The scope of the present invention is intended to be limited only by the appended claims. ​

Claims

1. Dividing a current picture into one or more coding units (CUs); Obtaining a plurality of prediction samples in a mapped domain of a luminance component of a current CU encoded in an inter mode under a framework of luminance mapping with chroma scaling (LMCS); Obtaining a plurality of residual samples in the mapped domain of the luminance component of the current CU; Adding the plurality of prediction samples in the mapped domain to the plurality of residual samples in the mapped domain to obtain a plurality of reconstructed samples in the mapped domain of the luminance component of the current CU; Converting the plurality of reconstructed samples of the luminance component from the mapped domain to the original domain based on a plurality of predefined inverse mapping scaling coefficients; Obtaining prediction information of the current CU based on the plurality of reconstructed samples in the original domain to form a bitstream; comprising; Obtaining the plurality of prediction samples in the mapped domain of the luminance component of the current CU comprises: Deriving a plurality of inter prediction samples of the luminance component of the current CU from a temporal reference picture of the current CU in the original domain; Converting the plurality of inter prediction samples of the luminance component from the original domain to the mapped domain based on a predefined coding bit depth and a plurality of predefined forward mapping scaling coefficients within a predefined forward mapping accuracy; comprising; A method for video coding.

2. Converting the plurality of inter prediction samples of the luminance component from the original domain to the mapped domain based on the predefined coding bit depth and the plurality of predefined forward mapping scaling coefficients within the predefined forward mapping accuracy comprises: Converting the plurality of inter prediction samples of the luminance component from the original domain to the mapped domain using the plurality of predefined forward mapping scaling coefficients without performing a clipping operation; The method according to claim 1, comprising.

3. The method according to claim 1 or 2, wherein the predefined forward mapping accuracy is 11 bits.

4. A computing device, comprising: one or more processors; a non-transitory storage device coupled to the one or more processors; a plurality of programs stored in the non-transitory storage device, which, when executed by the one or more processors, cause the computing device to execute the method according to any one of claims 1 to 3; A computing device comprising the above.

5. A computer program stored on a computer-readable medium that enables a computer to execute the steps of the method according to any one of claims 1 to 3 and obtain a corresponding bitstream.

6. A method of storing a bitstream, comprising: generating a bitstream; storing the bitstream; wherein the encoding method comprises: dividing a current picture into one or more coding units (CUs); obtaining a plurality of prediction samples in a mapped domain of a luminance component of the current CU encoded in inter mode under a framework of luminance mapping with chroma scaling (LMCS); obtaining a plurality of residual samples in the mapped domain of the luminance component of the current CU; adding the plurality of prediction samples in the mapped domain to the plurality of residual samples in the mapped domain to obtain a plurality of reconstructed samples in the mapped domain of the luminance component of the current CU; converting the plurality of reconstructed samples of the luminance component from the mapped domain to the original domain based on a plurality of predefined inverse mapping scaling coefficients; obtaining prediction information of the current CU based on the plurality of reconstructed samples in the original domain to generate a bitstream; Obtaining the plurality of prediction samples in the mapped domain of the luminance component of the current CU comprises: Deriving a plurality of inter-prediction samples of the luminance component of the current CU in the original domain from the time-reference picture of the current CU; Converting the plurality of inter-prediction samples of the luminance component from the original domain to the mapped domain based on a predefined coding bit depth and a plurality of predefined forward mapping scaling factors within a predefined forward mapping accuracy; comprising; A method for storing a bitstream.

Images