Apparatus, method and computer program for determining cross-component parameters

Abstract

11. A method comprising: determining a reference value for a parameter of a mapping function from a first color component to a second color component; decoding a magnitude of an initial update term; decoding a syntax element to be used in determining a sign of the update term; determining the sign of the update term by interpreting the decoded syntax element based on the reference value; and determining a value of the parameter of the mapping function using the reference value, the decoded magnitude of the update term, and the determined sign of the update term.

Description

[Technical field]

[0001] The present invention relates to an apparatus, a method and a computer program for cross-component parameter determination in video encoding and decoding. [Background technology]

[0002] In video coding, video and image samples are generally encoded using color representations such as YUV or YCbCr, which consist of one luminance (luma) channel and two chrominance (chroma) channels. In these cases, the luminance channel, which primarily represents the illumination of the scene, is generally coded at a particular resolution, while the chrominance channels, which generally represent the differences between particular color components, are coded at a second resolution, often lower than the resolution of the luminance signal. The intent of this kind of differential representation is to decorrelate the color components, allowing the data to be compressed more efficiently.

[0003] In the Versatile Video Coding (VVC / H.266) standard, a cross-component linear model (CCLM) is used as a linear model to predict samples in chroma channels (e.g., Cb and Cr). This process produces a linear model that can be used to map luma sample values ​​to chroma sample values. The parameters of the linear model are constructed using the available reconstructed luma and chroma reference samples outside the boundaries of the prediction block. Once the parameters are constructed, the linear model specified by those parameters is used to predict the chroma sample values ​​inside the prediction block.

[0004] Applying cross-component linear model prediction to predict chroma samples can be efficient compared to intra-component spatial prediction, since the texture of the reconstructed luma block can sometimes produce a very good predictor for the chroma texture when there is a strong linear correlation between the luma and chroma channels. However, the reconstructed reference samples used to generate the linear model parameters may be noisy or may not adequately represent the content inside the actual predicted block. In these cases, the prediction fails and the process results in suboptimal coding efficiency for the content. Summary of the Invention [Means for solving the problem]

[0005] Next, to at least alleviate the above-mentioned problems, an enhanced method for achieving better correlation between the luma and chroma channels is introduced herein.

[0006] According to some aspects, a method is provided in which a reference value for a parameter used in a luma to chroma mapping model is generated and an update term to be applied to refine the reference value is determined. Determining the value of the update term may include inverting the sign of a displayed value of the update term depending on the sign or magnitude of the generated reference value.

[0007] It should be noted that some of the aspects may be applied to a mapping model from a first color component to a second color component, e.g., the first color component may be a luma component and the second color component may be a chroma component, or the first color component may be a chroma component and the second color component may be a luma component. Some of the aspects may also be applied to mapping models that operate between different types of data. For example, some aspects may be applied to a mapping that converts an image sample value into a depth map value that indicates the distance of the sample from a particular point or plane.

[0008] The device according to the first aspect comprises: determining reference values ​​of parameters of a mapping function from the first color component to the second color component; Decode the magnitude of the initial update term; Decode a syntax element to be used in determining the sign of the update term; determining the sign of the update term by interpreting the decoded syntax element based on the reference value; Determine values ​​for the parameters of the mapping function using the reference value, the decoded magnitude of the update term, and the determined sign of the update term. The measures are:

[0009] According to one embodiment, the apparatus comprises: Comparing the reference value to a threshold value; If the reference value is greater than the threshold and the syntax element has a first representation value, or if the reference value is less than or equal to the threshold and the syntax element has a second representation value different from the first representation value, determine the sign of the update term to be positive, or otherwise determine the sign of the update term to be negative. The device further comprises means for:

[0010] According to one embodiment, the first display value is 0 and the second display value is 1, or the first display value is 1 and the second display value is 0.

[0011] According to one embodiment, the threshold is zero.

[0012] According to one embodiment of the apparatus, the means for determining a sign comprises means for determining a sign such that the update term refines the reference value towards zero when a decoded binary syntax element representing the sign of the update term is 0, and refines the reference value away from zero when the decoded binary syntax element representing the sign of the update term is 1.

[0013] According to one embodiment of the apparatus, the means for determining a sign comprises means for determining a sign such that the update term refines the reference value towards zero when a decoded binary syntax element representing the sign of the update term is 1, and refines the reference value away from zero when the decoded binary syntax element representing the sign of the update term is 0.

[0014] According to one embodiment, the device comprises means for determining the value of the update term by inverting the sign of the initial update term when the value of the reference parameter is greater than or equal to zero.

[0015] According to one embodiment, the device comprises means for determining the value of the update term by leaving the initial update term unchanged if the value of the reference parameter is less than or equal to zero.

[0016] According to one embodiment of the apparatus, the first color component is a luma component and the second color component is one chroma component.

[0017] The method according to the second aspect comprises the steps of: determining reference values ​​of parameters of a mapping function from the first color component to the second color component; Decoding the magnitude of the initial update term; Decoding syntax elements to be used in determining the signs of the update terms; determining the sign of the update term by interpreting the decoded syntax element based on the reference value; determining values ​​of parameters of a mapping function using the reference value, the decoded magnitude of the update term, and the determined sign of the update term; Includes.

[0018] An apparatus according to a third aspect comprises at least one processor and at least one memory, the at least one memory having stored therein code that, when executed by the at least one processor, causes the apparatus to perform at least: determining reference values ​​of parameters of a mapping function from the first color component to the second color component; Decoding the magnitude of the initial update term; Decoding syntax elements to be used in determining the signs of the update terms; determining the sign of the update term by interpreting the decoded syntax element based on the reference value; determining values ​​of parameters of a mapping function using the reference value, the decoded magnitude of the update term, and the determined sign of the update term; Have them carry out the procedure.

[0019] An apparatus according to a fourth aspect comprises: determining reference values ​​of parameters of a mapping function from the first color component to the second color component; determining a value for an initial update term of the reference value; if the reference value is greater than the threshold, determining a displayed value of the update term such that the sign of the displayed value is opposite to the sign of the determined value of the update term; and encoding the sign and magnitude of the represented value of the update term into a video bitstream.

[0020] According to one embodiment, the device comprises means for inverting the sign of the update term when the value of the reference parameter is greater than or equal to zero.

[0021] According to one embodiment, the apparatus comprises means for determining the displayed value of the update term by leaving the update term unchanged if the value of the reference parameter is less than or equal to zero.

[0022] According to one embodiment, the apparatus comprises: performing a rate-distortion decision to determine values ​​of the update terms; Means are provided for determining the displayed update terms based on the values ​​of the update terms.

[0023] According to one embodiment, the apparatus comprises: performing a rate-distortion determination to determine the value of the indicated update term; Means are provided for determining the update term based on the represented value of the update term.

[0024] The method according to the fifth aspect comprises the steps of: determining reference values ​​of parameters of a mapping function from the first color component to the second color component; determining a value of an initial update term for the reference value; if the reference value is greater than the threshold, determining a displayed value of the update term such that the sign of the displayed value is opposite to the sign of the determined value of the update term; encoding the sign and magnitude of the represented value of the update term into the video bitstream; Includes.

[0025] An apparatus according to a sixth aspect comprises at least one processor and at least one memory, the at least one memory having stored therein code that, when executed by the at least one processor, causes the apparatus to perform at least: determining reference values ​​of parameters of a mapping function from the first color component to the second color component; determining a value of an initial update term for the reference value; if the reference value is greater than the threshold, determining a displayed value of the update term such that the sign of the displayed value is opposite to the sign of the determined value of the update term; encoding the sign and magnitude of the represented value of the update term into the video bitstream; Have them carry out the procedure.

[0026] Thus, the apparatus and the computer readable storage medium having code stored thereon are configured to perform one or more of the above-mentioned methods and their associated embodiments, as described above.

[0027] For a better understanding of the present invention, reference will now be made, by way of example, to the accompanying drawings in which: [Brief description of the drawings]

[0028] [Figure 1] 1 is a schematic diagram of an electronic device utilizing an embodiment of the present invention; [Diagram 2] FIG. 2 illustrates a schematic diagram of user equipment suitable for utilizing embodiments of the present invention; [Diagram 3] FIG. 2 further illustrates, in schematic form, electronic devices utilizing embodiments of the present invention connected using wireless and wired network connections. [Figure 4a] 1 illustrates a schematic diagram of an encoder suitable for implementing embodiments of the present invention; [Figure 4b] 2 illustrates a schematic diagram of a decoder suitable for implementing embodiments of the present invention; [Figure 5a] FIG. 2 illustrates a linear model mapping from luma values ​​to chroma values ​​according to one embodiment of the present disclosure. [Figure 5b] FIG. 13 illustrates an updated mapping. [Figure 5c] FIG. 13 illustrates selecting control points pr outside the determined initial mapping line and performing gradient updates on such points. [Figure 5d] FIG. 13 is a diagram showing how the offset parameter b is only updated using the control points pr outside the determined initial mapping line while leaving the gradient unchanged. [Figure 6a] A diagram illustrating the selection of a reference sample R above a block B according to one embodiment of the present disclosure. [Figure 6b] A diagram illustrating the selection of a reference sample R above a block B according to one embodiment of the present disclosure. [Figure 6c] A diagram illustrating the selection of a reference sample R above a block B according to one embodiment of the present disclosure. [Figure 7a] 1 is a flow diagram of a method according to one embodiment. [Figure 7b] 4 is a flow diagram of a method according to another embodiment. [Figure 8]1 is a schematic diagram of an example multimedia communication system in which various embodiments may be implemented; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] The following describes in more detail suitable apparatus and possible mechanisms for initiating a viewpoint switch. In this regard, reference is first made to Figures 1 and 2, in which Figure 1 shows a block diagram of a video coding system according to an exemplary embodiment as a schematic block diagram of an exemplary apparatus or electronic device 50 that may incorporate a codec according to an embodiment of the present invention. Figure 2 shows the layout of the apparatus according to an exemplary embodiment. The elements of Figures 1 and 2 are now described.

[0030] The electronic device 50 may be, for example, a mobile terminal or user equipment of a wireless communication system, however, it will be appreciated that embodiments of the present invention may be implemented in any electronic device or apparatus that may require the encoding and / or decoding of video images.

[0031] The device 50 may include a housing 30 for mounting and protection of the device. The device 50 may further include a display 32 in the form of a liquid crystal display. In other embodiments of the invention, the display may be any suitable display technology suitable for displaying images or video. The device 50 may further include a keypad 34. In other embodiments of the invention, any suitable data or user interface mechanism may be utilized. For example, the user interface may be implemented as a virtual keyboard or data entry system as part of a touch-sensitive display.

[0032] The device may include a microphone 36 or any suitable audio input, which may be a digital or analog signal input. The device 50 may further include an audio output device, which in embodiments of the present invention may be any one of earphones 38, a speaker, or an analog or digital audio output connection. The device 50 may further include a battery (or in other embodiments of the present invention, the device may be powered by any suitable portable energy device, such as a solar cell, a fuel cell, or a clockwork generator). The device may further include a camera, which may record or capture images and / or videos. The device 50 may further include an infrared port for short range line of sight communication with other devices. In other embodiments, the device 50 may further include any suitable short range communication solution, such as a Bluetooth wireless connection, or a USB / Firewire wired connection.

[0033] The device 50 may comprise a controller 56, processor or processor circuitry for controlling the device 50. The controller 56 may, in embodiments of the present invention, be connected to a memory 58 which may store data, both in the form of image and audio data, and / or may further store instructions for implementation in the controller 56. The controller 56 may further be connected to a codec circuitry 54 suitable for performing coding and decoding of audio and / or video data or for assisting in the coding and decoding performed by the controller.

[0034] The device 50 may further comprise a card reader 48 and a smart card 46, for example a UICC and UICC reader suitable for providing user information and providing authentication information for authentication and authorization of the user in the network.

[0035] The device 50 may include a radio interface circuit 52 connected to the controller and suitable for generating radio communication signals for communication with, for example, a cellular communication network, a radio communication system, or a wireless local area network. The device 50 may further include an antenna 44 connected to the radio interface circuit 52 for transmitting radio frequency signals generated by the radio interface circuit 52 to other devices and receiving radio frequency signals from other devices.

[0036] The device 50 may include a camera capable of recording or detecting individual frames, which are then passed to the codec 54 or controller for processing. The device may receive video image data for processing from another device prior to transmission and / or storage. The device 50 may further receive images for coding / decoding wirelessly or by a wired connection. The structural elements of the device 50 described above illustrate examples of means for performing the corresponding functions.

[0037] With reference to Figure 3, an example of a system in which embodiments of the present invention may be utilized is shown. System 10 includes a number of communication devices capable of communicating over one or more networks. System 10 may include any combination of wired or wireless networks, including, but not limited to, wireless cellular telephone networks (such as GSM, UMTS, CDMA networks, etc.), wireless local area networks (WLANs) such as those defined by any of the IEEE 802.x standards, Bluetooth personal area networks, Ethernet local area networks, token ring local area networks, wide area networks, and the Internet.

[0038] The system 10 may include both wired and wireless communication devices and / or apparatus 50 suitable for implementing embodiments of the present invention.

[0039] 3 shows a cellular network 11 and a representation of the Internet 28. Connectivity to the Internet 28 can include, but is not limited to, long-range wireless connections, short-range wireless connections, and various wired connections, including, but not limited to, telephone lines, cable lines, power lines, and similar communication paths.

[0040] Exemplary communication devices shown in system 10 may include, but are not limited to, electronic device or apparatus 50, a combination personal digital assistant (PDA) and cell phone 14, a PDA 16, an integrated messaging device (IMD) 18, a desktop computer 20, and a notebook computer 22. The apparatus 50 may be stationary or may be portable if carried by an individual who is moving. The apparatus 50 may also be located in a vehicle including, but not limited to, an automobile, truck, taxi, bus, train, boat, airplane, bicycle, motorcycle, or similar suitable transportation means.

[0041] Embodiments may also be implemented in set-top boxes; digital TV receivers that may or may not have display or wireless capabilities, in tablets or (laptop) personal computers (PCs) having hardware, software, or combinations of encoder / decoder implementations, in various operating systems, and in chipsets, processors, DSPs, and / or embedded systems that provide hardware / software based coding.

[0042] Some or further devices may send and receive calls and messages and communicate with a service provider through a wireless connection 25 to a base station 24. The base station 24 may be connected to a network server 26 that enables communication between the cellular network 11 and the Internet 28. The system may include additional communication devices and different types of communication devices.

[0043] The communication devices may communicate using a variety of transmission technologies, including, but not limited to, code division multiple access (CDMA), global systems for mobile communications (GSM), universal mobile telecommunications system (UMTS), time division multiple access (TDMA), frequency division multiple access (FDMA), transmission control protocol-internet protocol (TCP-IP), short message service (SMS), multimedia message service (MMS), email, instant message service (IMS), Bluetooth, IEEE 802.11, and any similar wireless communication technology. The communication devices involved in implementing various embodiments of the present invention may communicate using a variety of mediums, including, but not limited to, radio, infrared, laser, cable connections, and any suitable connection.

[0044] In telecommunications and data networks, a channel may refer to either a physical channel or a logical channel. A physical channel may refer to a physical transmission medium such as a wire, while a logical channel may refer to a logical connection on a multiplex medium that can carry several logical channels. A channel may be used to carry an information signal, e.g., a bit stream, from one or several senders (or transmitters) to one or several receivers.

[0045] The MPEG-2 Transport Stream (TS), specified in ISO / IEC 13818-1, or equivalently in ITU-T Recommendation H.222.0, is a format for carrying audio, video, and other media, as well as program or other metadata, in a multiplexed stream. Packet Identifiers (PIDs) are used to identify elementary streams (aka packetized elementary streams) within a TS. Thus, logical channels within an MPEG-2 TS can be considered to correspond to specific PID values.

[0046] Available media file format standards include the ISO Base Media File Format (ISO / IEC 14496-12, sometimes abbreviated as ISOBMFF), and a file format for NAL unit structured video (ISO / IEC 14496-15), which is derived from ISOBMFF.

[0047] A video codec consists of an encoder that converts an input video into a compressed representation suitable for storage / transmission, and a decoder that decompresses the compressed video representation back into a displayable form. The video encoder and / or the video decoder may also be separate from each other, i.e., they do not have to form a codec. In general, an encoder discards some information of the original video sequence in order to represent the video in a more compact form (i.e., at a lower bit rate).

[0048] A typical hybrid video encoder, e.g., many encoder implementations of ITU-T H.263 and H.264, encodes video information in two phases. First, pixel values ​​of a particular picture area (or "block") are predicted, e.g., by motion compensation means (finding and indicating an area in one of the previously coded video frames that closely corresponds to the block to be coded) or by spatial means (using pixel values ​​around the block to be coded in a specified way). Second, the prediction error, i.e., the difference between the predicted block of pixels and the original block of pixels, is coded. This is generally done by transforming the pixel value differences using a specified transform (e.g., the discrete cosine transform (DCT) or a variant thereof), quantizing the coefficients, and entropy coding the quantized coefficients. By varying the fidelity of the quantization process, the encoder can control the balance between the precision of the pixel representation (picture quality) and the size of the resulting coded video representation (file size or transmission bit rate).

[0049] In temporal prediction, the source of prediction is a previously decoded picture (aka reference picture). In intra block copy (IBC; aka intra block copy prediction), prediction is applied similarly to temporal prediction, but the reference picture is the current picture, and only previously decoded samples can be referenced in the prediction process. Inter-layer or inter-view prediction can be applied similarly to temporal prediction, but the reference picture is a decoded picture from another scalable layer or another view, respectively. In some cases, inter prediction can only refer to temporal prediction, while in other cases, inter prediction can jointly refer to temporal prediction and any of intra block copy, inter-layer prediction, and inter-view prediction, provided that they are performed in the same or similar process as temporal prediction. Inter prediction or temporal prediction may sometimes be referred to as motion compensation or motion compensated prediction.

[0050] Motion compensation can be performed with either full sample precision or sub-sample precision. For full sample precision motion compensation, the motion may be represented as a motion vector with integer values ​​for horizontal and vertical displacements, and the motion compensation process uses those displacements to effectively copy samples from the reference picture. For sub-sample precision motion compensation, the motion vector is represented with fractional or decimal values ​​for the horizontal and vertical components of the motion vector. If the motion vector points to a non-integer position in the reference picture, a sub-sample interpolation process is typically invoked to calculate a predicted sample value based on the reference sample and the selected sub-sample position. The sub-sample interpolation process typically consists of horizontal filtering to compensate for the horizontal offset relative to the full sample position followed by vertical filtering to compensate for the vertical offset relative to the full sample position. However, vertical processing may also occur before horizontal processing, depending on the environment.

[0051] Inter prediction, sometimes called temporal prediction, motion compensation, or motion compensated prediction, reduces temporal redundancy. In inter prediction, the source of prediction is a previously decoded picture. Intra prediction exploits the fact that adjacent pixels in the same picture are likely to be correlated. Intra prediction can be performed in the spatial domain or the transform domain, i.e., either sample values ​​or transform coefficients can be predicted. Intra prediction is generally utilized in intra coding, where inter prediction is not applied.

[0052] One result of the coding procedure is a set of coding parameters, such as motion vectors and quantized transform coefficients. Many parameters can be entropy coded more efficiently if they are first predicted from spatially or temporally neighboring parameters. For example, motion vectors can be predicted from spatially neighboring motion vectors and only the difference to the motion vector predictor can be coded. Prediction of coding parameters and intra prediction are sometimes collectively referred to as in-picture prediction.

[0053] Figures 4a and 4b show an encoder and a decoder suitable for utilizing an embodiment of the present invention. A video codec consists of an encoder that converts an input video into a compressed representation suitable for storage / transmission, and a decoder that can restore the compressed video representation back to a displayable form. In general, an encoder discards and / or loses some information of the original video sequence in order to represent the video in a more compact form (i.e., at a lower bit rate). An example of an encoding process is shown in Figure 4a. Figure 4a shows the image to be encoded (In); a predicted representation of an image block (P'n), a prediction error signal (Dn); a reconstructed prediction error signal (D'n); a preliminary reconstructed image (I'n), a final reconstructed image (R'n); a transform (T) and an inverse transform (T-1); a quantization (Q) and an inverse quantization (Q-1); an entropy encoding (E); a reference frame memory (RFM); an inter prediction (Pinter); an intra prediction (Pintra); a mode selection (MS) and a filtering (F).

[0054] An example of the decoding process is shown in Figure 4b, which shows the predicted representation of an image block (P'n); the reconstructed prediction error signal (D'n); the preliminary reconstructed image (I'n); the final reconstructed image (R'n); the inverse transform (T-1); the inverse quantization (Q-1); the entropy decoding (E-1); the reference frame memory (RFM); the prediction (either inter or intra) (P); and the filtering (F).

[0055] Many hybrid video encoders encode video information in two phases. First, pixel values ​​of a particular picture area (or "block") are predicted, for example, by motion compensation means (finding and indicating an area in one of the previously coded video frames that closely corresponds to the block to be coded) or by spatial means (using pixel values ​​around the block to be coded in a specified way). Second, the prediction error, i.e., the difference between the predicted block of pixels and the original block of pixels, is coded. This is generally done by transforming the pixel value differences using a specified transform (e.g., the discrete cosine transform (DCT) or a variant thereof), quantizing the coefficients, and entropy coding the quantized coefficients. By varying the fidelity of the quantization process, the encoder can control the balance between the precision of the pixel representation (picture quality) and the size of the resulting coded video representation (file size or transmission bit rate). Video codecs can also provide transform skip modes that the encoder can choose to use. In transform skip mode, the prediction error is coded in the sample domain, for example, by deriving a sample-by-sample difference value for certain neighboring samples and coding the sample-by-sample difference value with an entropy coder.

[0056] Entropy coding / decoding can be performed in many ways. For example, context-based coding / decoding can be applied, where both the encoder and the decoder modify the context state of the coding parameters based on previously coded / decoded coding parameters. The context-based coding can be, for example, Context-Adaptive Binary Arithmetic Coding (CABAC) or Context-Based Variable Length Coding (CAVLC) or any similar entropy coding. Entropy coding / decoding can alternatively or additionally be performed using a variable length coding scheme such as Huffman coding / decoding or Exponential Golomb coding / decoding. The decoding of coding parameters from an entropy-coded bitstream or codewords is sometimes referred to as parsing.

[0057] A phrase along the bitstream (e.g., indicating along the bitstream) may be defined to refer to out-of-band transmission, signaling, or storage in a manner in which the out-of-band data is associated with the bitstream. A phrase such as decoding along the bitstream may refer to decoding of referenced out-of-band data (which may be obtained from out-of-band transmission, signaling, or storage) associated with the bitstream. For example, an indication along the bitstream may refer to metadata within a container file that encapsulates the bitstream.

[0058] In the following, some embodiments will be described in the context of one video coding configuration. However, it should be noted that the embodiments are not necessarily limited to this particular configuration. The embodiments relate to fine-grained gradual decoding refresh with CTU-aligned boundary between clean and dirty areas of a picture.

[0059] The Advanced Video Coding standard (sometimes abbreviated as AVC or H.264 / AVC) was developed by the Video Coding Experts Group (VCEG) of the Telecommunication Standardization Sector of the International Telecommunication Union (ITU-T) and the Joint Video Team (JVT) of the Moving Picture Experts Group (MPEG) of the International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC). The H.264 / AVC standard has been published by both standards bodies and is referred to as ITU-T Recommendation H.264 and ISO / IEC International Standard 14496-10, also known as MPEG-4 Part 10 Advanced Video Coding (AVC). There are numerous versions of the H.264 / AVC standard, each incorporating new extensions or features into the specification. These extensions include Scalable Video Coding (SVC) and Multiview Video Coding (MVC).

[0060] The High Efficiency Video Coding standard (sometimes abbreviated HEVC or H.265 / HEVC) was developed by the Joint Cooperative Team - Video Coding (JCT-VC) of VCEG and MPEG. The standard has been published by both standardization bodies and is referred to as ITU-T Recommendation H.265 and ISO / IEC International Standard 23008-2, also known as MPEG-H Part 2 High Efficiency Video Coding (HEVC). Extensions to H.265 / HEVC include scalable, multi-view, three-dimensional, and range-of-fidelity extensions, which are sometimes referred to as SHVC, MV-HEVC, 3D-HEVC, and REXT, respectively. References herein to H.265 / HEVC, SHVC, MV-HEVC, 3D-HEVC, and REXT are made for the purpose of understanding the definitions, structure, or concepts of these standard specifications and should be understood to be references to the latest versions of these standards available prior to the date of this application, unless otherwise indicated.

[0061] Versatile Video Coding (VVC, H.266, or sometimes abbreviated as H.266 / VVC) is a video compression standard developed as a successor to HEVC. VVC is specified in ITU-T Recommendation H.266 and, equivalently, in ISO / IEC 23090-3, also known as MPEG-I Part 3.

[0062] The specification for the AV1 bitstream format and decoding process was developed by the Alliance of Open Media (AOM). The AV1 specification was published in 2018. AOM is reportedly working on the AV2 specification.

[0063] Some important definitions, bitstreams, and coding structures, and concepts of H.264 / AVC, HEVC, VVC, and / or AV1, and some of their extensions, are described in this section as an example of a video encoder, decoder, encoding method, decoding method, and bitstream structure on which the embodiments may be implemented. Aspects of various embodiments are not limited to H.264 / AVC, HEVC, VVC, and / or AV1, or their extensions, but rather, the description is given on one possible basis on which the embodiments may be realized in part or in full.

[0064] A video codec may include an encoder that converts an input video into a compressed representation suitable for storage / transmission, and a decoder that decompresses the compressed video representation back into a displayable form. The compressed representation may be called a bitstream or a video bitstream. A video encoder and / or a video decoder may also be separate from each other, i.e., they do not have to form a codec. An encoder may discard some information of the original video sequence to represent the video in a more compact form (i.e., at a lower bitrate). The notation "(de)coder" refers to an encoder and / or a decoder.

[0065] Hybrid video codecs, e.g., ITU-T H.264, can encode video information in two phases. First, pixel values ​​of a certain picture area (or "block") are predicted, e.g., by motion compensation means (finding and indicating an area in one of the previously coded video frames that closely corresponds to the block to be coded) or by spatial means (using pixel values ​​around the block to be coded in a specified way). Then, the prediction error, i.e., the difference between the predicted block of pixels and the original block of pixels, is coded. This can be done by transforming the pixel value differences using a specified transformation (e.g., the discrete cosine transform (DCT) or a variant thereof), quantizing the coefficients, and entropy coding the quantized coefficients. By varying the fidelity of the quantization process, the encoder can control the balance between the precision of the pixel representation (picture quality) and the size of the resulting coded video representation (file size or transmission bit rate).

[0066] In temporal prediction, the source of prediction is a previously decoded picture (aka reference picture). In intra block copy (IBC; aka intra block copy prediction or current picture reference), prediction is applied similarly to temporal prediction, but the reference picture is the current picture, and only previously decoded samples can be referenced in the prediction process. Inter-layer or inter-view prediction can be applied similarly to temporal prediction, but the reference picture is a decoded picture from another scalable layer or another view, respectively. In some cases, inter prediction can only refer to temporal prediction, while in other cases, inter prediction can jointly refer to temporal prediction and any of intra block copy, inter-layer prediction, and inter-view prediction, provided that they are performed in the same or similar process as temporal prediction. Inter prediction or temporal prediction may sometimes be referred to as motion compensation or motion compensated prediction.

[0067] Intra prediction exploits the fact that adjacent pixels in the same picture are likely to be correlated. Intra prediction can be performed in the spatial domain or the transform domain, i.e., either sample values ​​or transform coefficients can be predicted. Intra prediction can be utilized in intra coding, where inter prediction is not applied.

[0068] One result of the coding procedure is a set of coding parameters, such as motion vectors and quantized transform coefficients. Many parameters can be entropy coded more efficiently if they are first predicted from spatially or temporally neighboring parameters. For example, motion vectors can be predicted from spatially neighboring motion vectors and only the difference to the motion vector predictor can be coded. Prediction of coding parameters and intra prediction are sometimes collectively referred to as in-picture prediction.

[0069] Entropy coding / decoding can be performed in many ways. For example, context-based coding / decoding can be applied, where both the encoder and the decoder modify the context state of the coding parameters based on previously coded / decoded coding parameters. The context-based coding can be, for example, context adaptive binary arithmetic coding (CABAC) or context-based variable length coding (CAVLC) or any similar entropy coding. Entropy coding / decoding can alternatively or additionally be performed using variable length coding schemes such as Huffman coding / decoding or exponential-Golomb coding / decoding. The decoding of coding parameters from an entropy-coded bitstream or codewords is sometimes referred to as parsing.

[0070] A video coding standard may specify bitstream syntax and semantics, as well as a decoding process for an error-free bitstream, while the encoding process may not be specified, but an encoder may simply be required to generate a conforming bitstream. Bitstream and decoder conformance may be verified with a Hypothetical Reference Decoder (HRD). A standard may include coding tools that help deal with transmission errors and losses, but using the tools for encoding may be optional, and the decoding process for erroneous bitstreams may not be specified.

[0071] The basic unit for input to an encoder and output of a decoder, respectively, is most often a picture. A picture provided as input to an encoder is sometimes called a source picture, and a picture decoded by a decoder is sometimes called a decoded picture or a reconstructed picture.

[0072] The source picture and the decoded picture each consist of one or more sample arrays, such as one of the following sets of sample arrays: - Luma (Y) only (monochrome) - Luma and two chromas (YCbCr or YCgCo). - Green, Blue, and Red (GBR, also known as RGB). - Arrays representing other unspecified monochrome or tristimulus color samplings (e.g., also known as YZX, XYZ).

[0073] In the following, these arrays may be referred to as luma (or L or Y) and chroma, and the two chroma arrays may be referred to as Cb and Cr, regardless of the actual color representation method in use. The actual color representation method in use may be indicated, for example, in the coded bitstream, using a Video Usability Information (VUI) syntax, such as HEVC. A component may be defined as an array or a single sample from one of the three sample arrays (luma and two chromas), or an array or a single sample of an array that constitutes a monochrome format picture.

[0074] A picture may be defined to be either a frame or a field. A frame contains a matrix of luma samples and possibly corresponding chroma samples. A field is a set of alternating sample rows of a frame when the source signal is interlaced and may be used as an encoder input. There may be no chroma sample array (and thus monochromatic sampling may be used), or the chroma sample array may be subsampled when compared to the luma sample array.

[0075] The chroma format can be summarized as follows: - In monochromatic sampling, there is only one sample array, which can nominally be thought of as the luma array. - In 4:2:0 sampling, each of the two chroma arrays has half the height and half the width of the luma array. - In 4:2:2 sampling, each of the two chroma arrays has the same height as the luma array and half the width of the luma array. - In 4:4:4 sampling, when separate color planes are not used, each of the two chroma arrays has the same height and width as the luma array.

[0076] A coding format or standard may allow the sample array to be coded into a bitstream as separate color planes and the separately coded color planes to be decoded from the bitstream separately. When separate color planes are used, each of them is processed separately (by the encoder and / or decoder) as a picture with monochromatic sampling.

[0077] When chroma subsampling (e.g., 4:2:0 or 4:2:2 chroma sampling) is used, the location of the chroma samples relative to the luma samples may be determined at the encoder side (e.g., as a pre-processing step or as part of the encoding). The chroma sample positions relative to the luma sample positions may be predefined in a coding standard, e.g., H.264 / AVC or HEVC, or may be indicated in the bitstream, e.g., as part of the VUI of H.264 / AVC or HEVC.

[0078] In general, a source video sequence provided as input for encoding can represent either interlaced or progressive source content. In interlaced source content, fields of opposite parity are captured at different times. Progressive source content includes captured frames. An encoder can encode fields of interlaced source content in two ways: pairs of interlaced fields can be coded into coded frames, or fields can be coded as coded fields. Similarly, an encoder can encode frames of progressive source content in two ways: frames of progressive source content can be coded into coded frames, or coded field pairs. A field pair or complementary field pair can be defined as two fields that are next to each other in decoding order and / or output order, have opposite parity (i.e., one is a top field and the other is a bottom field), and neither belongs to the other complementary field pair. Some video coding standards or schemes allow mixing of coded frames and coded fields within the same coded video sequence. Moreover, predicting coded fields from fields within a coded frame and / or predicting coded frames relative to complementary field pairs (coded as fields) may be enabled during encoding and / or decoding.

[0079] Partitioning may be defined as the division of a set into subsets such that each element of the set is in exactly one of the subsets.

[0080] In H.264 / AVC, a macroblock is a 16x16 block of luma samples and a corresponding block of chroma samples. For example, in a 4:2:0 sampling pattern, a macroblock contains one 8x8 block of chroma samples for each chroma component. In H.264 / AVC, a picture is partitioned into one or more slice groups, which in turn contain one or more slices. In H.264 / AVC, a slice consists of an integer number of macroblocks consecutively ordered in a raster scan within a particular slice group.

[0081] The following terminology may be used when describing HEVC encoding and / or decoding operations: A coding block may be defined as an N×N block of samples for some value of N such that the division of a coding tree block into coding blocks is a partition. A coding tree block (CTB) may be defined as an N×N block of samples for some value of N such that the division of a component into coding tree blocks is a partition. A coding tree unit (CTU) may be defined as a coding tree block of luma samples, two corresponding coding tree blocks of chroma samples for a picture with three sample arrays, or a coding tree block of samples for a monochromatic picture, or a picture coded using three separate color planes and a syntax structure used to code the samples. A coding unit (CU) may be defined as a coding block of luma samples, two corresponding coding tree blocks of chroma samples for a picture with three sample arrays, or a coding block of samples for a monochromatic picture, or a picture coded using three separate color planes and a syntax structure used to code the samples. A CU of the largest allowed size may be named an LCU (largest coding unit) or a coding tree unit (CTU), and a video picture is divided into non-overlapping LCUs.

[0082] In some video codecs, such as the High Efficiency Video Coding (HEVC) codec, a video picture may be divided into coding units (CUs) that cover an area of ​​the picture. A CU consists of one or more prediction units (PUs) that define a prediction process of samples within the CU, and one or more transform units (TUs) that define a prediction error coding process of samples within the CU. A CU may consist of a square block of samples with a size selectable from a predefined set of possible CU sizes. A CU of the largest allowed size may be named an LCU (largest coding unit) or coding tree unit (CTU), and a video picture is divided into non-overlapping LCUs. An LCU may be further split into a combination of smaller CUs, for example, by recursively splitting the LCU and the resulting CU. Each resulting CU may have at least one PU and at least one TU associated with it. Each PU and TU may be further split into smaller PUs and TUs to increase the granularity of the prediction and prediction error coding processes, respectively. Each PU has prediction information associated with it that defines what type of prediction should be applied to the pixels within that PU (e.g., motion vector information for inter-predicted PUs, and intra prediction directionality information for intra-predicted PUs).

[0083] Each TU may be associated with information (e.g., including DCT coefficient information) describing the prediction error decoding process for samples within the TU. Whether prediction error coding is applied to each CU is generally signaled at the CU level. If there is no prediction error residual associated with a CU, the CU can be considered to have no TUs. The division of an image into CUs and the division of CUs into PUs and TUs is generally signaled in the bitstream, allowing a decoder to reproduce the intended structure of these units.

[0084] In HEVC, a picture may be partitioned into tiles, which are rectangular and contain an integer number of LCUs. In HEVC, the partitioning into tiles forms a regular grid, and the heights and widths of the tiles differ from each other by at most one LCU. In HEVC, a slice is defined as an integer number of coding tree units contained in one independent slice segment and all subsequent dependent slice segments (if any) that precede the next independent slice segment (if any) in the same access unit. In HEVC, a slice segment is defined as an integer number of coding tree units that are consecutively ordered in a tile scan and contained in a single NAL unit. The division of each picture into slice segments is a partitioning. In HEVC, an independent slice segment is defined as a slice segment in which values ​​of syntax elements of a slice segment header are not inferred from values ​​of preceding slice segments, and a dependent slice segment is defined as a slice segment in which values ​​of some syntax elements of a slice segment header are inferred from values ​​of preceding independent slice segments in decoding order. In HEVC, a slice header is defined to be the slice segment header of an independent slice segment that is the current slice segment or the slice segment header of an independent slice segment that precedes the current dependent slice segment, and a slice segment header is defined to be the part of a coded slice segment that contains data elements related to the first or all coding tree units represented in the slice segment. CUs are scanned in the raster scan order of LCUs within a tile, or within a picture if tiles are not used. Within an LCU, CUs have a particular scan order.

[0085] The decoder reconstructs the output video by applying prediction means similar to the encoder to form a predicted representation of pixel blocks (using motion or spatial information created by the encoder and stored in the compressed representation) and prediction error decoding (the inverse operation of prediction error coding, which recovers the quantized prediction error signal in the spatial pixel domain). After applying the prediction and prediction error decoding means, the decoder sums the prediction and prediction error signals (pixel values) to form the output video frame. The decoder (and encoder) may further apply additional filtering means to improve the quality of the output video before sending it for display and / or storing it as a predictive reference for upcoming frames of the video sequence.

[0086] The filtering may include, for example, one or more of the following: deblocking, sample adaptive offset (SAO), and / or adaptive loop filtering (ALF). H.264 / AVC includes deblocking, while HEVC includes both deblocking and SAO.

[0087] The motion information may be indicated by a motion vector associated with each motion-compensated image block, such as a prediction unit. Each of these motion vectors represents the displacement between an image block of a picture to be coded (at the encoder side) or decoded (at the decoder side) and a prediction source block of one of the previously coded or decoded pictures. To represent the motion vectors efficiently, they are generally differentially coded with respect to a block-specific predicted motion vector. In a typical video codec, the predicted motion vector is created in a predefined manner, for example, by calculating the median value of the encoded or decoded motion vectors of neighboring blocks. Another way to create a motion vector prediction is to generate a list of candidate predictions from neighboring and / or co-located blocks in a temporal reference picture and signal the chosen candidate as a motion vector predictor. In addition to predicting the motion vector value, which reference picture is used for the motion compensation prediction may be predicted, and this prediction information may be represented, for example, by a reference index of a previously coded / decoded picture. The reference index is generally predicted from neighboring and / or co-located blocks in the temporal reference picture. Besides, common high-efficiency video codecs utilize an additional motion information coding / decoding mechanism, often called merging / merge mode, in which all motion field information including motion vectors and corresponding reference picture indexes for each available reference picture list are predicted and used without any modification / correction. Similarly, the prediction of motion field information is performed using the motion field information of neighboring and / or co-located blocks in the temporal reference picture, and the used motion field information is signaled in a list of motion field candidate lists filled with the motion field information of the available neighboring / co-located blocks.

[0088] In common video codecs, the prediction residual after motion compensation is first transformed with a transform kernel (such as DCT) and then coded. The reason for this is that there is often still some correlation between the residuals, and transforms can often help to reduce this correlation and provide more efficient coding.

[0089] Video coding standards and specifications may allow an encoder to split a coded picture into coded slices, etc. In-picture prediction may be disabled across slice boundaries. Thus, slices may be viewed as a way to split a coded picture into independently decodable pieces. In H.264 / AVC and HEVC, in-picture prediction across slice boundaries may be disabled. Thus, slices may be viewed as a way to split a coded picture into independently decodable pieces, and therefore slices may often be viewed as the basic unit of transmission. Often, an encoder may indicate in the bitstream what type of in-picture prediction has been stopped across slice boundaries, and a decoder operation may take this information into account, for example, when concluding which prediction sources are available. For example, if neighboring CUs are in different slices, samples from the neighboring CUs may be considered unavailable for intra prediction.

[0090] The basic unit of output of an H.264 / AVC or HEVC encoder and input of an H.264 / AVC or HEVC decoder, respectively, is a Network Abstraction Layer (NAL) unit. For transport over a packet-oriented network or storage in a structured file, the NAL units may be encapsulated in packets or similar structures. A byte stream format is specified in H.264 / AVC and HEVC for transmission or storage environments that do not provide a framing structure. The byte stream format separates NAL units from each other by prepending a start code to each NAL unit. To avoid false detection of NAL unit boundaries, the encoder implements a byte-oriented start code emulation prevention algorithm, which adds an emulation prevention byte to the NAL unit payload if the start code would otherwise occur. To allow direct gateway operation between packet-oriented and stream-oriented systems, start code emulation prevention may always be performed regardless of whether the byte stream format is used or not. A NAL unit may be defined as a syntax structure that includes an indication of the type of data that follows and bytes that contain that data in the form of RBSP, optionally interrupted by emulation prevention bytes. A raw byte sequence payload (RBSP) may be defined as a syntax structure that includes an integer number of bytes encapsulated in a NAL unit. An RBSP has the form of a data bit that is either empty or contains a syntax element, followed by an RBSP stop bit, followed by a string of zero or more subsequent bits equal to 0.

[0091] A NAL unit consists of a header and a payload. In H.264 / AVC and HEVC, the NAL unit header indicates the type of the NAL unit.

[0092] In HEVC, a two-byte NAL unit header is used for all specified NAL unit types. The NAL unit header contains one reserved bit, a six-bit NAL unit type indication, a three-bit nuh_temporal_id_plus1 indication at the temporal level (which may be required to be greater than or equal to 1), and a six-bit nuh_layer_id syntax element. The temporal_id_plus1 syntax element may be considered as a temporal identifier of the NAL unit, and the zero-based TemporalId variable may be derived as follows: TemporalId=temporal_id_plus1-1. The abbreviation TID may be used interchangeably with the TemporalId variable. A TemporalId equal to 0 corresponds to the lowest temporal level. The value of temporal_id_plus1 is required to be non-zero to avoid start code emulation involving two NAL unit header bytes. A bitstream created by excluding all VCL NAL units with a TemporalId equal to or greater than the selected value and including all other VCL NAL units remains conforming. As a result, a picture with TemporalId equal to tid_value does not use a picture with TemporalId greater than tid_value as an inter-prediction reference. A sublayer or temporal sublayer may be defined to be a temporal scalable layer (or temporal layer, TL) of a temporal scalable bitstream consisting of VCL NAL units with a particular value of the TemporalId variable and associated non-VCL NAL units. nuh_layer_id may be understood as a scalability layer identifier.

[0093] NAL units can be classified into video coding layer (VCL) NAL units and non-VCL NAL units. A VCL NAL unit is generally a coded slice NAL unit. In HEVC, a VCL NAL unit includes a syntax element that represents one or more CUs.

[0094] A non-VCL NAL unit may be, for example, one of the following types: sequence parameter set, picture parameter set, supplemental enhancement information (SEI) NAL unit, access unit delimiter, end of sequence NAL unit, end of bitstream NAL unit, or filler data NAL unit. Although parameter sets may be required for the reconstruction of a decoded picture, many of the other non-VCL NAL units are not required for the reconstruction of decoded sample values.

[0095] Parameters that remain invariant throughout a coded video sequence may be included in a sequence parameter set. In addition to parameters that may be required by the decoding process, a sequence parameter set may optionally include video usability information (VUI), which includes parameters that may be important for buffering, picture output timing, rendering, and resource reservation. In HEVC, a sequence parameter set RBSP includes parameters that may be referenced by one or more picture parameter sets RBSP or one or more SEI NAL units, including a buffering period SEI message. A picture parameter set includes parameters that are likely to be invariant across several coded pictures. A picture parameter set RBSP may include parameters that may be referenced by coded slice NAL units of one or more coded pictures.

[0096] In HEVC, a video parameter set (VPS) may be defined as a syntax structure that contains syntax elements that apply to zero or more entire coded video sequences as determined by the content of syntax elements found in an SPS, which is referenced by syntax elements found in a PPS, which is referenced by syntax elements found in each slice segment header.

[0097] A video parameter set RBSP may contain parameters that may be referenced by one or more sequence parameter sets RBSP.

[0098] The relationship and hierarchy among video parameter sets (VPS), sequence parameter sets (SPS), and picture parameter sets (PPS) may be described as follows: A VPS exists one level above an SPS in the parameter set hierarchy, and in the context of scalability and / or 3D video. A VPS may contain parameters that are common to all slices across all (scalability or view) layers in the entire coded video sequence. An SPS contains parameters that are common to all slices of a particular (scalability or view) layer in the entire coded video sequence, and may be shared by multiple (scalability or view) layers. A PPS contains parameters that are common to all slices of a particular layer representation (a representation of one scalability or view layer in one access unit), and are likely to be shared by all slices of multiple layer representations.

[0099] The VPS can provide information about layer dependency relationships in the bitstream, as well as many other pieces of information that are applicable to all slices across all (scalability or view) layers in the entire coded video sequence. The VPS can be thought of as including two parts: a base VPS and a VPS extension, which can be present as an option.

[0100] The out-of-band transmission, signaling, or storage may additionally or alternatively be used for purposes other than transmission error resistance, such as facilitating access or session negotiation. For example, a sample entry for a track in a file conforming to the ISO Base Media File Format may include a parameter set, while the bitstream coding data is stored elsewhere in the file or in a separate file. The phrase along the bitstream (e.g., indicating along the bitstream) or along the coding units of the bitstream (e.g., indicating along the coding tiles) may be used in the claimed and described embodiments to refer to out-of-band transmission, signaling, or storage such that the out-of-band data is associated with the bitstream or coding units, respectively. Phrases such as decoding along the bitstream or decoding along the coding units of the bitstream may refer to decoding of the referenced out-of-band data (which may be obtained from out-of-band transmission, signaling, or storage) associated with the bitstream or coding units, respectively.

[0101] An SEI NAL unit can contain one or more SEI messages, which are not required for decoding of the output picture but can assist in related processes such as picture output timing, rendering, error detection, error concealment, and resource reservation.

[0102] A coded picture is a coded representation of a picture.

[0103] In HEVC, a coded picture may be defined as a coded representation of a picture that includes all of the coding tree units of the picture. In HEVC, an access unit (AU) may be defined as a set of NAL units that are associated with each other according to a specified classification rule, are consecutive in decoding order, and include at most one picture with a particular value of nuh_layer_id. In addition to including the VCL NAL units of a coded picture, an access unit may further include non-VCL NAL units. The specified classification rule may, for example, associate pictures with the same output time or picture output count value to the same access unit.

[0104] A bitstream may be defined as a sequence of bits in the form of a NAL unit stream or a byte stream that forms a representation of coded pictures and associated data that form one or more coded video sequences. A first bitstream may be followed by a second bitstream in the same logical channel, such as in the same file or in the same connection of a communication protocol. An elementary stream (in the context of video coding) may be defined as a sequence of one or more bitstreams. The end of a first bitstream may be indicated with a specific NAL unit, which may be called the end of bitstream (EOB) NAL unit, which is the last NAL unit of the bitstream. In HEVC and its current draft extensions, the EOB NAL unit is required to have a nuh_layer_id equal to 0.

[0105] In H.264 / AVC, a coded video sequence is defined to be a sequence of consecutive access units in decoding order, from an IDR access unit (including itself) to the next IDR access unit (excluding itself) or to the end of the bitstream, whichever occurs first.

[0106] A coded video sequence (CVS) may be defined, for example, as a sequence of coded pictures in decoding order that are independently decodable and are followed by another coded video sequence or the end of the bitstream.

[0107] In HEVC, a coded video sequence may additionally or alternatively (to the specifications above) be specified to end when a specific NAL unit, sometimes called an end-of-sequence (EOS) NAL unit, appears in the bitstream and has a nuh_layer_id equal to 0.

[0108] Groups of pictures (GOPs) and their characteristics may be defined as follows: A GOP may be decoded regardless of whether previous pictures have been decoded. An open GOP is a group of pictures where pictures prior to the first intra picture in output order may not be correctly decoded if decoding starts from the first intra picture of the open GOP. In other words, pictures of an open GOP may refer to pictures belonging to previous GOPs (in inter prediction). A HEVC decoder can recognize the intra picture that starts an open GOP because a specific NAL unit type, the CRA NAL unit type, may be used for coded slices. A closed GOP is a group of pictures where all pictures may be correctly decoded when decoding starts from the first intra picture of the closed GOP. In other words, pictures of a closed GOP do not refer to pictures of previous GOPs. In H.264 / AVC and HEVC, a closed GOP may start with an IDR picture. In HEVC, a closed GOP may also start at a BLA_W_RADL or BLA_N_LP picture. An open GOP coding structure is potentially more efficient in compression compared to a closed GOP coding structure due to greater flexibility in the selection of reference pictures.

[0109] A Decoded Picture Buffer (DPB) may be used in an encoder and / or a decoder. There are two reasons for buffering decoded pictures: for reference in inter prediction and for reordering decoded pictures in output order. Because H.264 / AVC and HEVC provide a lot of flexibility for both reference picture marking and output reordering, separate buffers for reference picture buffering and output picture buffering may waste memory resources. Therefore, the DPB may include a unified decoded picture buffering process for reference pictures and output reordering. A decoded picture may be removed from the DPB when it is no longer used as a reference and is not needed for output.

[0110] In many coding modes of H.264 / AVC and HEVC, reference pictures for inter prediction are indicated by an index into a reference picture list. The index may be coded with variable length coding, so that typically the smaller the index, the shorter the value of the corresponding syntax element. In H.264 / AVC and HEVC, two reference picture lists (reference picture list 0 and reference picture list 1) are generated for each bi-predictive (B) slice, and one reference picture list (reference picture list 0) is formed for each inter-coded (P) slice.

[0111] Many coding standards, including H.264 / AVC and HEVC, may have a decoding process to derive a reference picture index into a reference picture list that may be used to indicate which of multiple reference pictures is used for inter prediction of a particular block. The reference picture index may be coded into the bitstream by the encoder in some inter coding modes, or may be derived (by the encoder and decoder) using, for example, neighboring blocks in some other inter coding modes.

[0112] The motion parameter types or motion information may include, but are not limited to, one or more of the following types: - an indication of the prediction type (e.g. intra-prediction, uni-prediction, bi-prediction) and / or a number of reference pictures, - an indication of a prediction direction such as inter (aka temporal) prediction, inter-layer prediction, inter-view prediction, view synthesis prediction (VSP), and inter-component prediction (which may be indicated per reference picture and / or per prediction type, and in some embodiments inter-view prediction and view synthesis prediction may be considered together as one prediction direction); and / or - an indication of the reference picture type, such as short-term and / or long-term and / or inter-layer reference picture (e.g., may be indicated for each reference picture); - a reference index to a reference picture list and / or other identifier of the reference picture (for example, may be indicated for each reference picture, the type of which may depend on the prediction direction and / or the reference picture type, which may be accompanied by other relevant information such as the reference picture list to which the reference index applies, etc.); - a horizontal motion vector component (which may be indicated, for example, per prediction block or per reference index, etc.); - a vertical motion vector component (which may be indicated, for example, per prediction block or per reference index, etc.); - one or more parameters, such as a picture order count difference and / or a relative camera separation between a picture that includes or is associated with a motion parameter and its reference picture, that may be used for scaling horizontal and / or vertical motion vector components in one or more motion vector prediction processes (wherein the one or more parameters may be indicated, for example, for each reference picture or for each reference index, etc.); - the coordinates of the block to which the motion parameters and / or motion information apply, e.g. the coordinates of the top-left sample of the block in luma sample units; - The extent of the block (e.g. width and height) to which the motion parameters and / or motion information apply.

[0113] Compared to previous video coding standards, the Versatile Video Codec (H.266 / VVC) introduces several new coding tools, such as: Intra prediction - 67 Intra modes with wide angle mode extension - Block size and mode dependent 4-tap interpolation filter - Position-dependent intra-prediction combining (PDPC) - Cross-component linear model intra prediction (CCLM) - Multi-reference intra-line prediction - Intra-subpartition - Weighted intra prediction using matrix multiplication · Inter-picture prediction - Block motion copy with spatial, temporal, history-based and pairwise average merge candidates - Affine motion inter prediction - Sub-block based temporal motion vector prediction - Adaptive motion vector resolution - 8x8 block-based motion compression for temporal motion prediction - High-precision (1 / 16-pel) motion vector storage and motion compensation using an 8-tap interpolation filter for the luma component and a 4-tap interpolation filter for the chroma component - Triangular partition - Combination of intra- and inter-prediction - Merge with MVD (MMVD) - Symmetric MVD coding - Bidirectional Optical Flow - Decoder-side motion vector refinement - Bi-prediction with CU level weighting Transformation, quantization, and coefficient coding - Multiple primary transformation options with DCT2, DST7, and DCT8 - Secondary transformation of low frequency zone - Sub-block transformation of inter prediction residuals - Dependent quantization max QP increased from 51 to 63 - Transform coefficient coding with sign data hiding - Transform skip residual coding Entropy coding - Arithmetic coding engine with adaptive double window probability updates In-loop filter - In-loop reshaping - Deblocking filter with stronger longer filters - Sample Adaptive Offset - Adaptive Loop Filter Screen Content Coding: - Referencing the current picture by limiting the reference area 360° video coding - Horizontal wraparound motion compensation High-level syntax and parallel processing - Reference picture management using direct reference picture list signaling - Tile Group with Rectangle Tile Group

[0114] Scalable video coding can refer to a coding structure in which one bitstream can contain multiple representations of the content, e.g., at different bit rates, resolutions, or frame rates. In these cases, the receiver can extract the desired representation according to its characteristics (e.g., the resolution that best matches the display device). Alternatively, a server or network element can extract a portion of the bitstream to be transmitted to the receiver, e.g., according to the network characteristics or the processing capabilities of the receiver. A meaningful decoded representation can be created by decoding only certain portions of the scalable bitstream. A scalable bitstream generally consists of a "base layer" that provides the lowest quality video available, and one or more enhancement layers that enhance the video quality when received and decoded together with the lower layers. To improve the coding efficiency of an enhancement layer, the coded representation of that layer generally relies on the lower layers. For example, the motion and mode information of the enhancement layer can be predicted from the lower layers. Similarly, pixel data of the lower layers can be used to create a prediction of the enhancement layer.

[0115] In some scalable video coding schemes, a video signal may be encoded into a base layer and one or more enhancement layers. An enhancement layer may, for example, enhance the temporal resolution (i.e., frame rate), spatial resolution, or simply the quality of the video content represented by another layer or a portion thereof. Each layer, together with all its subordinate layers, is, for example, one representation of a video signal at a particular spatial resolution, temporal resolution, and quality level. In this document, a scalable layer, together with all its subordinate layers, is referred to as a "scalable layer representation." A portion of the scalable bitstream corresponding to a scalable layer representation may be extracted and decoded to produce a representation of the original signal with a particular fidelity.

[0116] Scalability modes or scalability dimensions may include, but are not limited to, the following: - Quality scalability: Base layer pictures are coded at a lower quality than enhancement layer pictures, which may be achieved, for example, by using a larger quantization parameter value (i.e., a larger quantization step size for transform coefficient quantization) in the base layer than in the enhancement layer. Quality scalability may be further classified into fine-grain or fine-grain scalability (FGS), medium-grain or medium-grain scalability (MGS), and / or coarse-grain or coarse-grain scalability (CGS), as described below. - Spatial scalability: Base layer pictures are coded at a lower resolution (i.e., have fewer samples) than enhancement layer pictures. Spatial scalability and quality scalability, especially the coarse-grained scalability type, can sometimes be considered as the same type of scalability. - Bit depth scalability: Base layer pictures are coded at a lower bit depth (eg, 8 bits) than enhancement layer pictures (eg, 10 or 12 bits). - Dynamic range scalability: The scalable layers represent images obtained with different dynamic ranges and / or using different tone mapping functions and / or different optical transfer functions. - Chroma format scalability: A base layer picture provides a lower spatial resolution in the chroma sample array (eg, coded in a 4:2:0 chroma format) than an enhancement layer picture (eg, in a 4:4:4 format). - Color gamut scalability: Enhancement layer pictures have a richer / wider color representation range than base layer pictures, e.g., the enhancement layer can have a UHDTV (ITU-R BT.2020) color gamut and the base layer can have an ITU-R BT.709 color gamut. - View scalability, sometimes called multiview coding. The base layer represents the first view, while the enhancement layer represents the second view. A view may be defined as a sequence of pictures representing one camera or viewpoint. In stereoscopic or two-view video, one video sequence or view can be considered to be presented to the left eye, while a parallel view is presented to the right eye. - Depth scalability, sometimes called depth enhancement coding: one or several layers of the bitstream can represent a texture view, while one or more other layers can represent a depth view. - Region of interest scalability (as explained below). - Interlace-to-progressive scalability (also known as field-to-frame scalability): The coded interlaced source content material in the base layer is enhanced in the enhancement layer to represent progressive source content. The coded interlaced source content in the base layer may include coded fields, coded frames representing field pairs, or a mixture thereof. In interlace-to-progressive scalability, the base layer picture may be resampled so that it is a suitable reference picture for one or more enhancement layer pictures. - Hybrid Codex Scalability (also known as Coding Standard Scalability): In hybrid codex scalability, the bitstream syntax, semantics, and decoding process of the base layer and enhancement layer are specified in different video coding standards. Thus, the base layer pictures are coded according to a different coding standard or format than the enhancement layer pictures. For example, the base layer may be coded using H.264 / AVC and the enhancement layer may be coded using the HEVC multi-layer extension.

[0117] For example, in the case of spatial, bit depth, and chroma format scalability mentioned above, the base layer information may be used to code the enhancement layer to minimize additional bit rate overhead.

[0118] Scalability can be enabled in at least two basic ways: by introducing new coding modes to perform prediction of pixel values ​​or syntax from lower layers of the scalable representation, or by placing lower layer pictures in a reference picture buffer (decoded picture buffer, DPB) of the higher layer. The first approach can be more flexible and therefore provide better coding efficiency in most cases. However, the second reference frame-based scalability approach can be implemented very efficiently with minimal changes to a single layer codec while still achieving most of the available coding efficiency gains. According to one approach, the reference frame-based scalability codec is implemented by utilizing the same hardware or software implementation for all layers, and only DPB management can be addressed by external means.

[0119] To enable parallel processing, an image may be split into independently codable and decodable image segments (slices or tiles). A slice generally refers to an image segment composed of a certain number of basic coding units that are processed in a default coding or decoding order, while a tile generally refers to an image segment defined as a rectangular image region that is processed, at least to some extent, as an individual frame.

[0120] Video may be encoded in the YUV or YCbCr color space, since it is known to reflect some properties of the human visual system and allows the use of lower quality representations for the Cb and Cr channels since human perception is less sensitive to the chrominance fidelity that these channels represent.

[0121] Cross-component linear model prediction is used, for example, in the VVC / H.266 video codec. In its variants, there are three chroma prediction modes that use cross-component linear model prediction. One of them can be selected by the encoder as the prediction mode for a chroma prediction block and signaled to the decoder in the bitstream. The difference between the three modes is the set of reference samples used to generate the parameters of the linear model. One of the modes uses only samples above the prediction block; one of the modes uses only samples to the left of the prediction block; one of the modes uses samples both above and to the left of the prediction block. To keep the complexity of the parameter generation low, the parameters are calculated using only a subset of the reference samples available at the block boundaries.

[0122] The VVC / H.266 cross-component linear model prediction predicts (or maps) the luma sample values ​​lumaVal to predicted chroma sample values ​​chromaVal by applying the following formula: chromaVal=((lumaVal*a)>>k)+b Here, the parameters a and k determine the slope of the linear model, and b determines the offset value of the linear model. The notation ">>" is used to represent a bit-shift operation to the right, which corresponds to division by a power of 2. The parameters a, k, and b are determined using a determined set of available reference samples.

[0123] An example of using linear regression to calculate linear model parameters is given in JVET (Joint Video Experts Team) contribution JVET-D0110. In that implementation, there may be two linear models operating at different ranges of the luma spectrum.

[0124] In the following, some exemplary embodiments of the present disclosure are described in more detail.

[0125] First, some operations of the encoder are described. The encoder receives the luminance and chrominance components of the pixels of the image to be encoded. The image is divided into smaller blocks, and the luminance and chrominance components of one image can be processed on a block basis. The information of the previously encoded and then decoded blocks of the image or parts thereof is stored by the encoder in a reference frame memory (RFM) and can be used, for example, in the prediction of subsequent images.

[0126] A linear model for mapping luma values ​​to chroma values ​​may be used to generate predicted chroma values ​​based on the decoded luma values. This type of model may be given as follows, using a gradient parameter "a" and an offset parameter "b": chromaVal=lumaVal*a+b

[0127] The encoder can encode the gradient parameter "a" and the offset parameter "b" in the bitstream, and the decoder can obtain those parameters from the bitstream, or those parameters may be initially known by the decoder and only changes to those parameters are signaled to the decoder.

[0128] 5a illustrates a mapping according to one embodiment of the present disclosure. Each valid luma sample value may be mapped to a chroma sample value using a model. In this example, two points or luma-chroma pairs p0 and p1 with luma values ​​y0 and y1 and chroma values ​​c0 and c1 define the gradient parameter a and offset parameter b of the mapping function. In a practical implementation using integer arithmetic, the formula may further include a scaling parameter "k" that defines the basis or precision of the gradient parameter a: chromaVal=((lumaVal*a)>>k)+b

[0129] The scaling parameter k may be chosen during the process of generating the model's parameters, for example by the encoder, or it may also be indicated to the decoder in a different way, or a fixed k may be used for all mappings the encoder does. The parameter k determines how many bits the result of the multiplication between lumaVal and the gradient parameter a is shifted down, using a bitwise right shift operation >>, to reach the luma-chroma value space. In other words, the result is 2 k It is divided by.

[0130] In one embodiment, the update term "u" for the gradient parameter a is defined by the encoder.

[0131] The update term u can have a basis or precision “s” that is different from the basis or precision k determined for the gradient parameter a. For example, the update term u can be calculated by shifting the received update term u down by 3 bits (i.e., shifting it down by 2 3 As another example, update term u can have a basis of 3, which corresponds to shifting the received update term u down by 4 bits (i.e., dividing it by 2 4For example, the basis s may have a basis of 4, which corresponds to a 1 / 2 division by s = 16. This basis may be fixed or signaled in the bitstream, or may be adaptively determined by the decoder depending on, for example, the size of the block to be predicted or processed. As an example, the basis s may be larger, corresponding to a finer accuracy update term, when the prediction block size is above a threshold, and smaller when the prediction block size is below the threshold.

[0132] To add the update term u to the gradient parameter a, the bases of these parameters must be made equal, which can be achieved by bitwise shifting u up until its base matches that of k, if k is greater than s, or by bitwise shifting a up until its base matches that of k, if k is less than s.

[0133] In this example, the updated linear model with parameters a', k', and b' is: chromaVal=((lumaVal*a')>>k')+b' It can be written as:

[0134] The updated linear model parameters may be calculated, for example, using the following pseudocode: a,b,k,refLuma = estimateModel (set of reference samples) u=decodeUpdateTerm(bitstream) k'=k if(k <s) { / / Make the final shift at least the size of the update precision a=a<<(sk) k'=s } else if(k>s) { / / Final shift is larger than the precision of the update: scale up the update to the final precision u=u<<(ks) } a'=a+u b'=b-((u*yr)>>k')

[0135] The original linear model parameters a, b, and k may be calculated, for example, by the estimateModel function according to the process defined in the H.266 / VVC specification, or in an alternative manner, such as using linear regression. In addition to the original parameters a, b, and k, an additional reference luma parameter y r is determined and used as a reference value when calculating the updated offset parameter b'. r The parameters can be determined in various ways. For example, if two luma-chroma pairs are used to calculate the linear model, then y r may be set to the average of the luma values ​​of those pairs. If four luma-chroma pairs are used to compute the linear model, y r y may be set to, for example, the average of the largest and smallest luma values ​​of the pairs; or y r may be set to the average of the second-largest and third-largest luma values ​​of the pair; or y r may be set to the average of the luma values ​​of those four pairs. r may be calculated as the average of the luma values ​​in the determined set of reference luma-chroma pairs or luma reference values, the median of those values, or the average of the maximum and minimum of those values, or in other manners.

[0136] y r The manner in which y is calculated may also be determined based on bitstream signaling. For example, the average of the determined luma reference samples is y r Whether it is used as or y r It may be signaled whether y is calculated as a weighted average of a set of reference luma values ​​and what their weights are. In addition to or instead of using conventional reference values ​​taken from outside the block boundaries, rThe parameter determination may include reconstructed luma values ​​inside a prediction block or prediction unit.

[0137] Figure 5b shows the updated mapping. The update term u is taken as the control point around which the mapping is rotated, and the luma value y r Reference point p r is applied to the gradient of the mapping using the updated gradient parameter a' and the updated offset parameter b', which now define the new mapping. The scale parameters k and s are omitted to simplify the illustration.

[0138] The update term u may be signaled as two separate components: magnitude and sign. The magnitude indicates the absolute value u of the update term u, i.e., |u|, and the sign indicates whether the actual value is positive, i.e., equal to the absolute value |u|, or negative, i.e., equal to -1*|u|. The magnitude may be binarized and coded using arithmetic coding, or signaled as a variable-length or fixed-length codeword. The sign is coded as an arithmetically coded binary codeword, where either the symbol 0 or the symbol 1 is chosen to indicate that the update term is refining the original or reference value "a" towards zero; the other symbol is chosen to indicate that the reference value is refining away from zero. The two alternative choices are summarized in Tables 1 and 2 below.

[0139] The expression "towards zero" means that the actual change in the reference value is such that the magnitude (absolute value) of the reference value decreases when the update term is added to the reference value. Correspondingly, the expression "away from zero" means that an update term can be used such that the actual change in the reference value is such that the magnitude (absolute value) of the reference value increases when the update term is added to the reference value.

[0140] In Table 1, the sign of the update term u is indicated depending on the syntax element representing the sign and reference value a, where syntax element 0 represents the code that refines the reference value towards zero and syntax element 1 represents the code that refines the reference value away from zero.

[0141] [Table 1]

[0142] For example, if the encoder determines that the reference value a is +4 and the update term is −2, the encoder may set the syntax element representing the sign to 0 and encode the magnitude of the update term of 2 and syntax element 0 to indicate to the decoder that the update term is negative. As another example, if the encoder determines that the reference value a is −3 and the update term is −2, the encoder may set the syntax element representing the sign to 1 and encode the magnitude of the update term of 2 and syntax element 1 to indicate to the decoder that the update term is negative.

[0143] In Table 2, the sign of the update term u is indicated depending on the syntax element representing the sign and reference value a, when syntax element 1 represents the code that refines the reference value towards zero and syntax element 0 represents the code that refines the reference value away from zero.

[0144] [Table 2]

[0145] For example, if the encoder determines that the reference value a is +4 and the update term is −2, the encoder may set the syntax element representing the sign to 1. As another example, if the encoder determines that the reference value a is −3 and the update term is −2, the encoder may set the syntax element representing the sign to 0.

[0146] It should be noted that while zero is used as an example of a threshold, the same principle may also be implemented with thresholds other than zero. As an example, the threshold may be 2, and the syntax element would indicate whether the reference value should be changed towards or away from 2. Thus, in the leftmost column, the zero would be replaced with a non-zero threshold, e.g., 2 in this example. Still, the syntax element representing the sign together with the reference value (before adding the update term) would reveal whether the sign of the update term is positive or negative.

[0147] A syntax element value of 0 may also be referred to as a first representation value, and a syntax element value of 1 may also be referred to as a second representation value. It should also be noted that the first representation value need not be 0 and the first representation value need not be 1, respectively, and instead, other values ​​may also be used.

[0148] Furthermore, the magnitude of the update term u, either independently or in combination with a reference value, can determine different zones, and different methods for coding the sign of the update term may be selected for such zones. For example, if the magnitude of the update term is below a threshold, the sign may be indicated with a fixed-length coding, and otherwise with a sign-inversion approach.

[0149] At the time the syntax element is decoded, the decoder may not know the reference value a. In this case, an initial value of the update term u may be decoded for the update term u by setting the initial value negative if the decoded syntax element representing the initial sign of the update term is 1 and positive if the decoded syntax element is 0, or vice versa, i.e., by setting the initial value negative if the decoded syntax element representing the initial sign of the update term is 0 and positive if the decoded syntax element is 1. The final value of the update term u may then be assigned once the reference value a is determined. In this case, the final value of the update term u may be determined using the following pseudocode, which inverts the sign of the decoded update term for positive values ​​of the reference value a, or preserves the sign of the decoded update term for negative and zero values ​​of the reference value a: uDec=decodeUpdateTerm(bitstream) a,b,k,refLuma = estimateModel (set of reference samples) u=a>0?-uDec:uDec

[0150] This can also be expressed in the following pseudocode: uDec=decodeUpdateTerm(bitstream) a,b,k,refLuma = estimateModel (set of reference samples) if a>0: u=-uDec else: u=uDec

[0151] In other words, the encoded update term received in the bitstream is decoded to obtain uDec (the magnitude of u). The values ​​a, b, k, and refLuma are obtained based on a set of reference samples. Then, if the reference value a is greater than zero, the update term is set to the inverted value of the decoded update term. If the reference value a is not greater than zero (i.e., is less than or equal to zero), the update term is set to the value of the decoded update term without inverting it.

[0152] In one embodiment, an update term “u” for gradient parameter a is defined by the encoder, encoded into the bitstream by the encoder, and received from the bitstream by the decoder, a reference point consisting of a luma-chroma value pair is determined, and parameters a, k, and b are updated by the decoder based on the update term and the reference point.

[0153] In one embodiment, an update term u for gradient parameter a is encoded into a bitstream by an encoder and received from the bitstream by a decoder, a reference luma value is determined, and parameters a, k, and b are updated by the decoder based on the update term and the reference luma value.

[0154] In one embodiment, a video or image decoder performs the following steps.

[0155] The sign and magnitude of the initial update term are decoded. A value of the initial update term is determined based on the decoded sign and magnitude of the initial update term. A reference value of a parameter of a luma-to-chroma mapping function is determined. The value of the update term is determined by inverting the sign of the initial update term if the value of the reference parameter is greater than zero, or the sign is not inverted otherwise. The value of the parameter of the luma-to-chroma mapping function is determined using the reference value and the value of the update term.

[0156] In one embodiment, the value of the update term is determined by reversing the sign of the initial update term when the value of the reference parameter is greater than or equal to zero.

[0157] In one embodiment, the value of the update term is determined by leaving the initial update term unchanged if the value of the reference parameter is less than or equal to zero.

[0158] In one embodiment, the value of the update term is determined by leaving the initial update term unchanged if the value of the reference parameter is less than zero.

[0159] In an alternative embodiment, the video or image decoder performs the following steps.

[0160] A reference value for a parameter of a luma-to-chroma mapping function is calculated. A magnitude of an update term for the parameter of the luma-to-chroma mapping function is determined. A binary syntax element to be used in determining the sign of the update term is also determined. The sign of the update term is determined by interpreting the binary syntax element based on the reference value. A value for the parameter of the luma-to-chroma mapping function is determined using the reference value, the decoded magnitude of the update term, and the determined sign of the update term.

[0161] In one embodiment, the sign of the update term is determined to be positive if the reference value is greater than zero and the decoded syntax element representing the sign is 1; or if the reference value is less than or equal to zero and the decoded syntax element representing the sign is 0. Similarly, the sign of the update term is determined to be negative if the reference value is greater than zero and the decoded syntax element representing the sign is 0; or if the reference value is less than or equal to zero and the decoded syntax element representing the sign is 1.

[0162] In one embodiment, the sign of the update term is determined to be negative if the reference value is greater than zero and the decoded syntax element representing the sign is 1; or if the reference value is less than or equal to zero and the decoded syntax element representing the sign is 0. Similarly, the sign of the update term is determined to be positive if the reference value is greater than zero and the decoded syntax element representing the sign is 0; or if the reference value is less than or equal to zero and the decoded syntax element representing the sign is 1.

[0163] In one embodiment, the sign is determined such that the update term refines the reference value towards zero if the decoded binary syntax element representing the sign of the update term is 0 and refines the reference value away from zero if the decoded binary syntax element representing the sign of the update term is 1.

[0164] In one embodiment, the sign is determined such that the update term refines the reference value towards zero when the decoded binary syntax element representing the sign of the update term is 1 and refines the reference value away from zero when the decoded binary syntax element representing the sign of the update term is 0.

[0165] In one embodiment, a video or image encoder performs the following steps:

[0166] A reference value for a parameter of a luma-to-chroma mapping function is determined. A value for an update term of the reference value is determined. A displayed value of the update term is determined such that if the reference value is greater than zero, the sign of the displayed value is opposite to the sign of the determined value of the update term. The sign and magnitude of the displayed value of the update term are encoded in the video bitstream.

[0167] In one embodiment, the displayed value of the update term is determined by inverting the sign of the update term when the value of the reference parameter is greater than or equal to zero.

[0168] In one embodiment, the displayed value of the update term is determined by leaving the update term unchanged if the value of the reference parameter is less than or equal to zero.

[0169] In one embodiment, the displayed value of the update term is determined by leaving the update term unchanged if the value of the reference parameter is less than zero.

[0170] In one embodiment, a rate-distortion determination is made to determine the value of the update term, and the displayed update term is determined based on the value of the update term.

[0171] In one embodiment, a rate-distortion determination is made to determine values ​​of the displayed update terms, and the update terms are determined based on the values ​​of the displayed update terms.

[0172] Figure 5c shows the control points p r We select a point such that the gradient update is performed on that point. In this case, the offset parameter b should be updated accordingly. For example, p r If a is a particular amount of chroma value on the mapping line, then b' should be raised by the same amount. Alternatively, b' can be calculated by multiplying the updated gradient parameter a' and the control point p r This can be calculated, for example, using the coordinates of the control point p r Luma value of y r and the chroma value c r This can be done using: b'=c r -((a'*y r )>>k')

[0173] Figure 5d shows the control points p r to indicate that only an update to the offset parameter b is performed while leaving the gradient unchanged. In this alternative, the gradient parameter is calculated based on two or more luma-chroma value pairs, or control points. In addition, the additional control points p r is calculated by using a different or the same set of reference luma-chroma pairs. For example, the average luma value of the set of reference luma values ​​is calculated for the control point p r Luma value of y r The average chroma value of the set of reference chroma values ​​can be used as r Chroma value c rThe set of reference chroma values ​​may include, as an example, all values ​​of the reconstructed boundary chroma samples immediately above and immediately to the left of the block. The set of reference luma values ​​may be the corresponding luma sample values, which may be obtained, for example, by interpolation, when the chroma and luma resolutions are different. The boundary chroma samples may include the samples immediately above and to the left of the block, but may also include an extension of their arrays. For example, the set of boundary samples above a block of width w may include w samples adjacent to the block, or 2*w samples, further including w additional samples in the same row of the first w samples immediately above the block. The updated offset value b' is then calculated by subtracting the reference luma value y from the reference luma value y. r and the reference chroma value cr can be calculated as follows: b'=cr-((a*y r )>>k)

[0174] Advantageously, a set of reference luma and chroma values ​​for calculating the gradient parameter a is c for generating the offset parameter b′. r and y r The set of reference luma and chroma values ​​for calculating the parameters is selected to be smaller than the set of reference luma and chroma values ​​for calculating the parameters. This selection allows the computational complexity of calculating the primary gradient parameter a to be kept low, while still allowing the linear gradient parameter c to be calculated. r and y r A relatively simple calculation of, e.g., c r averaging the reference chroma values ​​to produce y r by averaging the reference luma values ​​to produce c r This can be done more accurately by determining

[0175] The set of reference luma and chroma values ​​for computing the bias term b' of the linear model may advantageously be selected to be a superset of the reference luma and chroma values ​​used to compute the gradient parameter a of the linear model. This may reduce the computational complexity associated with determining the linear model, since already generated or fetched reference samples may be used in determining the bias term b', while improving the accuracy and stability of the offset parameter b', since a larger set of reference samples is made available to compute it.

[0176] Figures 6a to 6c show various choices for the reference samples R above block B. Figure 6a shows a set of reference samples selected immediately above the block. The example of Figure 6b shows an extension of the set of reference samples to further include an additional sample at the top right of the block, and Figure 6c shows an extension of the reference sample array by one sample to the left. Of course, other choices may be made, further including, for example, 2*w+1 samples in the set. A similar choice may be made on the left side of block B, and further a combination of samples or sample arrays from the left and top of the block may be used.

[0177] In one embodiment, the video or image decoder performs the following steps with reference to the flow diagram of FIG. 7a:

[0178] The decoder decodes the sign and magnitude of the initial update term (702). The decoder determines a value of the initial update term based on the decoded sign and magnitude of the initial update term (704). The decoder determines a reference value of a parameter of a luma-to-chroma mapping function (706), and the decoder determines the value of the update term by inverting the sign of the initial update term if the value of the reference parameter is greater than zero (708). The decoder further determines a value of the parameter of the luma-to-chroma mapping function using the reference value and the value of the update term (710).

[0179] In one embodiment, the video or image decoder performs the following steps with reference to the flow diagram of FIG. 7b:

[0180] The decoder calculates at least a reference value for a parameter of a luma-to-chroma mapping function (722). The decoder then decodes a magnitude of an update term u for the parameter of the luma-to-chroma mapping function (724). The decoder also decodes a binary syntax element to be used in determining the sign of the update term (726). The decoder determines the sign of the update term by interpreting the binary syntax element based on the reference value (728). The decoder also determines a value for the parameter of the luma-to-chroma mapping function using the reference value, the decoded magnitude of the update term, and the determined sign of the update term (730).

[0181] The calculation of the parameters that define the mapping from the values ​​of one color component to the values ​​of the other color components can be done in various ways. For example, linear regression or other statistical methods can be used to minimize the error that the model may create for the set of reference samples. The reference samples can be obtained from the boundaries of the predicted block or using another set of data. There may be multiple candidate reference sets, from which the encoder picks one and indicates the selection in the bitstream to the decoder. A subset of the boundary samples or other sample sets may also be used. In such a case, for example, there may be four sample pairs selected from the available reference samples, each sample pair consisting of a luma sample and a corresponding chroma sample. The two pairs with the lowest luma values ​​of the four pairs may form a first set, and the two pairs with the highest luma values ​​may form a second set. The luma values ​​of the first set may be averaged to form a first average luma value, and the chroma values ​​of the first set may be averaged to form a first average chroma value. Similarly, the second set of luma values ​​may be averaged to form a second average luma value, and the second set of chroma values ​​may be averaged to form a second average chroma value. Linear model parameters may be calculated from the two average luma values ​​and the two average chroma values, as in the H.266 / VVC standard.

[0182] The update term for the gradient value can be determined in various ways. For example, a video encoder can use a rate-distortion optimization technique to determine an appropriate update term for the gradient value. A video decoder can determine the update term for the gradient value by parsing it from the bitstream and predicting it, or by combining prediction and bitstream signaling, or by other means.

[0183] The update term for the gradient value can be scaled to the same basis as the calculated gradient value a, or the calculated gradient value a can be scaled to the same basis as the update term, or both values ​​can be scaled to a third basis, or scaling can be omitted. The third basis can be determined, for example, by the maximum allowed value of the basis used for the update term and the gradient parameter.

[0184] The update terms may be limited to a particular range. For example, the update terms may be defined to have integer values ​​ranging from -3 to 3, or from -4 to 4, or from -N to N, or from N to M, where N and M may be constants or may be determined by various means. For example, they may be determined based on characteristics of the block being predicted, coded, or decoded, such as the size of the block.

[0185] In one embodiment, there is a predefined set of update terms that are signaled in the bitstream and can be decoded by the decoder.

[0186] In one embodiment, the set of update terms that can be signaled in the bitstream and decoded by the decoder depends on the characteristics of the block of samples, such as the size of the block.

[0187] In one embodiment, the offset term b′ in the linear mode is: b′=c r -((a*y r )>>k), where c r is the average of the block's top border chroma values, and y r is the average of the upper boundary luma values ​​of the block, a is a gradient parameter, and k is a shifting parameter.

[0188] In one embodiment, the offset term b′ in the linear mode is: b′=c r -((a*y r )>>k), where c ris the average of the block's left border chroma values, and y r is the average of the left boundary luma values ​​of the block, a is a gradient parameter, and k is a shifting parameter.

[0189] In one embodiment, the offset term b′ in the linear mode is: b′=c r -((a*y r )>>k), where c r is the average of the block's left and top border chroma values, and y r is the average of the left and top boundary luma values ​​of the block, a is a gradient parameter, and k is a shifting parameter.

[0190] In one embodiment, the offset parameter b' is determined by r and y r The set of reference luma and chroma values ​​for calculating the parameters is selected to be greater than the set of reference luma and chroma values ​​for calculating the gradient parameter a.

[0191] In one embodiment, the offset parameter b' is determined by r and y r The set of reference luma and chroma values ​​for computing the parameters is selected to be a superset of the set of reference luma and chroma values ​​for computing the gradient parameter a.

[0192] The determination of the reference luma value or reference point may be performed in various ways. For example, the reference luma value may be the average of two luma values ​​used in determining the gradient parameters of the linear model; or may be a weighted average of such luma values; or may be determined in other ways.

[0193] There may be multiple linear models that form a mapping from luma values ​​to chroma values. For example, if the luma value is below a threshold, a first linear model may be used, and if the luma value is equal to or greater than the threshold, a second model may be used. In such a case, each of the linear models may receive their independent update terms based on bitstream signaling. Further, it may be signaled that only a certain subset of the linear models is updated using the signaled one or more update terms. Furthermore, the same update term may be determined to be used in multiple models based on bitstream signaling or other means. In such a case, a method of applying the signaled update term to the various models may be further determined. For example, the same update term may be used for two models, or a negative version of the update term may be used for one model and a positive version of the update term may be used for the other model.

[0194] Although the mapping process and parameter update process are described herein as having luma values ​​as inputs and chroma values ​​as outputs, the inputs and outputs are not limited to such color components. For example, the same process can be applied between chroma channels with one chroma channel, such as a Cb channel, as an input and another chroma channel, such as a Cr channel, as an output. As a further example, the input channel can be a luma channel and the output channel can be an auxiliary information channel, for example consisting of depth, distance, disparity, transparency, or other types of values.

[0195] According to one embodiment, a chroma block may also correspond to either the red, green, or blue color components of the RGB color space.

[0196] According to one embodiment, an apparatus includes: means for calculating at least two parameters defining a mapping from a first color component to a second color component, the at least two parameters including at least a gradient parameter and an offset parameter; means for determining an update term for the gradient parameter; means for applying the update term to the gradient parameter by adding the update term to a value of the gradient parameter to generate an updated gradient parameter; means for determining a reference value for the first color component; and means for calculating an updated offset parameter based on the reference value of the first color component and the updated gradient parameter.

[0197] As a further aspect, an apparatus is provided that has at least one processor and at least one memory, the at least one memory having code stored therein that, when executed by the at least one processor, causes the apparatus to at least: calculate at least two parameters defining a mapping from a first color component to a second color component, the at least two parameters including at least a gradient parameter and an offset parameter; determine an update term for the gradient parameter; apply the update term to the gradient parameter by adding the update term to a value of the gradient parameter to generate an updated gradient parameter; determine a reference value for the first color component; and calculate an updated offset parameter based on the reference value of the first color component and the updated gradient parameter.

[0198] Such an apparatus may, for example, comprise the functional units disclosed in any of Figures 1, 2, 4a and 4b for implementing the embodiments.

[0199] Such an apparatus further includes code stored in the at least one memory that, when executed by the at least one processor, causes the apparatus to perform one or more of the embodiments disclosed herein.

[0200] FIG. 8 is a graphical representation of an example multimedia communication system in which various embodiments may be implemented. A data source 1510 provides a source signal in an analog format, an uncompressed digital format, or a compressed digital format, or any combination of these formats. An encoder 1520 may include or be connected to pre-processing, such as data format conversion and / or filtering, of the source signal. The encoder 1520 encodes the source signal into a coded media bitstream. It should be noted that the bitstream to be decoded may be received directly or indirectly from a remote device located in virtually any type of network. Additionally, the bitstream may be received from local hardware or software. The encoder 1520 may be capable of encoding two or more media types, such as audio and video, or two or more encoders 1520 may be required to code source signals of different media types. The encoder 1520 may also obtain synthetically produced inputs, such as graphics and text, or produce coded bitstreams of synthetic media. In the following, only the processing of one coded media bitstream of one media type is considered for ease of explanation. However, it should be noted that in general, a real-time broadcast service includes several streams (generally at least one audio, video, and text subtitle stream). It should also be noted that while a system may include many encoders, in the figure only one encoder 1520 is shown for ease of explanation without loss of generality. It should be further appreciated that while the text and examples contained herein may specifically describe an encoding process, those skilled in the art should understand that the same concepts and principles apply to the corresponding decoding process and vice versa.

[0201] The coded media bitstreams may be transferred to storage 1530. Storage 1530 may include any type of mass memory for storing coded media bitstreams. The format of the coded media bitstreams in storage 1530 may be a basic self-contained bitstream format, or one or more coded media bitstreams may be encapsulated in a container file, or the coded media bitstreams may be encapsulated in a segment format suitable for DASH (or a similar streaming system) and stored as a sequence of segments. If one or more media bitstreams are encapsulated in a container file, a file generator (not shown) may be used to store the one or more media bitstreams in a file and create file format metadata, which may also be stored in the file. Encoder 1520 or storage 1530 may comprise a file generator, or the file generator may be operatively attached to either encoder 1520 or storage 1530. Some systems operate "live", i.e., omitting storage and forwarding the coded media bitstreams directly from the encoder 1520 to the sender 1540. The coded media bitstreams may then be forwarded to the sender 1540, also called a server, as needed. The format used for transmission may be a basic self-contained bitstream format, a packet stream format, a segment format suitable for DASH (or similar streaming systems), or one or more coded media bitstreams may be encapsulated in a container file. The encoder 1520, storage 1530, and server 1540 may reside on the same physical device or may be included in separate devices.The encoder 1520 and server 1540 may operate on live real-time content, in which case the coded media bitstream is generally not permanently stored, but rather buffered for short periods in the content encoder 1520 and / or server 1540 to smooth out processing delays, transmission delays, and fluctuations in the coded media bitrate.

[0202] The server 1540 sends the coded media bitstream using a communication protocol stack. The stack may include, but is not limited to, one or more of Real-time Transport Protocol (RTP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), Transmission Control Protocol (TCP), and Internet Protocol (IP). If the communication protocol stack is packet-oriented, the server 1540 encapsulates the coded media bitstream into packets. For example, when RTP is used, the server 1540 encapsulates the coded media bitstream within RTP packets according to an RTP payload format. Generally, each media type has its own RTP payload format. Note again that although the system may include more than one server 1540, for simplicity, the following description considers only one server 1540.

[0203] If the media content is encapsulated in a container file for storage 1530 or for inputting data to the sender 1540, the sender 1540 may include or be operatively attached to a "sending file parser" (not shown). In particular, if the container file is not so transmitted, but rather at least one of the contained coded media bitstreams is encapsulated for transport over a communication protocol, the sending file parser locates the appropriate portion of the coded media bitstream to be carried over the communication protocol. The sending file parser may also help create the correct format for the communication protocol, such as packet headers and payloads. The multimedia container file may include encapsulation instructions, such as hint tracks in ISOBMFF, to encapsulate at least one of the contained media bitstreams based on the communication protocol.

[0204] The server 1540 may or may not be connected to the gateway 1550 through a communication network, which may be, for example, a CDN, the Internet, and / or a combination of one or more access networks. The gateway may additionally or alternatively be referred to as a middlebox. In DASH, the gateway may be an edge server (of the CDN) or a web proxy. Note that the system may generally include any number of gateways, etc., but for simplicity, the following description considers only one gateway 1550. The gateway 1550 may perform various types of functions, such as converting packet streams according to one communication protocol stack to another, merging and forking data streams, and manipulating data streams according to downlink and / or receiver capabilities, such as controlling the bit rate of the forwarding stream according to the prevailing downlink network conditions. The gateway 1550 may be a server entity in various embodiments.

[0205] The system typically includes one or more receivers 1560 capable of receiving, demodulating, and deencapsulating the transmitted signal into a coded media bitstream. The coded media bitstream may be forwarded to a recording storage 1570. The recording storage 1570 may include any type of mass memory for storing the coded media bitstream. The recording storage 1570 may alternatively or additionally include a computation memory, such as a random access memory. The format of the coded media bitstream in the recording storage 1570 may be a basic self-contained bitstream format, or one or more coded media bitstreams may be encapsulated within a container file. When there are multiple coded media bitstreams, such as audio and video streams that are related to each other, a container file is typically used, and the receiver 1560 includes or is attached to a container file generator that creates a container file from the input streams. Some systems operate "live", i.e., omit the recording storage 1570 and forward the coded media bitstream directly from the receiver 1560 to the decoder 1580. In some systems, only the most recent portion of the recorded stream, for example only the most recent 10 minute excerpt of the recorded stream, is maintained in recording storage 1570, while older recorded data is discarded from recording storage 1570.

[0206] The coded media bitstreams may be transferred from the recording storage 1570 to the decoder 1580. If there are many coded media bitstreams, such as audio and video streams, related to each other and encapsulated in a container file, or if a single media bitstream is encapsulated in a container file, for example for easier access, a file parser (not shown) is used to decapsulate each coded media bitstream from the container file. The recording storage 1570 or the decoder 1580 may include the file parser, or the file parser is attached to either the recording storage 1570 or the decoder 1580. It should also be noted that while the system may include many decoders, only one decoder 1570 is discussed here for ease of explanation without loss of generality.

[0207] The coded media bitstream may be further processed by a decoder 1570, the output of which is one or more uncompressed media streams. Finally, a renderer 1590 may play the uncompressed media streams using, for example, loudspeakers or a display. The receiver 1560, recording storage 1570, decoder 1570, and renderer 1590 may reside on the same physical device or may be included in separate devices.

[0208] The sender 1540 and / or the gateway 1550 may be configured to switch between different representations, for example, for switching between different viewports of 360-degree video content, view switching, bitrate adaptation, and / or fast start-up, and / or the sender 1540 and / or the gateway 1550 may be configured to select the representation to be transmitted. Switching between different representations may occur for many reasons, such as in response to a request of the receiver 1560 or to prevailing conditions, such as the throughput of the network over which the bitstream is carried. In other words, the receiver 1560 may initiate the switch between representations. A request from the receiver may be, for example, a request for a segment or sub-segment with a different representation than before, a request for a change in the scalability layer and / or sub-layer to be transmitted, or a change in a rendering device with different capabilities compared to the previous one. A request for a segment may be an HTTP GET request. A request for a sub-segment may be an HTTP GET request with a byte range. Additionally or alternatively, bitrate adjustment or bitrate adaptation may be used, for example, to provide so-called fast start-up in streaming services, where the bitrate of the transmitted stream is lower than the channel bitrate at the start of streaming or after random access, in order to start playback immediately and to achieve a buffer occupancy level that tolerates occasional packet delays and / or retransmissions. Bitrate adaptation may involve multiple representation or layer up switching and representation or layer down switching operations occurring in various orders.

[0209] The decoder 1580 may be configured to switch between different representations, e.g., for switching between different viewports of 360-degree video content, view switching, bitrate adaptation, and / or fast start-up, and / or the decoder 1580 may be configured to select a transmitted representation. Switching between different representations may occur for many reasons, such as to achieve faster decoding operations or to adapt the transmitted bitstream, e.g., in terms of bitrate, to prevailing conditions, such as the throughput of the network over which the bitstream is carried. For example, faster decoding operations may be required when a device including the decoder 1580 is multitasking and uses computing resources for purposes other than decoding the video bitstream. In another example, faster decoding operations may be required when the content is played back at a faster pace than normal playback speed, e.g., two or three times faster than a conventional real-time playback rate.

[0210] In the above, some embodiments have been described with reference to and / or using HEVC and / or VVC terminology, it should be understood that the embodiments may be similarly implemented with any video encoder and / or video decoder.

[0211] In the above, where example embodiments are described with reference to an encoder, it should be understood that the resulting bitstream and decoder may have corresponding elements therein. Similarly, where example embodiments are described with reference to a decoder, it should be understood that the encoder may have a structure and / or computer program for generating a bitstream to be decoded by the decoder. For example, some embodiments are described in relation to generating a predictive block as part of the encoding. An embodiment may be similarly realized by generating a predictive block as part of the decoding, but with the difference that coding parameters such as horizontal and vertical offsets are decoded from the bitstream rather than being determined by the encoder.

[0212] The above-described embodiments of the invention describe the codec in terms of separate encoder and decoder devices to aid in understanding the processes involved. However, it will be understood that the devices, structures, and operations may be implemented as a single encoder-decoder device / structure / operation. Furthermore, it is possible that the coder and decoder may share some or all common elements.

[0213] Although the above examples describe embodiments of the invention operating within a codec in an electronic device, it will be appreciated that the invention as defined in the claims may be implemented as part of any video codec. Thus, for example, embodiments of the invention may be implemented in a video codec capable of implementing video coding over a fixed or wired communication path.

[0214] Thus, the user equipment may include a video codec such as those described in the embodiments of the present invention above. It will be appreciated that the term user equipment is intended to encompass any suitable type of wireless user equipment, such as a mobile phone, a portable data processing device, or a mobile web browser.

[0215] Additionally, elements of the public land mobile network (PLMN) may further include the video codecs mentioned above.

[0216] In general, various embodiments of the present invention may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. For example, certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, but the present invention is not limited thereto. Although various aspects of the present invention may be illustrated and described as block diagrams, flow charts, or using other graphical representations, it will be appreciated that these blocks, devices, systems, techniques, or methods described herein may be implemented in, by way of non-limiting examples, hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controller, or other computing device, or combinations thereof.

[0217] The embodiments of the present invention may be implemented by computer software executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Furthermore, in this regard, it should be noted that any block of logic flow as in the figures may represent program steps, or interconnected logic circuits, blocks, and functions, or a combination of program steps and logic circuits, blocks, and functions. The software may be stored on physical media, such as memory chips, or memory blocks implemented in a processor, magnetic media, such as hard disks or floppy disks, and optical media, such as DVDs and CDs of their data variants.

[0218] The memory may be of any type suitable for the local technology environment and may be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed and removable memories, etc. The data processor may be of any type suitable for the local technology environment and may include, by way of non-limiting examples, one or more of general purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs), and processors based on multi-core processor architectures.

[0219] Embodiments of the present invention may be practiced in a variety of components, such as integrated circuit modules. The design of integrated circuits is generally a highly automated process. Complex and powerful software tools are available to convert logic level designs into semiconductor circuit designs that are ready to be etched and formed into semiconductor substrates.

[0220] Programs such as those offered by Synopsys, Inc. of Mountain View, Calif., and Cadence Design of San Jose, Calif., use well-established design rules and libraries of pre-stored design modules to automatically route conductors and place components on a semiconductor chip. Once the design of a semiconductor circuit is complete, the resulting design in a standard electronic format (e.g., Opus, GDSII, etc.) can be sent to a semiconductor manufacturing facility or "fab" for production.

[0221] The foregoing description provides a complete and informative description of the exemplary embodiments of the present disclosure, by way of illustrative and non-limiting examples. However, various changes and modifications will become apparent to those skilled in the art in view of the foregoing description, when read in conjunction with the accompanying drawings and the appended claims. However, all such and similar modifications of the teachings of the present disclosure will still fall within the scope of the present disclosure.

Claims

1. determining reference values ​​for parameters of a mapping function from the first color component to the second color component; Decode the magnitude of the initial update term, Decoding syntax elements to be used in determining the signs of the update terms; determining the sign of the update term by interpreting the decoded syntax element based on the reference value; Determine values ​​for the parameters of the mapping function using the reference value, the decoded magnitude of the update term, and the determined sign of the update term. An apparatus comprising a means for:

2. said means for determining a sign Comparing the reference value to a threshold value; If the reference value is greater than the threshold and the syntax element has a first representation value, or if the reference value is less than or equal to the threshold and the syntax element has a second representation value different from the first representation value, determine the sign of the update term to be positive, or otherwise determine the sign of the update term to be negative. The apparatus of claim 1 , comprising:

3. 3. The apparatus of claim 2, wherein the first display value is zero and the second display value is one, or the first display value is one and the second display value is zero.

4. 4. The apparatus of claim 1, 2 or 3, wherein the threshold value is zero.

5. said means for determining a sign Determining the sign of the update term such that the reference value is refined toward zero when the decoded binary syntax element representing the sign of the update term is 0, and is refined away from zero when the decoded binary syntax element representing the sign of the update term is 1.

5. Apparatus according to claim 1, comprising means for:

6. said means for determining a sign Determining the sign of the update term such that the reference value is refined toward zero when the decoded binary syntax element representing the sign of the update term is 1, and is refined away from zero when the decoded binary syntax element representing the sign of the update term is 0.

5. Apparatus according to claim 1, comprising means for:

7. If the value of the reference parameter is greater than or equal to zero, the value of the update term is determined by reversing the sign of the initial update term.

7. Apparatus according to any one of claims 1 to 6, comprising means for:

8. If the value of the reference parameter is less than or equal to zero, determine the value of the update term by leaving the initial update term unchanged.

7. Apparatus according to any one of claims 1 to 6, comprising means for:

9. 9. The apparatus of claim 1, wherein the first color component is a luma component and the second color component is one chroma component.

10. determining reference values ​​for parameters of a mapping function from the first color component to the second color component; Decoding the magnitude of the initial update term; and Decoding syntax elements to be used in determining the signs of the update terms; and determining the sign of the update term by interpreting the decoded syntax element based on the reference value; determining values ​​of parameters of a mapping function using the reference value, the decoded magnitude of the update term, and the determined sign of the update term; The method includes:

11. An apparatus comprising at least one processor and at least one memory, the at least one memory having stored therein code that, when executed by the at least one processor, causes the apparatus to perform at least: determining reference values ​​of parameters of a mapping function from the first color component to the second color component; Decoding the magnitude of the initial update term; and Decoding syntax elements to be used in determining the signs of the update terms; and determining the sign of the update term by interpreting the decoded syntax element based on the reference value; determining values ​​of parameters of a mapping function using the reference value, the decoded magnitude of the update term, and the determined sign of the update term; A device that performs the above function.

12. determining reference values ​​for parameters of a mapping function from the first color component to the second color component; determining a value for an initial update term for the reference value; if the reference value is greater than the threshold, determining a displayed value of the update term such that the sign of the displayed value is opposite to the sign of the determined value of the update term; The sign and magnitude of the represented value of the update term are encoded into the video bitstream. An apparatus comprising a means for:

13. If the value of the reference parameter is greater than or equal to zero, the sign of the update term is reversed. The apparatus of claim 12 comprising means for:

14. Determines the displayed value of the update term by leaving the update term unchanged if the value of the reference parameter is less than or equal to zero 14. Apparatus according to claim 12 or 13, comprising means for:

15. performing a rate-distortion decision to determine values ​​of the update terms; Determine which updates are displayed based on the value of the update item 15. Apparatus according to claim 12, 13 or 14, comprising means for:

16. performing a rate-distortion determination to determine the value of the indicated update term; Determine the update term based on the displayed update term value 15. Apparatus according to claim 12, 13 or 14, comprising means for:

17. determining reference values ​​of parameters of a mapping function from the first color component to the second color component; determining a value for an initial update term of the reference value; if the reference value is greater than the threshold, determining a displayed value of the update term such that the sign of the displayed value is opposite to the sign of the determined value of the update term; encoding the sign and magnitude of the represented value of the update term into the video bitstream; The method includes:

18. An apparatus comprising at least one processor and at least one memory, the at least one memory having stored therein code that, when executed by the at least one processor, causes the apparatus to perform at least: determining reference values ​​of parameters of a mapping function from the first color component to the second color component; determining a value for an initial update term of the reference value; if the reference value is greater than the threshold, determining a displayed value of the update term such that the sign of the displayed value is opposite to the sign of the determined value of the update term; encoding the sign and magnitude of the represented value of the update term into the video bitstream; A device that performs the above function.

Images