Inter-component prediction of adaptive blocks

Abstract

A method and apparatus for encoding or decoding video are provided. The chroma component of a video block is reconstructed. The video block is coded in an intercoding mode, an inter-component prediction mode for the video block is determined, one or more syntactic elements are encoded for the video block, and the decoder side derives an inter-component prediction mode from a set of inter-component prediction modes for the one or more syntactic elements. The chroma component of the video block is encoded based on the reconstructed chroma component and the determined inter-component prediction mode.

Description

Technical Field

[0001] (Claim of Priority) This application claims priority to European Patent Application No. 23315190.1 filed on May 11, 2023, and European Patent Application No. 23305991.4 filed on Jun. 22, 2023, the entire contents of which are incorporated herein by reference.

[0002] (Field of the Invention) This embodiment generally relates to video compression. This embodiment relates to methods and apparatuses for encoding or decoding images or videos. More particularly, this embodiment relates to improving intercoding.

Background Art

[0003] To achieve high compression efficiency, coding schemes for images and videos typically use prediction and transformation that exploit the spatial and temporal redundancy of video content. Generally, intra or inter prediction is used to exploit intra or inter picture correlation, and then, often referred to as prediction error or prediction residue, the difference between the original block and the predicted block is transformed, quantized, and entropy coded. In inter prediction, the motion vectors used in motion compensation are often predicted from motion vector predictors. To reconstruct the video, the compressed data is decoded by inverse processes corresponding to entropy coding, quantization, transformation, and prediction.

Summary of the Invention

[0004] According to one embodiment, a method for encoding video is provided. The method includes: reconstructing the luma component of a video block, wherein the video block is coded in an intercoding mode; determining an inter-component prediction mode for the video block; encoding one or more syntactic elements for the video block, wherein the one or more syntactic elements provide for the decoder to derive an inter-component prediction mode from a set of inter-component prediction modes; and encoding the chroma component of the video block based on the reconstructed luma component and the determined inter-component prediction mode.

[0005] In another embodiment, a device for encoding video is provided. The device comprises one or more processors capable of: reconstructing the luma component of a video block, wherein the video block is coded in an intercoding mode; determining an inter-component prediction mode for the video block; encoding one or more syntactic elements for the video block, wherein the one or more syntactic elements provide for the decoder to derive an inter-component prediction mode from a set of inter-component prediction modes; and encoding the chroma component of the video block based on the reconstructed luma component and the determined inter-component prediction mode.

[0006] In another embodiment, a method for decoding video is provided. This method includes decoding one or more syntactic elements for a block of video, wherein the video block is coded in an intercoding mode; determining an intercomponent prediction mode for the video block from a set of intercomponent prediction modes using one or more syntactic elements; reconstructing the luma components of the video block; and reconstructing one or more chroma components of the video block from the reconstructed luma components using the intercomponent prediction modes.

[0007] In another embodiment, a device for decoding video is provided. The device comprises one or more processors capable of decoding one or more syntactic elements for a block of video, wherein the video block is coded in an intercoding mode; determining an intercomponent prediction mode for the video block from a set of intercomponent prediction modes using one or more syntactic elements; reconstructing the luma components of the video block; and reconstructing one or more chroma components of the video block from the reconstructed luma components using the intercomponent prediction modes.

[0008] Further embodiments that may be used individually or in combination are described herein.

[0009] One or more embodiments also provide a computer program that, when executed by one or more processors, causes one or more processors to perform a method for encoding / decoding video according to any of the embodiments described herein. One or more embodiments also provide a non-temporary computer-readable medium and / or computer-readable storage medium storing instructions for encoding / decoding video according to the method described herein.

[0010] One or more embodiments also provide a computer-readable storage medium storing a bitstream generated according to the methods described herein. One or more embodiments also provide a method and apparatus for transmitting or receiving a bitstream generated according to the methods described above. [Brief explanation of the drawing]

[0011] [Figure 1] A block diagram of a system in which an embodiment of this model may be implemented is shown. [Figure 2] A block diagram of an embodiment of a video encoder in which an aspect of this embodiment may be implemented is shown. [Figure 3] A block diagram of an embodiment of a video decoder in which an aspect of this embodiment may be implemented is shown. [Figure 4] An example of the sample location used to derive the α and β parameters of the intercomponent linear model is shown. [Figure 5] This shows the effect of the slope adjustment parameter "u". The left side shows the model created using standard CCLM, and the right side shows the model updated with slope adjustment. [Figure 6] An example of the spatial part of a convolutional filter in a convolutional component model is shown. [Figure 7] Examples of reference regions and their padding used to derive filter coefficients for inter-component models are shown. [Figure 8] Here are four examples of Sobel-based gradient patterns for gradient linear models. [Figure 9] An example of the spatial portion of a gradient linear CCCM convolutional filter is shown. [Figure 10] An example of the spatial part of a non-downsampled CCCM Luma term used in a convolutional filter is shown. [Figure 11] An example of a downsampling filter applied to a luma sample is shown. [Figure 12] This document presents an example of an inter-CU prediction and reconstruction process using inter-component prediction. [Figure 13] Examples of luma samples L0, ..., L5 related to chroma sample C are shown. [Figure 14] An example flowchart of a method for encoding video blocks according to one embodiment is shown. [Figure 15] An example flowchart of a method for decoding a video block according to one embodiment is shown. [Figure 16] An example flowchart of a method for determining the intercoding mode and inter-component prediction mode for encoding a video block is shown, according to the first modification. [Figure 17] An example flowchart of a method for encoding video blocks, based on the first modification, is shown. [Figure 18]An example flowchart of a method for decoding the syntax associated with the video block, using the first variation, is shown. [Figure 19] An example flowchart of a method for reconstructing a video block using the first modification is shown. [Figure 20] A second variation shows an example flowchart of a method for decoding the syntax related to the video block. [Figure 21] A flowchart example of a method for reconstructing a video block is shown using a second variation. [Figure 22] A flowchart example of a method for reconstructing a video block using a third modification is shown. [Figure 23] A flowchart example of a method for reconstructing a video block using a fourth modification is shown. [Figure 24] A block diagram of a system in which an aspect of this embodiment may be implemented is shown according to another embodiment. [Figure 25] This example illustrates two remote devices communicating via a communication network based on this principle. [Figure 26] The syntax of a signal is shown using an example of this principle. [Modes for carrying out the invention]

[0012] This application describes various embodiments, including tools, features, embodiments, models, and approaches. Many of these embodiments are described in detail, often in a manner that sounds restrictive, at least to illustrate individual features. However, this is for clarity and not to limit the uses or scope of these embodiments. In fact, all of the different embodiments can be combined and interchangeable to provide further embodiments. Furthermore, these embodiments can also be combined with or interchangeable with embodiments described in previous applications.

[0013] The embodiments described and intended in this application can be realized in many different forms. Figures 1, 2, and 3 below provide some embodiments, but other embodiments are intended, and the considerations in Figures 1, 2, and 3 are not intended to limit the scope of implementation forms. At least one of these embodiments generally relates to video encoding and decoding, and at least one other embodiment generally relates to transmitting a generated or encoded bitstream. These and other embodiments can be realized as a computer-readable storage medium storing instructions for encoding or decoding video data according to any of the methods, apparatus, or described methods, and / or a computer-readable storage medium storing a bitstream generated according to any of the described methods.

[0014] In this application, the terms “reconstructed” and “decoded” may be used interchangeably, the terms “pixel” and “sample” may be used interchangeably, and the terms “image,” “picture,” and “frame” may be used interchangeably.

[0015] Various methods are described herein, each of which includes one or more steps or actions to achieve the described method. Unless a particular order of steps or actions is required for the normal operation of the method, the order and / or use of any particular steps and / or actions may be modified or combined. Furthermore, terms such as “first,” “second,” etc., may be used in various embodiments to modify elements, components, steps, actions, etc., such as “first decryption” and “second decryption.” The use of such terms does not imply a modified order of actions unless specifically required. Thus, in this example, the first decryption does not need to be performed before the second decryption, and may occur, for example, before, during, or over a period of overlap with the second decryption.

[0016] These embodiments are not limited to VVC or HEVC and can be applied, for example, to other standards and recommendations, whether prior to exist or to be developed in the future, as well as to any extensions of such standards and recommendations (including VVC and HEVC). Unless otherwise specifically indicated or technically excluded, the embodiments described herein can be used individually or in combination.

[0017] Figure 1 shows a block diagram of an example of a system in which various embodiments and forms can be implemented. System 100 may be embodied as a device comprising various components described below and configured to perform one or more of the embodiments described in this application. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. The elements of System 100 may be embodied individually or in combination as a single integrated circuit, a plurality of ICs, and / or individual components. For example, in at least one embodiment, the processing and encoder / decoder elements of System 100 are distributed across a plurality of ICs and / or individual components. In various embodiments, System 100 is communicably coupled to other systems or other electronic devices, for example, via a communication bus or via dedicated input and / or output ports. In various embodiments, System 100 is configured to implement one or more of the embodiments described in this application.

[0018] System 100 includes at least one processor 110, which is configured to execute instructions loaded therein to realize, for example, various embodiments described in this application. The processor 110 may include embedded memory, input / output interfaces, and various other circuits known in the art. System 100 includes at least one memory 120 (e.g., a volatile memory device and / or a non-volatile memory device). System 100 includes a storage device 140 which may include non-volatile memory and / or volatile memory, including, but not limited to, EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash, magnetic disk drives, and / or optical disk drives. The storage device 140 may, in non-limiting examples, include an internal storage device, a removable storage device, and / or a network-accessible storage device.

[0019] System 100 includes, for example, an encoder / decoder module 130 configured to process data to provide encoded or decoded video, the encoder / decoder module 130 may include its own processor and memory. The encoder / decoder module 130 represents a module that may be included in a device that performs encoding and / or decoding functions. As is known, the device may include one or both of the encoding module and the decoding module. Furthermore, the encoder / decoder module 130 may be implemented as a separate element of System 100 or may be incorporated into the processor 110 as a combination of hardware and software, as is known to those skilled in the art.

[0020] To perform the various embodiments described in this application, program code to be loaded into the processor 110 or encoder / decoder 130 may be stored in the storage device 140 and subsequently loaded onto the memory 120 for execution by the processor 110. According to various embodiments, one or more of the processor 110, memory 120, storage device 140, and encoder / decoder module 130 may store one or more of various items during the execution of the processes described in this application. Such stored items may include, but are not limited to, input video, decoded video or a portion of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, expressions, operations, and arithmetic logic.

[0021] In some embodiments, the internal memory of the processor 110 and / or the encoder / decoder module 130 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, external memory of the processing device (for example, the processing device can be either the processor 110 or the encoder / decoder module 130) is used for one or more of these functions. The external memory can be memory 120 and / or storage device 140, and can be, for example, dynamic volatile memory and / or non-volatile flash memory. In some embodiments, external non-volatile flash memory is used to store the television's operating system. In at least one embodiment, high-speed external dynamic volatile memory such as RAM is used as working memory for video coding and decoding operations such as MPEG-2 (MPEG stands for Moving Picture Experts Group, also known as ISO / IEC 13818, 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC stands for High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, a standard developed by JVET, i.e., Joint Video Experts Team).

[0022] Inputs to the elements of system 100 can be provided through various input devices, as shown in block 105. Such input devices include, but are not limited to, (i) a radio frequency (RF) section for receiving RF signals wirelessly transmitted by, for example, a broadcasting station; (ii) a component (COMP) input terminal (or a set of COMP input terminals); (iii) a Universal Serial Bus (USB) input terminal; and / or (iv) a High Definition Multimedia Interface (HDMI) input terminal. Another example not shown in Figure 1 is composite video.

[0023] In various embodiments, the input device of block 105 has associated input processing elements as known in the art. For example, the RF section may be associated with elements suitable for (i) selecting a desired frequency (also called selecting a signal or band-limiting a signal to a certain frequency band), (ii) down-converting the selected signal, (iii) again band-limiting to a narrower frequency band so as to select a signal frequency band that may (for example) be called a channel in certain embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section of various embodiments includes one or more elements that perform these functions, for example, a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a downconverter, a demodulator, an error corrector, and a demultiplexer. The RF section may include a tuner that performs various of these functions, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a frequency close to the baseband) or to the baseband. In one embodiment of a set-top box, the RF unit and its associated input processing elements perform frequency selection by receiving, filtering, down-converting, and filtering again to a desired frequency band of RF signals transmitted via a wired (e.g., cable) medium. In various embodiments, the order of these (and other) elements is rearranged, some of these elements are removed, and / or other elements performing similar or different functions are added. Adding elements may include inserting elements between existing elements, for example, an amplifier and an analog-to-digital converter. In various embodiments, the RF unit includes an antenna.

[0024] Furthermore, the USB and / or HDMI terminals may include their respective interface processors for connecting the system 100 to other electronic devices over USB and / or HDMI connections. It should be understood that various aspects of input processing, such as Reed-Solomon error correction, may be performed, for example, in a separate input processing IC or within the processor 110, as needed. Similarly, aspects of USB or HDMI interface processing may be implemented in a separate interface IC or within the processor 110, as needed. For example, demodulated, error-corrected, and demultiplexed streams may be provided to various processing elements, including a processor 110 and an encoder / decoder 130, which may, as needed, process the data stream for display on an output device.

[0025] Various elements of system 100 may be provided within an integrated housing, where the various elements are interconnected and data can be transmitted between them using an internal bus known in the art, such as a suitable connection arrangement 115, including an I2C bus, wiring, and a printed circuit board.

[0026] System 100 includes a communication interface 150 that enables communication with other devices via a communication channel 190. The communication interface 150 may include, but is not limited to, a transceiver configured to send and receive data via the communication channel 190. The communication interface 150 may also include, but is not limited to, a modem or a network card, and the communication channel 190 may be implemented, for example, in a wired and / or wireless medium.

[0027] In various embodiments, data is streamed to system 100 using a Wi-Fi network such as IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). In these embodiments, the Wi-Fi signal is received via a communication channel 190 and a communication interface 150 adapted for Wi-Fi communication. In these embodiments, the communication channel 190 is typically connected to an access point or router that provides access to an external network, including the Internet, to enable streaming of the application and other over-the-top communication. In other embodiments, the streamed data is provided to system 100 using a set-top box that distributes data via an HDMI connection of input block 105. Still other embodiments provide the streamed data to system 100 using an RF connection of input block 105. As described above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0028] System 100 can provide output signals to various output devices, including a display 165, a speaker 175, and other peripheral devices 185. In various embodiments, the display 165 includes, for example, one or more of a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display 165 may be for a television, tablet, laptop, mobile phone, or other device. The display 165 may also be integrated with other components (e.g., a smartphone) or be separate (e.g., an external monitor for a laptop). In various examples of embodiments, the other peripheral devices 185 include one or more of a standalone digital video disc (or digital multi-purpose disc) (both terms DVR), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 185 that provide functions based on the output of System 100. For example, a disc player performs the function of playing back the output of System 100.

[0029] In various embodiments, control signals are communicated between the system 100 and the display 165, speaker 175, or other peripheral devices 185 using signaling such as AV Link, CEC, or other communication protocols that enable device-to-device control with or without user intervention. Output devices may be communicably coupled to the system 100 via dedicated connections through their respective interfaces 160, 170, and 180. Alternatively, output devices may be connected to the system 100 using a communication channel 190 via a communication interface 150. The display 165 and speaker 175 may be integrated into a single unit with other components of the system 100 in an electronic device such as a television. In various embodiments, the display interface 160 includes a display driver, such as a timing controller (TCon) chip.

[0030] The display 165 and speaker 175 may, alternatively, be separate from one or more of the other components, for example, if the RF portion of input 105 is part of a separate set-top box. In various embodiments where the display 165 and speaker 175 are external components, the output signal may be provided via a dedicated output connection, for example, including an HDMI port, a USB port, or a COMP output.

[0031] The embodiments can be implemented by computer software implemented by the processor 110, by hardware, or by a combination of hardware and software. In a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory 120 can be of any type appropriate for the technical environment and, in a non-limiting example, can be implemented using any appropriate data storage technology such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. The processor 110 can be of any type appropriate for the technical environment and, in a non-limiting example, can include one or more of microprocessors, general-purpose computers, dedicated computers, and processors based on multi-core architectures.

[0032] Figure 2 shows an example of a block-based hybrid video encoder 200. While variations of this encoder 200 are intended, the encoder 200 is described below for clarity without explaining all anticipated variations.

[0033] In some embodiments, Figure 2 also shows encoders that have been improved upon the HEVC or VVC standard (Multipurpose Video Coding, standard ITU-TH.266, ISO / IEC23090-3, 2020), or encoders that employ HEVC or VVC-like technologies, such as the ECM encoder currently under development by JVET (Joint Video Exploration Team).

[0034] Before encoding, the video sequence may undergo pre-encoding (201), for example, applying a color conversion to the input color picture (e.g., from RGB4:4:4 to YCbCr4:2:0), or remapping the input picture components to obtain a more compression-resilient signal distribution (e.g., using histogram equalization of color components), or resizing (e.g., downscaling) the picture. Metadata may be associated with the pre-processing and attached to the bitstream.

[0035] In encoder 200, the picture is encoded by encoder elements as described below. The encoded picture is divided (202) and processed in units of, for example, coding units (CUs). Different expressions may be used in this disclosure to refer to such units or blocks resulting from the division of the picture. Such terms may be coding units or CUs, coding blocks or CBs, luminance CBs, or blocks. A coding tree unit (CTU) may refer to a group of blocks or a group of units. In some embodiments, a CTU may be considered a block or a unit in itself.

[0036] Each unit is encoded using either intra-mode or inter-mode, for example. If the unit is encoded in intra-mode, intra-prediction is performed (260). In inter-mode, motion estimation (275) and motion compensation (270) are performed. The encoder decides whether to use intra-mode or inter-mode to encode the unit (205), and indicates the intra / inter decision, for example, by a prediction mode flag. The encoder may also blend the intra-prediction results and the inter-prediction results, or blend the results from different intra / inter-prediction methods (263). The prediction residual is calculated, for example, by subtracting the predicted blocks from the original image blocks (210).

[0037] The motion enhancement module (272) uses already available reference pictures to enhance a block's motion field without referencing the original block. The motion field of a region can be thought of as the set of motion vectors for all pixels in that region. If the motion vectors are subblock-based, the motion field can also be represented as the set of all subblock motion vectors in the region (all pixels in a subblock have the same motion vector, and the motion vectors may differ for each subblock). If a single motion vector is used for a region, the region's motion field can also be represented by a single motion vector (the same motion vector for all pixels in the region).

[0038] Next, the predicted residuals are transformed (225) and quantized (230). In addition to the quantized transformation coefficients, the motion vector and other syntactic elements are entropy coded to output a bitstream (245). The encoder can skip the transformation and apply quantization directly to the untransformed residual signal. The encoder can bypass both the transformation and quantization, i.e., the residuals are coded directly without applying any transformation or quantization process.

[0039] The encoder decodes the encoded blocks and provides a reference for further prediction. The quantized transformation coefficients are inversely quantized (240), inversely transformed (250), and the prediction residuals are decoded. The decoded prediction residuals and the prediction blocks are combined (255) to reconstruct the image blocks. An in-loop filter (265) is applied to the reconstructed picture to reduce encoding artifacts, for example, by performing deblocking / SAO (Sample Adaptive Offset) filtering. The filtered image is stored in a reference picture buffer (280).

[0040] Figure 3 shows a block diagram of the video decoder 300. In the decoder 300, the bitstream is decoded by the decoder elements as described below. The video decoder 300 generally performs a decoding path that is the reverse of the encoding path described in Figure 2. The encoder 200 also generally performs video decoding as part of encoding the video data.

[0041] Specifically, the input to the decoder includes a video bitstream, which may be generated by a video encoder 200. First, the bitstream is entropy-decoded (330) to obtain transformation coefficients, motion vectors, and other coded information. Picture partitioning information indicates how the picture is partitioned. Thus, the decoder may partition the picture according to the decoded picture partitioning information (335). The transformation coefficients are inversely quantized (340), inversely transformed (350), and the predicted residuals are decoded. The decoded predicted residuals and predicted blocks are combined (355) to reconstruct the image blocks.

[0042] The prediction block can be obtained from intra-prediction (360) or motion-compensated prediction (i.e., inter-prediction) (375) (370). The decoder may blend the intra-prediction result and the inter-prediction result, or blend the results from multiple intra / inter-prediction methods (373). Before motion compensation, the motion field may be improved by using an already available reference picture (372). An in-loop filter (365) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (380).

[0043] The decoded picture may further undergo post-decoding (385), such as reverse color transformation (e.g., conversion from YCbCr4:2:0 to RGB4:4:4), or reverse remapping, which is the reverse of the remapping performed in the pre-encoding process (201), or resizing (e.g., upscaling) the reconstructed picture. The post-decoding process may use metadata derived in the pre-encoding process and signaled in the bitstream.

[0044] Some of the embodiments described herein relate to inter-prediction, and more specifically to adaptive inter-component prediction for intercoded blocks.

[0045] Any one of the embodiments described herein can be implemented, for example, in an interpretation module of a video encoder or video decoder. Any one of the embodiments described herein can be implemented, for example, in a motion compensation module 270 of a video encoder 200, or in a motion compensation module 375 of a video decoder 300.

[0046] For the intercomponent linear model (CCLM) for intra-prediction, please refer to ECM 7 (M. Coban, F. Le Leannec, RL. Liao, K. Naser, J. Stroem, L. Zhang, "Algorithm description of Enhanced Compression Model 7 (ECM 7)", JVET-AB2025, 28th conference held remotely, October 2022). In CCLM, chroma samples are predicted based on reconstructed chroma samples of the same CU by using a linear model as follows: pred_C(i,j)=α·rec_L'(i,j)+β (Equation 1) In the formula, pred_C(i,j) represents the predicted chroma sample in the CU, and rec_L(i,j) represents the downsampled and reconstructed chroma sample in the same CU.

[0047] The CCLM parameters (α and β) are derived from adjacent chroma samples (the upper row and the left column) and their corresponding downsampled luma samples (LM mode).

[0048] In a variant, up to four adjacent chroma samples are used. In another variant, the adjacent chroma samples are selected only from above (LM-A) or only from the left (LM-L), and the selected mode is signaled to the decoder.

[0049] The selected adjacent luma samples at the selected position are downsampled and compared to find two smaller values, namely, x 0 A and x 1 A and two larger values, namely, x 0 B and x 1 B Their corresponding chroma sample values are shown as y 0 A 、y 1 A 、y 0 B 、and y 1 B Then, X a 、X b 、Y a 、and Y b are derived as follows. X a =(x 0 A + x 1 A +1)>>1 X b =(x 0 B + x 1 B +1)>>1 Y a =(y 0 A + y 1 A +1)>>1 Y b =(y 0B + y 1 B +1)>>1

[0050] Finally, the linear model parameters α and β are obtained by the following equations.

[0051]

number

[0052] Figure 4(400) shows the sample Rec for the luma component on the left and above. L , and the chroma component of the left and top samples Rec c Examples of the positions are shown (the samples on the left and top are shown as circles in Figure 4), and the current 2N×2N block sample is involved in the CCLM mode.

[0053] In another variation, three multi-model LM (MMLM) modes are added. In each MMLM mode, the reconstructed neighboring samples are classified into two classes using a threshold that is the mean of the reconstructed neighboring samples. A linear model for each class is derived using the Least-Mean-Square (LMS) method. In the CCLM mode, the LMS method is also used to derive the linear model.

[0054] In another variation, slope adjustment is applied to intercomponent linear models (CCLM) and multi-model LM predictions. The adjustment involves sloping the linear function that maps the lumens to the chromens with respect to a center point determined by the mean lumens of the reference samples, as shown in Figure 5(500). On the left is the model created with a normal CCLM with parameters a and b, and on the right is the model updated with the slope adjustment parameter u, Y t This is the average luma value of the reference sample.

[0055] In ECM7, the Convolutional-Cross-Convolutional Model (CCCM) is also performed for intra-prediction. The CCCM predicts the chroma sample from the reconstructed luma sample in a similar manner to that performed by CCLM. When chroma subsampling is used, the reconstructed luma sample is downsampled to match a lower resolution of the chroma grid.

[0056] Additionally, there is the option of using a single-model or multi-model variation of CCCM. The multi-model CCCM mode can be selected for blocks with at least 128 available reference samples. The multi-model variation uses two models: one applied to samples above the mean Luma reference value, and another for the remaining samples (in accordance with the spirit of the CCLM design).

[0057] CCCM uses 7 input samples {s i} i=0,.6 A convolutional filter consisting of seven weighted parameters is used. Five coefficients are applied to the luminance pixel values ​​corresponding to the plus-sign shape, one coefficient is applied to the squared term (P), and the last coefficient is applied to the bias term (B). The input to the five spatial tap components of the filter consists of a central (C) chroma sample located at the same position as the predicted chroma sample, and its adjacent samples to the top / north (N), bottom / south (S), left / west (W), and right / east (E), as shown in Figure 6(620).

[0058] Term P is expressed as a power of 2 of the central sample C and scaled to the sample value range of the content. P=(C * C+midVal)>>bitDepth Here, midVal is a rounding term. That is, for 10 bits of content, it is calculated as follows: P=(C * C+512)>>10

[0059] The bias term B (similar to the offset term in CCLM) represents a scalar offset between the input and output and is set to the intermediate chroma value (512 for 10-bit content).

[0060] The output of the filter is the filter coefficient c. i It is calculated as a convolution between the input value and clipped to the range of valid chroma samples. predChromaVal=c0C+c1N+c2S+c3E+c4W+c5P+c6B (Formula 2)

[0061] Filter coefficient c i This is calculated by minimizing the mean squared error (MSE) between the predicted chroma sample and the reconstructed chroma sample in the reference region (710). Figure 7(710) shows a reference region consisting of six rows / columns of chroma samples above and to the left of the block / PU. The reference region extends 1 PU width to the right of the PU boundary and 1 PU height below. The region is adjusted to include only available samples. The extension to the reference region, shown in blue, is necessary to support the "side samples" of the plus-shaped spatial filter and is padded when they are in the unavailable region.

[0062] MSE minimization is performed by calculating the autocorrelation matrix of the Luma input and the cross-correlation vector between the Luma input and the Chroma output.

[0063]

number

[0064] coefficient {c i To derive}, the autocorrelation matrix is ​​inverted. For example, the autocorrelation matrix is ​​LDL T The decomposition is performed, and the final filter coefficients are calculated using back substitution. This process roughly follows the calculation of ALF filter coefficients in ECM, but LDL is used instead of Cholesky decomposition to avoid the use of square root operations. TWe chose decomposition (also known as alternative Cholesky decomposition). Another variation involves inverting the matrix using Gaussian elimination, as in the VVC standard. In ECM, the calculations use 64-bit integer arithmetic.

[0065] ECM7 also provides gradient linear models. For the YUV 4:2:0 color format, the gradient linear model (GLM) method can be used to predict chroma samples from chroma sample gradients. Two modes are supported: m2-parameter GLM mode and 3-parameter GLM mode.

[0066] Compared to CCLM, the two-parameter GLM utilizes the luma sample gradient to derive a linear model, instead of downsampled luma values. Specifically, when a two-parameter GLM is applied, the input to the CCLM process, i.e., the downsampled luma sample L, is replaced by the luma sample gradient G. Other parts of the CCLM (e.g., parameter derivation, linear transformation of predictive samples) remain unchanged. C = α·G + β

[0067] In a 3-parameter GLM, chroma samples can be predicted based on both the luma sample gradient and downsampled luma values ​​with different parameters. The model parameters for the 3-parameter GLM are derived from six row and column adjacent samples by an LDL decomposition-based MSE minimization method, similar to that used in CCCM. C = α0·G + α1·L + α2·β

[0068] Regarding signaling, when the CCLM mode is enabled for the current CU, one flag is signaled to indicate whether the GLM is enabled for both the Cb and Cr components. If the GLM is enabled, another flag is signaled to indicate which of the two GLM modes is selected, and one more syntactic element is further signaled to select one of four gradient filters for gradient calculation.

[0069] As shown in Figure 8, four gradient filters are enabled for the GLM.

[0070] ECM7 also provides a gradient and position-based convolutional component model (GL-CCCM). This is a variation of CCCM in which convolution is applied to gradient and position information instead of the four spatially adjacent samples in the CCCM filter, as in VVC. This variation is shown in Figure 9(930). The GL-CCCM filter for prediction is as follows:

[0071] predChromaVal=c0C+c1Gy+c2Gx+c3Y+c4X+c5P+c6B (Equation 3) In the formula, Gy and Gx are the vertical and horizontal slopes of Luma, respectively, and are calculated as follows (see 930 in Figure 9). Gy=(2N+NW+NE)-(2S+SW+SE) Gx = (2W + NW + SW) - (2E + NE + SE)

[0072] Furthermore, the Y and X parameters are the vertical and horizontal coordinates of the central luma sample position.

[0073] The remaining parameters are the same as those used in the CCCM tool. The reference region for parameter calculation is the same as that used in the CCCM method.

[0074] In another modification, the reconstructed luma samples are not downsampled to match a low-resolution chroma grid, and the six reconstructed luma samples at the same location are used directly, as shown in Figure 10. In this modification, four terms constructed from L0, L1, L2, and L4 with bias B (VVC) added are also used. In this case, the number of parameters is 10.

[0075] In another variation, multiple (e.g., four) downsampling filters can be used to derive coefficient-weighted input samples. The downsampling filter model is signaled per CU, and the chroma sample predictions are derived as follows: Model 1: predChroma=c0 * H(C)+c1 * G1(C)+c2 * G2(C)+c3 * G3(C)+c4 * G4(C)+c5 * P+c6 * B Model 2: predChroma=c0 * H(C)+c1 * H(W)+c² * H(E)+c³ * G1(C)+c4 * G1(W)+c5 * G1(E)+c6 * B Model 3: predChroma=c0 * H(C)+c1 * H(N)+c² * H(S) + c³ * G2(C)+c4 * G2(N)+c5 * G2(S)+c6 * B Model 4: predChroma=c0 * H(C)+c1 * H(NE)+c² * H(SW)+c3 * G4(C)+c4 * G4(NE)+c5 * G4(SW)+c6 * B Here, H(·), G1(·), G2(·), G3(·), and G4(·) are downsampling filters applied to the luma sample, as shown in Figure 11(1150).

[0076] K. Zhang, L. Zhang, Z. Deng, Chia-Ming Tsai, Hsin-Yi Tseng, Cheng-Yen Chuang, Chih-Wei Hsu, Ching-Yeh Chen, Tzu-Der Chuang, Olena Chubach, Yi-Wen Chen, Yu-Wen Huang, Shaw-Min Lei, "EE2-1.6: Non-local cross-component prediction and cross-component merge mode," document JVET-AD0188, presented at the 30th conference held remotely from April 21-28, 2023.

[0077] In ECM, for intrablocks, the selection of the CU level of the inter-component prediction mode from luma to chroma is enabled, and several syntactic elements (LMCMode flag, cclmFlag, mmlmFlag, cccmFlag, cccmNoSubFlag, glCccmFlag, glmFlag) are enabled. Therefore, JVET-AD0188 proposes that it is possible to derive the inter-component prediction (CCP) used for a given block from an already coded block within the same picture.

[0078] A list of CCP merge candidates is constructed. For example, the CCP merge candidate list may include spatially adjacent candidates, spatially non-adjacent candidates, or history-based candidates. After including these candidates, default models are further included to fill any remaining empty positions in the merge list. Pruning operations are applied to remove redundant CCP models from the list. After constructing the list, the CCP models in the list are sorted according to the SAD cost obtained using the adjacency template of the current block. Further details are provided below.

[0079] Spatially adjacent and non-adjacent candidates: The positions and inclusion order of spatially adjacent and non-adjacent candidates are the same as those defined in the ECM for normal intermergency prediction candidates.

[0080] History-based candidates: The history-based table is maintained to include recently used CCP models, and the table is reset at the beginning of each CTU row. If the current list is not full after including spatially adjacent and non-adjacent candidates, CCP models in the history-based table are added to the list.

[0081] Default Candidates: CCLM candidates with default scaling parameters are considered only if the list is not full after including spatially adjacent candidates, spatially non-adjacent candidates, or history-based candidates. If the current list does not contain any candidates with a single-model CCLM mode, the default scaling parameter is {0, 1 / 8, -1 / 8, 2 / 8, -2 / 8, 3 / 8, -3 / 8, 4 / 8, -4 / 8, 5 / 8, -5 / 8, 6 / 8}. Otherwise, the default scaling parameter is {0, scaling parameter of the first CCLM candidate + {1 / 8, -1 / 8, 2 / 8, -2 / 8, 3 / 8, -3 / 8, 4 / 8, -4 / 8, 5 / 8, -5 / 8, 6 / 8}}. The offset parameter is derived according to the default scaling parameter, the mean adjacent reconstructed CCLM sample value, and the mean adjacent reconstructed Cb / Cr sample value.

[0082] A flag is signaled to indicate whether CCP merge mode is applied. If CCP merge mode is applied, an index is signaled to indicate which candidate model is used by the current block. In addition, CCP merge mode is not permitted for the current chromacoding block if the current CU is coded by an intra-subpartition (ISP) with a single tree, or if the current chromacoding block size is 16 or less.

[0083] In P. Astola and J. Lainema (Nokia), "AHG12: Cross-component residual model (CCRM) for inter prediction," reference JVET-AD0108, April 2023, an inter-component residual model (CCRM) for inter prediction is provided. An eight-tap filter consisting of six spatial luma samples, a nonlinear term, and a bias term is used to predict chroma using only the luma channels of the intercoded block. The spatial luma samples (L0,...,L5) are obtained from a luma grid that selects the six luma samples closest to chroma position C without downsampling, as shown in Figure 13. Figure 13 shows the luma samples L0,...L5 related to chroma sample C in a half-per-luma grid.

[0084] The predicted chroma values ​​are obtained as follows: predChromaVal = c0L0 + c1L1 + c2L2 + c3L3 + c4L4 + c5L5 + c6nonlinear((L0+L3+1)>>1) + c7B, where nonlinear is the nonlinear operator of CCCM and B is the bias.

[0085] The filter coefficients are derived using the undivided Gaussian elimination method of the ECM, and the required offset is applied to the sample before the filter is derived.

[0086] When a block has fewer than 64 chroma samples, intra-reference samples are used as additional input samples in the filter derivation. A CCCM design of up to 6 rows and columns of the intra-reference samples is used.

[0087] Blocks containing more than 256 chroma samples are divided into subblocks containing a maximum of 256 chroma samples. Subblocks containing zero chroma residuals are skipped.

[0088] CCRM tools are currently used as an alternative to temporal chroma prediction (i.e., prediction using motion-compensated chroma) which is commonly used in interblock prediction of chroma components. A limitation of the CCRM approach in JVET-AD0108 is that a single CCP method is used to predict chroma from chroma in interblocks, while several possible CCP methods are permitted for intrablocks.

[0089] As a result, the compression performance of the interblock may be limited due to this lack of flexibility in the CCP within the interblock.

[0090] The object of the embodiments provided herein is to introduce more flexibility in the selection of the CCP method for interblocking in order to improve the compression efficiency of modern video codecs.

[0091] In some embodiments, the interblock CCP is improved by introducing the possibility of using a CCP method for interblocks (hereinafter also referred to as a CCP mode) among a variety of CCP methods. In this specification, the term “block” may also be referred to as a predictive unit (PU) or coding unit (CU), as is commonly used in video coding standards or implementations.

[0092] Figure 14 shows an example of a method (1400) for encoding a video block according to one embodiment. In this embodiment, the video block is coded using intermode or interprediction, and is therefore referred to as interblock below. In 1401, the luma portion of the interblock under consideration is predicted and reconstructed according to the interprediction mode determined for the interblock. For example, a reference block is determined in a reference picture using motion compensation for the interblock with motion data determined for the interblock. On the encoder side, in 1402, one CCP mode is selected for the interblock from among various CCP modes. The CCP method selected for the interblock is then signaled in the bitstream encoding the video (1403). This signaling can be explicit (i.e., in the form of syntactic elements used to identify the CCP mode) or implicit (i.e., in the form of syntactic elements used on the decoder side to derive the CCP mode for the current interblock). In some embodiments, only the use of a CCP mode is signaled, and the selected CCP mode is determined in the same manner on the encoder and decoder side, for example, using an interblock template (reconstructed sample surrounding the interblock) or an interblock reference block. In 1404, the chroma portion of the interblock is encoded based on the selected CCP mode, along with the interblock data such as the interprediction mode and chroma data.

[0093] Figure 15 shows an example of a method (1500) for decoding a video block according to one embodiment. On the decoder side, the syntactic elements associated with the video block are parsed (1501), i.e., decoded from the bitstream. In this embodiment, the video block is coded using intermode or interprediction. Therefore, it will be referred to as an interblock below. The parsed syntactic elements for the interblock are used to identify the CCP mode to be used for that interblock. In some embodiments, only the use of a CCP mode is signaled, and the selected CCP mode is determined in the same way on both the encoder and decoder sides, for example, using an interblock template or reference block. In 1502, the CCP mode to be used for a given interblock is derived from the decoded syntactic elements. According to the CCP mode used for the current interblock, the luma portion of the interblock under consideration is reconstructed (1503), the chroma portion (Cb block and Cr block) of the interblock is predicted, and reconstructed from the reconstructed luma block (1504).

[0094] Some variations of the embodiments provided above include the following embodiments which can be used individually or in combination:

[0095] When an interblock is coded in AMVP mode, the use of a CCP mode and the selected CCP mode are explicitly signaled along with the motion data. For example, AMVP mode is an interpredictive coding mode as known in the VVC standard. More generally, AMVP mode is an interpredictive coding mode in which motion vector differences (mvd) for motion predictors are coded. Motion predictors can be signaled in the bitstream, for example, by an index indicating a motion predictor candidate from a list of motion candidates.

[0096] In another variation, if the interblock is coded in merge mode, the interblock's CCP mode can be derived on the decoder side in the same way as the interblock's motion data derivation process. For example, merge mode is an interpredictive coding mode, as known in the VVC standard. More generally, merge mode is an interpredictive coding mode in which the motion vector difference (mvd) for motion predictors is assumed to be zero. Motion predictors can be signaled in the bitstream, for example, by an index indicating motion predictor candidates from a merge candidate list.

[0097] The merge candidate list is used together to derive motion information and CCP mode information from block to block, for example, as designed in existing video codecs. Thus, the merge_idx syntax element of an interblock is used for both motion information derivation and CCP mode derivation. If an interblock inherits the motion of a merge candidate, it also inherits the CCP method of that merge candidate.

[0098] Alternatively, a separate merge index dedicated to CCP mode derivation is signaled at the PU or CU level.

[0099] In another variation, CCP mode derivation can also be applied to temporal merge derivation. When TMVP (temporal motion vector predictor) or SbTMVP (subblock temporal motion vector predictor) motion vectors are inherited, the associated CCP mode may also be inherited from the temporal candidate. For example, TMVP and SbTMVP motion vectors can be obtained in the same way as in the VVC standard or ECM implementation.

[0100] In another variation, the CCP mode derivation may only apply when the interblock is in skip mode. In such a case, the interblock may inherit the CCP mode from the merge candidate selected in the candidate list, as well as the motion information.

[0101] In another modification, the CCP mode derivation may only be applicable if the merge candidates used to derive motion data correspond to spatially adjacent blocks of the current interblock. In that modification, non-adjacent merge candidates may not be used to derive the CCP method for a given interblock.

[0102] In another variation, the CCP mode selected for an interblock may be one that minimizes the distortion between the filtered predicted luma block and the predicted chroma block. In other words, the CCP mode does not need to be signaled because the same process is performed on both the encoder and decoder sides to select the CCP mode. This distortion is determined between the temporally predicted chroma portion of the interblock and the prediction of the chroma portion of the interblock from the temporally predicted luma portion of the current interblock using the candidate CCP modes. The CCP mode that yields the least distortion among all candidate CCP modes is selected on both the encoder and decoder sides.

[0103] In another variation, CCP mode inheritance may further include the inheritance of CCP filtering parameters in addition to the CCP prediction mode itself.

[0104] In further modifications, these CCP parameters may be inherited only if the current interblock is in skip mode. In further modifications, these CCP parameters may be inherited only if a spatially adjacent merge candidate is used in the derivation.

[0105] In another variation, the set and number of available CCP modes may depend on the intermode, for example, whether the intermode is AMVP, Merge, TMVP, or SbTMVP.

[0106] In another variation, the CCP modes can be sorted using the current interblock template. For each CCP mode candidate (or subset thereof), the CCP mode is used to predict a reconstructed sample of the template, and the cost of the CCP mode is determined on this template. The CCP mode cost is used to sort the CCP mode candidates.

[0107] In some embodiments, among a plurality of possible inter-component modes, an inter-component prediction mode is signaled at the block level for the interblock. This plurality of inter-component modes may include all or some of the following modes further described above, namely CCLM, MMLM, CCCM, GLM, and GLCCCM.

[0108] Regarding existing video codec implementations, such as the coding unit syntax modifications for the ECM 7 under consideration, this signaling can take the form shown in Table 1 below.

[0109] As can be seen, in the case of interCU, several syntactic elements are added to the existing bitstream syntax to identify the inter-component prediction mode used for a given interCU. The added elements are shown in bold in Table 1.

[0110] In this embodiment, for each CU, the flag inter_ccp_mode_flag is signaled to indicate the use of a CCP mode to predict the chromacoding block of the CU under consideration. When this flag is on, further syntactic elements may be signaled to enable the video decoder to identify the CCP mode actually used for the current CU. This takes the form of the inter_ccp_mode() syntactic structure shown in Table 1 below.

[0111] [Table 1-1]

[0112] [Table 1-2]

[0113] The function CcpAllowed(x0,y0) determines whether cross-component predictions are allowed for the current block (x0,y0). For example, it checks whether certain conditions are met from the cross-predictions that are allowed for the current block (x0,y0).

[0114] Table 2 below shows examples of possible syntactic arrangements for indicating the CCP mode and CCP parameters used to code a given interblock among known intercomponent prediction methods (CCLM, MMLM, MDLM, GLM, GLCCCM).

[0115] [Table 2-1]

[0116] [Table 2-2]

[0117] In Table 2 above, the semantics of the syntactic elements are as follows: The `cclm_flag` syntax element is similar to the `cclm_mode_flag` element in the VVC specification, but here it is used for interblocks. It indicates whether or not the CCLM cross-prediction mode is used to code and decode the current block.

[0118] ccp_mode_idx indicates the type of CCLM mode used for the current interblock, as well as the type of adjacent sample used to derive the intercomponent predictive linear model between the reconstructed left, top, and top-left samples surrounding the current block.

[0119] cccmAllowed(x0,y0) is a function that determines whether CCCM mode is enabled for the current block. Typically, CCCM utilizes a selected set of neighboring samples to derive filter parameters. This neighborhood is the same as that indicated by the ccp_mode_idx syntax element. Therefore, CCCM mode is typically permitted when there are enough neighboring samples available in the indicated neighborhood region to derive CCCM filter parameters. Note that this embodiment is not limited to such rules, and any other CCM enabling rules may apply.

[0120] The cccm_flag indicates the use of CCCM mode for the current interblock.

[0121] The `cccm_no_sub_sampling_flag` flag indicates the use of CCCM mode, which does not perform chroma downsampling to predict chroma samples.

[0122] The gl_cccm_flag flag indicates the use of the GL-CCCM crossover prediction mode for the current interblock.

[0123] GlmAllowed(x0,y0) is a function that determines whether GLM mode is allowed for the current interblock. Typically, GLM is allowed when CCCM is not used and the color format of the video being coded / decoded is 420.

[0124] Glm_flag indicates the use of GLM inter-component prediction for the current interblock.

[0125] Glm_idx indicates the gradient filter used for gradient calculation during GLM component-to-component prediction.

[0126] cclm_delta_flag indicates the use of CCLM tilt adjustment for the current interblock in the case of interblocks that do not use CCCM instead of GLM.

[0127] Cclm_cb0_flag indicates the use of slope adjustment in the linear model used for inter-component prediction of the Cb component.

[0128] cclm_cr0_flag indicates the use of slope adjustment in the linear model used for inter-component prediction of the Cr component.

[0129] cclm_cb0_delta_idx represents the slope adjustment parameter in the linear model used for inter-component prediction of the Cb component.

[0130] cclm_cr0_delta_idx represents the slope adjustment parameter in the linear model used for inter-component prediction of the Cr component.

[0131] Next, in the case of multi-model CCLM mutual prediction mode, the following flags may exist in the bitstream.

[0132] Cclm_cb1_flag indicates the use of slope adjustment in the second linear model used for inter-component prediction of the Cb component.

[0133] cclm_cr1_flag indicates the use of slope adjustment in the second linear model used for inter-component prediction of the Cr component.

[0134] Cclm_cb1_delta_idx, if present, indicates the slope parameter in the second linear model used for inter-component prediction of the Cb component.

[0135] cclm_cr1_delta_idx, if present, indicates the slope parameter in the second linear model used for inter-component prediction of the Cr component.

[0136] An example of encoder method 1600 for selecting the CCP mode for interCU is given by Figure 16. This is in a double embedding loop (1601, 1602, 1604, 1605) over all interprediction modes (1601) supported by the codec (skip mode, merge, AMVP, merge affine, AMVP affine, etc.) and over all supported possible intercomponent prediction modes (1602) (including CCLM, MMLM, CCCM, GLM, GLCCCM, and possibly others). inter and intercomponent prediction mode ccpm inter For each pair, the RD cost is evaluated for the CU (1603). A set of inter-prediction modes and inter-component prediction modes that provide the minimum rate distortion cost is selected (1606). In some embodiments, non-inter-modes can also be evaluated for the CU (1607). In 1608, the CU is then selected together with the inter-prediction modes m inter and intercomponent prediction mode ccpm inter It is compressed and encoded using [this method].

[0137] An example of the interCU entropy coding method (1700) according to this embodiment is shown in Figure 17. The input to this method is the CU to be coded, whose coding mode and coding parameters are selected by the RD optimization stage of the encoder.

[0138] In 1701, the CU prediction mode is coded into the output bitstream. Then, in 1702, it is checked whether the CU is intercoded or intracoded. In the case of intra-CU, it is not addressed in the embodiments provided herein and is therefore not shown here. In 1703 (CU is intercoded), the inter-component prediction information used to code the current interCU is coded. For example, the bitstream syntax proposed in Table 1 above can be used. In 1704 and 1705, the selected CU intermode information and associated motion data for the current CU are coded. In 1706, the CU residual block is coded and the process is completed.

[0139] The bitstream analysis process of the interCUs corresponding to the encoding process in Figure 17 is shown in Figure 18. In 1801, the prediction mode is analyzed. Then, if interprediction is used (yes in 1802), the CCP mode selected for the CU under consideration is analyzed (1803), and the CCP mode ccpm can be equivalent to no CCP (meaning the Cb / Cr component is predicted according to normal interprediction), CCLM, CCCM, GLM, GLCCM, and any other possible intercomponent prediction modes. inter Provides a modified version, CCP mode ccpm inter This is an index indicating a CCP mode within a set of possible CCP modes. In this variation, a particular value of the index indicates that no CCP mode is used for a given CU.

[0140] In another variation, as shown in Table 1, for example, a flag first signals whether or not CCP mode is used, and if the flag indicates so, the index is then signaled.

[0141] The decoded CCP information may also be present in the parameters associated with the CCP mode being considered. For example, if the CCP mode is CCLM, some slope information of the linear model may also be decoded.

[0142] In steps 1804 and 1805, the interpretation mode and associated motion data are decoded for the current CU and CU residual (1806).

[0143] Figure 19 shows an example of a CU decoding and reconstruction process 1900 used to reconstruct a CU based on the analyzed data issued from the process in Figure 18. In 1901, it is determined whether the current interCU is coded in merge mode. If not, in 1902, an AMVP candidate list is constructed, and in 1903, the motion data of the current CU is reconstructed from the selected candidates in the AMVP list and the decoded motion residuals. If the CU is coded in merge, in 1904, a merge candidate list is constructed, and in 1905, the motion data of the current CU is reconstructed from the selected candidates in the merge list. These steps above may be similar to, for example, the corresponding steps in a VVC or ECM codec implementation. In 1906, the CU is predicted from the reference block using the motion data derived in the previous step.

[0144] In this embodiment, the intercomponent prediction mode ccpm is used in the interCU. inter This is specified by a decoded CCP mode index issued from the newly introduced syntactic element inter_ccp_mode() introduced in this disclosure (Tables 1 and 2). The CCP modes shown in the bitstream may consist of, for example, CCLM mode, MMLM mode, CCM mode, GLM mode, and GLCCM mode. In 1907, the inter-component mode parameter (e.g., the tap for CCLM mode) is then determined between the predicted luma block and the predicted chroma block determined in 1906. Then, in 1908, the luma is reconstructed (by adding residuals to the prediction), and in 1909, the reconstructed luma and CCP mode ccpm inter This is used to predict the chroma block in which CCP will be performed. In 1910, the chroma Cb / Cr block is reconstructed by adding the residuals and the predicted chroma block.

[0145] Further embodiments relating to inter-component prediction mode inheritance for interCU are provided below.

[0146] According to another embodiment, the inter-component modes used in the interCU may be inherited from several other already coded or decoded CUs.

[0147] To do so, a list of candidate CCP modes can be constructed for the current CU to be decoded, based on already decoded CUs. The candidate list may consist of CCP modes for spatially adjacent candidates, non-adjacent candidates, history-based candidates from the current picture, and, if applicable, for temporal candidates from the same collated picture typically used for temporal motion data inheritance. Thus, CCP mode derivation can also be applied to temporal merge derivation. In this case, merge_idx would indicate that the CCP mode of the collated interblock of the current block is used to derive the CCP mode of the current interblock, if available in the collated picture. Note that the collated block can be determined, typically for TMVP or SbTMVP merge candidates of VVC or ECM, similar to what has already been done for temporal merge candidate derivation.

[0148] To identify the CCP mode selected for the current CU, a dedicated ccp_merge_idx syntax element can be used to indicate the candidate CCP selected in the list.

[0149] In a modified version, this merge CCP signaling mode can be used within the merge mode and affine merge mode of the video codec under consideration.

[0150] On the other hand, in the case of AMVP interCUs, explicit signaling of the CCP mode for the current CU is applied.

[0151] The advantage of this embodiment is that, compared to the embodiment provided above, it further enhances compression efficiency by reducing the signaling cost of CCP mode information.

[0152] An example of the corresponding syntax table for CU prediction data is given in Table 3, with added elements shown in bold.

[0153] [Table 3-1]

[0154] [Table 3-2]

[0155] Bitstream analysis of CU prediction and CCP data corresponding to this embodiment (2000) is given by Figure 20. In 2001, the CU prediction mode is decoded, and in 2002, it is checked whether the CU is coded in inter-mode. Here, only interCU is considered. As can be seen, different analyses of the CCP information are performed based on whether the CU is in merge mode (2003). In AMVP mode, the same analyses (2006, 2007, 2008, 2009) as in the embodiment provided with reference to Figure 18 are applied. In merge mode, the CCP merge index is analyzed (2004), and the normal merge index is used to derive the CU motion data (2005). In 2009, the CU residual is decoded.

[0156] An example of the CU decoding and reconstruction process (2100) based on the motion and CCP data analyzed in Figure 20 is provided in Figure 21. In 2101, it is determined whether the current interCU is coded in merge mode. If not, in 2102, an AMVP candidate list is constructed according to the coded standard or implementation, and in 2103, the motion data of the current CU is reconstructed from the selected candidates in the AMVP list and the decoded motion residuals. In 2104, the CCP mode for the interCU is derived from the analyzed CCP mode index.

[0157] If the CU is coded in merge mode, in 2105 a merge candidate list is constructed. In 2106, the CU motion data is derived from the candidates selected in the merge candidate list using the decoded merge index of the CU. In this embodiment, in 2107 a CCP merge candidate list is constructed for the interCU, and in 2108 the CCP mode is derived from the candidates selected from the CCP merge candidate list according to the analyzed CCP merge index.

[0158] The subsequent steps 2109-2113 for reconstructing the current CU are the same as the steps in Figure 19.

[0159] The following provides some variations of the above embodiment.

[0160] The CCP mode derivation can only be applied when the CU is in skip mode. A CU in skip mode is equivalent to a CU coded in merge mode, except that its associated residual block is not coded and is equal to 0. In such cases, the CU inherits the CCP mode from the merge candidate selected in the candidate list, as well as the motion information.

[0161] In another variation, the CCP mode selected for the CU is the CCP mode that minimizes the distortion between the filtered predicted luma block and the temporally predicted chroma block. In this variation, signaling of the CCP mode used for the interCU is not required, and there is no need to introduce a new syntax for signaling the CCP mode and parameters.

[0162] In another modification, a single candidate list is constructed on both the encoder and decoder sides, with each candidate in the list consisting of motion data and CCP mode information.

[0163] In this modification, no new merge index ccp_merge_idx syntactic element is used; instead, the single merge index already used for motion data is also used to inherit the CCP mode information. Such modifications have the advantage of improving coding efficiency, as shown in Figure 22. The steps in Figure 22 are similar to those in Figure 21, except that in Figure 22, no CCP merge candidate list is constructed, and the same merge candidate list determined for motion data in 2205, along with the parsed merge index, is used to derive the CCP mode in 2207.

[0164] In another variation, not only the CCP mode but also the CCP prediction parameters (e.g., CCCM filtering parameters, CCLM slope parameters, etc.) are derived from past CUs indicated by the merge index in the candidate list. In this case, the calculated CCP parameters are stored at the CU level and can be reused for decoding subsequent CUs.

[0165] This modification is illustrated by the exemplary CU decoding and reconstruction process 2300 in Figure 23. The advantage of this modification is that it saves computation by avoiding the calculation of CCP prediction parameters from the predicted luma and chroma blocks for each CU. In this modification, CCP parameters are calculated only when the CU is coded in AMVP mode, and when the CU is in merge mode, the CCP parameters are derived from CCP candidates.

[0166] Another advantage is the ability to apply CCPs to interblocks coded in merge-skip mode, which has not been achieved in prior art. In fact, by inheriting the merge candidate CCP mode and parameters, skip-mode CUs, i.e., CUs without associated residual blocks, can still potentially benefit from improved chroma prediction through the proposed merge mechanism extended to CCP mode and parameters.

[0167] In step 2301, it is determined whether the current interCU is coded in merge mode. If not, in step 2302, an AMVP candidate list is constructed according to the coded standard or implementation, and in step 2303, the motion data of the current CU is reconstructed from the selected candidates in the AMVP list and the decoded motion residuals. In step 2304, the CU is predicted using the derived motion data and reference picture. In step 2305, the CCP mode is derived from the analyzed CCP mode index, and the CCP parameters are determined from the predicted CU.

[0168] If the CU is coded in merge mode, in 2306, a merge candidate list is constructed. In 2307, the CU motion data is derived from the candidates selected in the merge candidate list using the decoded merge index of the CU. In this embodiment, in 2308, the CCP mode is derived from the merge index also used for the merge motion. In another modification, the CCP mode may be derived from the CCP mode candidates selected from the CCP merge candidate list using the analyzed CCP mode index.

[0169] In any of these variations, in 2309, the CCP prediction parameters for the CCP mode determined in 2308 are derived from candidates indicated by a merge index or CCP merge index, depending on the variation used. In 2310, the CU is predicted using the derived motion data and reference picture, as in 2304.

[0170] The subsequent steps 2311–2314 for reconstructing the current CU are similar to the corresponding steps in Figure 19 or Figure 22.

[0171] In the modified version, these CCP parameters can only be inherited if spatially adjacent merge candidates are used in the derivation.

[0172] In another modification, the derivation of the CCP mode from a selected candidate in the CCP merge candidate list may only apply if the merge candidate used to derive motion data corresponds to a spatially adjacent block of the current block. In that modification, non-adjacent merge candidates may not be used to derive the CCP mode of a given interblock.

[0173] In another variation, the CCP mode derivation may only be applicable when the CU is in skip mode.

[0174] In the modified embodiment described above for CCP mode merge derivation, the proposed CCP mode signaling and derivation in merge mode are performed only for interCUs (i.e., blocks coded in intermode) and not for blocks or coding units coded in IBC (intrablock copy) coding mode.

[0175] Alternatively, the proposed CCP mode signaling and derivation in merge mode in any of the embodiments described above may be permitted for both inter-case and IBC cases.

[0176] Figure 24 shows a block diagram of a system in which an aspect of this embodiment may be implemented according to another embodiment. Figure 24 shows one embodiment of a device 2400 for encoding or decoding video according to any one of the embodiments described herein. The device comprises a processor 2410 which can be interconnected to a memory 2420 via at least one port. Both the processor 2410 and the memory 2420 may also have one or more additional interconnections to external connections.

[0177] Processor 2410 is also configured using any one of the embodiments described herein. For example, Processor 1220 is configured using any one of the embodiments described herein to decode one or more syntactic elements of a video block, the video block is coded in intercoding mode, to derive an inter-component prediction mode of the video block from one or more syntactic elements, to reconstruct the luma components of the video block, and to reconstruct one or more chroma components of the video block from the reconstructed luma components using the inter-component prediction mode. For example, Processor 2410 is configured using a computer program product that includes code instructions to implement any one of the embodiments described herein.

[0178] In other embodiments, the processor 2410 is configured to use any one of the embodiments described herein to reconstruct the luma component of a video block, to encode the video block in intercoding mode, to determine an inter-component prediction mode for the video block, to encode one or more syntactic elements for the video block, to derive an inter-component prediction mode on the decoder side from the one or more syntactic elements, and to encode the chroma component of the video block based on the reconstructed luma component and the determined inter-component prediction mode. For example, the processor 2410 is configured using a computer program product that includes code instructions to implement any one of the embodiments described herein.

[0179] In the embodiment shown in Figure 25, in a transmission context between two remote devices A and B via a communication network NET, device A comprises a processor associated with memory RAM and ROM configured to perform a method for encoding video as described with reference to Figures 1-24, and device B comprises a processor associated with memory RAM and ROM configured to perform a method for decoding video as described in reference to Figures 1-24. For example, the network is a broadcast network and is adapted to broadcast / transmit coded video from device A to decoding devices including device B.

[0180] Figure 26 shows an example of the syntax of a signal transmitted via a packet-based transmission protocol. Each transmission packet P includes a header H and a payload PAYLOAD. In some embodiments, the payload PAYLOAD may include video data encoded by any one of the embodiments described above.

[0181] Various implementations involve decoding. As used in this application, “decoding” can encompass all or part of the processes performed on a received encoded sequence to produce, for example, a final output suitable for display. In various embodiments, such processes include one or more of the processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processes also, or alternatively, include processes performed by the decoder in the various implementations described in this application, such as entropy decoding a sequence of binary symbols to reconstruct image or video data.

[0182] As further examples, in one embodiment, “decoding” refers only to entropy decoding; in another embodiment, “decoding” refers only to differential decoding; in yet another embodiment, “decoding” refers to a combination of entropy decoding and differential decoding; and in yet another embodiment, “decoding” refers to the entire picture reconstruction process, including entropy decoding. Whether the phrase “decoding process” is intended to refer specifically to a subset of operations or generally to a broader decoding process will become clear from the context of the particular description and will be well understood by those skilled in the art.

[0183] Various implementations involve encoding. As with the above considerations regarding "decoding," "encoding" as used in this application may encompass all or part of the processes performed on the input video sequence to generate an encoded bitstream. In various embodiments, such processes typically include one or more processes performed by the encoder, e.g., segmentation, differential encoding, transformation, quantization, and entropy encoding. In various embodiments, such processes also, or alternatively, include processes performed by the encoder in the various implementations described in this application, e.g., determining resampling filter coefficients and resampling the decoded picture.

[0184] As further examples, in one embodiment, “encoding” refers only to entropy coding; in another embodiment, “encoding” refers only to differential coding; and in yet another embodiment, “encoding” refers to a combination of differential coding and entropy coding. Whether the phrase “encoding process” is intended to refer specifically to a subset of operations or to a broader encoding process in general will become clear from the context of the particular description and will be well understood by those skilled in the art.

[0185] It should be noted that the syntactic elements used herein are descriptive terms; therefore, they do not preclude the use of other syntactic element names.

[0186] This disclosure describes various types of information that can be transmitted or stored, such as syntax. This information can be packaged or configured in various ways, including methods common in video standards, such as including the information in SPS, PPS, NAL units, headers (e.g., NAL unit headers or slice headers), or SEI messages. Other methods are also available, including methods common to system-level or application-level standards, such as including the information in one or more of the following: a. SDP (Session Description Protocol), a format for describing multimedia communication sessions for the purpose of session announcement and session invitation, such as being described in an RFC and used in conjunction with RTP (Real-time Transport Protocol) transmission. b. For example, a DASH MPD (Media Presentation Description) descriptor used in DASH and transmitted over HTTP, where the descriptor is associated with a representation or set of representations to provide additional characteristics to the content representation. c. RTP header extensions, such as those used during RTP streaming. d. An ISO-based media file format that uses boxes, which are object-oriented building blocks defined by a unique type identifier and length, as used in OMAF, for example, and also known as "atoms" in some specifications. e. An HLS (HTTP Live Streaming) manifest sent via HTTP. The manifest can, for example, be associated with a version of content or a set of versions of content, and can provide characteristics of the version or set of versions.

[0187] When a diagram is presented as a flow chart, it should be understood that it also provides a block diagram of the corresponding device. Similarly, when a diagram is presented as a block diagram, it should be understood that it also provides a flow chart of the corresponding method / process.

[0188] Several embodiments refer to rate-distortion optimization. In particular, during the coding process, a balance or trade-off between rate and distortion is usually considered, often given the constraints of computational complexity. Rate-distortion optimization is usually formulated to minimize a rate-distortion function, which is a weighted sum of rate and distortion. There are different methods for solving rate-distortion optimization problems. For example, these methods may be based on extensive testing of all coding options, including all considered mode or coding parameter values, but with a complete evaluation of their coding costs, as well as the associated distortions of the reconstructed signals after coding and decoding. In particular, faster approaches can be used to reduce coding complexity by calculating approximate distortion based on the predicted or predicted residual signal rather than the reconstructed signal. These two approaches can also be combined, for example, by using approximate distortion for only some of the possible coding options and full distortion for others. Other approaches evaluate only a subset of the possible coding options. More generally, many methods employ one of various techniques to perform optimization, but the optimization is not necessarily a complete evaluation of both coding costs and associated distortions.

[0189] The implementations and embodiments described herein can be implemented, for example, in methods or processes, apparatus, software programs, data streams, or signals. Even if considered only in the context of a single form of implementation (e.g., considered only as a method), the implementations of the considered features can also be realized in other forms (e.g., apparatus or programs). Apparatus may be implemented, for example, in appropriate hardware, software, and firmware. These methods may be implemented, for example, in a processor, which refers to processing devices in general, including, for example, computers, microprocessors, integrated circuits, or programmable logic devices. Processors also include communication devices, such as computers, mobile phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate the communication of information between end users.

[0190] The terms "one embodiment" or "one embodiment," or "one implementation" or "one implementation," and any other variations thereof, mean that the specific features, structures, characteristics, etc., described in relation to the embodiments are included in at least one embodiment. Therefore, the appearance of phrases such as "in one embodiment" or "in a particular embodiment," or "in one implementation" or "in a particular implementation," and any other variations, found in various places throughout this application, do not necessarily all refer to the same embodiment.

[0191] Additionally, this application may refer to "determining" various types of information. Determining information may include, for example, one or more of the following: estimating information, calculating information, predicting information, or retrieving information from memory.

[0192] Furthermore, this application may also refer to “accessing” various types of information. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0193] Additionally, this application may refer to “receiving” various types of information. Receiving is intended to be a broad term, similar to “accessing.” Receiving information may include, for example, accessing information or retrieving information (for example, from memory). Furthermore, “receiving” typically accompanies, in some way, operations such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0194] For example, in the cases of "A / B," "A and / or B," and "at least one of A and B," the use of any of the following " / ," "and / or," and "at least one of" should be understood as intended to cover the selection of only the first option (A), only the second option (B), or both options (A and B). As a further example, in the cases of "A, B, and / or C" and "at least one of A, B, and C," such phrasing is intended to cover the selection of only the first option (A), only the second option (B), only the third option (C), only the first and second options (A and B), only the first and third options (A and C), only the second and third options (B and C), or all three options (A, B, and C). This may be extended as many times as there are listed items, as will be obvious to those skilled in the art.

[0195] Furthermore, as used herein, the word “signaling” refers, in particular, to pointing something to a corresponding decoder. In this way, in one embodiment, the same parameter is used on both the encoder and decoder sides. For example, the encoder can transmit a specific parameter to the decoder (explicit signaling), and as a result, the decoder can use the same specific parameter. Conversely, if the decoder already has a specific parameter as well as other parameters, signaling can be used without transmission (implicit signaling), allowing the decoder to easily recognize and select the specific parameter. Bit savings are achieved in various embodiments by avoiding the transmission of any actual function. It should be seen that signaling can be achieved in various ways. For example, one or more syntactic elements, flags, etc., can be used to signal information to the corresponding decoder in various embodiments. While this concerns the verb form of the word “signaling,” the word “signaling” can also be used as a noun in this specification.

[0196] As will be apparent to those skilled in the art, the implementations can generate various signals, for example, that are formatted to carry information that can be stored or transmitted. The information may include, for example, instructions for performing a method, or data generated by one of the implementations described. For example, a signal may be formatted to carry a bitstream of the embodiment described. Such a signal may be formatted, for example, as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. The signal may be transmitted over various different wired or wireless links, as is known. The signal may be stored on a processor-readable medium.

[0197] Several embodiments have been described above. Features of these embodiments may be provided individually or in any combination across various claim categories and types.

Claims

1. It is a method, Decoding one or more syntactic elements for a video block, wherein the video block is coded in intercoding mode, Using one or more syntactic elements, determine the inter-component prediction mode for the video block from the set of inter-component prediction modes, Reconstructing the luma component of the aforementioned video block, A method comprising: reconstructing one or more chroma components of the video block from the reconstructed chroma components using the inter-component prediction mode.

2. A device comprising one or more processors, wherein the one or more processors Decoding one or more syntactic elements for a video block, wherein the video block is coded in intercoding mode, Using one or more syntactic elements, determine the inter-component prediction mode for the video block from the set of inter-component prediction modes, Reconstructing the luma component of the aforementioned video block, A device capable of performing the following actions: reconstructing one or more chroma components of the video block from the reconstructed chroma components using the inter-component prediction mode.

3. It is a method, Reconstructing the luma component of a video block, wherein the video block is coded in intercoding mode, Determining the inter-component prediction mode for the aforementioned video block, The method involves encoding one or more syntactic elements for the video block, wherein the one or more syntactic elements are derived by the decoder from a set of inter-component prediction modes. A method comprising encoding one or more chroma components of the video block based on the reconstructed chroma components and the determined inter-component prediction modes.

4. A device comprising one or more processors, wherein the one or more processors Reconstructing the luma component of a video block, wherein the video block is coded in intercoding mode, Determining the inter-component prediction mode for the aforementioned video block, The method for encoding one or more syntactic elements for the video block, wherein the one or more syntactic elements are derived by the decoder from a set of inter-component prediction modes, A device capable of encoding one or more chroma components of the video block based on the reconstructed chroma components and the determined inter-component prediction modes.

5. The method according to claim 1 or 3, or the apparatus according to claim 2 or 4, wherein the one or more syntactic elements include an index indicating an inter-component prediction mode in the set of inter-component prediction modes.

6. The method according to claim 1, 3, or 5, wherein the set of inter-component prediction modes comprises at least one of a convolutional inter-component model, an inter-component linear model, a multi-model linear model, or a gradient linear model. The apparatus according to claim 2, 4, or 5.

7. The method or apparatus according to claim 5 or 6, wherein the set of inter-component prediction modes is constructed for the video block based on one or more reconstructed video blocks.

8. The method or apparatus according to claim 7, wherein the one or more reconstructed blocks comprise at least one of spatially adjacent blocks, non-adjacent blocks, history-based blocks, or temporal blocks from a reference picture.

9. The method according to claim 1 or 3, or the apparatus according to claim 2 or 4, wherein the inter-component prediction mode is determined for the video block using the video block template.

10. The method or apparatus according to any one of claims 5 to 9, wherein the one or more syntactic elements include a flag indicating whether or not an inter-component prediction mode is used for the video block, and determining the inter-component prediction mode corresponds to the determination that the flag indicates the use of an inter-component prediction mode for the video block.

11. The method or apparatus according to any one of claims 5 to 10, wherein the one or more syntactic elements comprises one or more parameters relating to the inter-component prediction mode.

12. The method or apparatus according to claim 5 or 6, wherein the set of inter-component prediction modes is a table known by the decoder.

13. The method or apparatus according to claim 7 or 8, wherein, depending on the determination that the video block is in merge mode or intrablock copy mode, one or more syntactic elements include an index indicating a candidate block in a merge candidate block list used to derive motion data for the video block.

14. The method according to claim 1 or 3, or the apparatus according to claim 2 or 4, wherein, in response to a determination that the video block is in AMVP mode, the inter-component prediction mode is decoded from one or more syntactic elements together with the motion data of the video block.

15. The method according to claim 1 or 3, or the apparatus according to claim 2 or 4, wherein, depending on the determination that the video block is in merge mode or skip mode, motion data of the video block is derived from a selected candidate block in the merge candidate block list, and the inter-component prediction mode is inherited from the selected candidate block.

16. The method or apparatus according to claim 13, wherein the inter-component prediction mode is inherited from the selected candidate block in response to the determination that the selected candidate block is a spatially adjacent block of the video block.

17. The method according to any one of claims 1, 3, or 5 to 16, wherein the set of inter-component prediction modes depends on the inter-prediction mode of the video block, or the apparatus according to any one of claims 2 or 4 to 16.

18. The method according to any one of claims 1, 3, or 5-17, wherein the inter-component prediction modes of the set are sorted based on the cost determined for the template of the video block, or the apparatus according to any one of claims 2, or 4-17.

19. The method according to claim 1 or 3, or the apparatus according to claim 2 or 4, wherein the derivation of the inter-component prediction mode for the video block is based on a distortion determined between the chroma component of the reference block of the video block and the chroma component predicted from the luma component of the reference block using the inter-component prediction mode.

20. A computer program product comprising instructions for causing one or more processors to perform the method according to any one of claims 1, 3, or 5 to 19.

21. A non-temporary computer-readable medium for storing executable program instructions, which cause a computer executing the program instructions to carry out the method according to any one of claims 1, 3, or 5 to 19.

22. A bitstream comprising data representing video encoded using the method described in any one of claims 1, 3, or 5 to 19.

23. A non-temporary computer-readable medium for storing the bitstream described in claim 22.

24. It is a device, The apparatus according to claim 2, A device comprising: (i) an antenna configured to receive or transmit a signal including data representing the video block; (ii) a band limiter configured to restrict the signal to a frequency band including the data representing the video block; or (iii) a display configured to display the video block.

25. The device according to claim 24, wherein the device includes at least one of a television, a mobile phone, a tablet, and a set-top box.

Images