Reference Sample Selection for Cross-Component Intra Prediction

Abstract

A cross-component model generally describes the relationship between luma samples and associated chroma samples. Using cross-component model parameters derived based on neighboring regions, chroma samples of the current block can be predicted from matched reconstructed luma samples. Because the cross-component model parameters are derived at both the encoder and the decoder, these model parameters do not need to be explicitly signaled, thus reducing signaling overhead. In one implementation, to make the derived model more accurate, the signal characteristics of the luma reference samples are selected to better match the signal characteristics of the reconstructed luma samples in the current block using at least one of the histograms of neighboring samples, the shape of the current block, and the intra-mode direction. After the neighboring region is selected, the region can be further refined by using a set of region variants.

Description

[Technical Field]

[0001] The present embodiments generally relate to methods and apparatus for cross-component intra prediction in video encoding and decoding. [Background technology]

[0002] To achieve high compression efficiency, image and video coding schemes typically use prediction and transform to exploit spatial and temporal redundancy in the video content. Generally, intra- or inter-prediction is used to exploit intra- or inter-picture correlation, and then the difference between the original block and the predicted block, often referred to as the prediction error or prediction residual, is transformed, quantized, and entropy coded. To reconstruct the video, the compressed data is decoded by an inverse process corresponding to entropy coding, quantization, transformation, and prediction. Summary of the Invention

[0003] According to one embodiment, a method for video encoding is presented, the method including: obtaining reconstructed luma samples of a block of a picture; selecting one neighboring region of the block from a plurality of candidate neighboring regions of the block; obtaining reconstructed luma samples and reconstructed chroma samples of the selected neighboring region; obtaining model parameters of a cross-component model that models a relationship between the reconstructed luma samples and reconstructed chroma samples of the selected neighboring region; predicting chroma samples of the block based on the reconstructed luma samples of the block and the cross-component model having the model parameters; and encoding the chroma samples of the block based on the predicted chroma samples.

[0004] According to another embodiment, there is provided a method for video decoding, the method including: obtaining reconstructed luma samples of a block of a picture; selecting one neighboring region of the block from a plurality of candidate neighboring regions of the block; obtaining reconstructed luma samples and reconstructed chroma samples of the selected neighboring region; obtaining model parameters of a cross-component model that models a relationship between the reconstructed luma samples and reconstructed chroma samples of the selected neighboring region; predicting chroma samples of the block based on the reconstructed luma samples of the block and the cross-component model having the model parameters; and decoding the chroma samples of the block based on the predicted chroma samples.

[0005] According to another embodiment, there is provided an apparatus for video encoding, the apparatus including at least one memory and one or more processors configured to obtain reconstructed luma samples of a block of a picture; select one neighboring region of the block from a plurality of candidate neighboring regions of the block; obtain reconstructed luma samples and reconstructed chroma samples of the selected neighboring region; obtain model parameters of a cross-component model that models a relationship between the reconstructed luma samples and reconstructed chroma samples of the selected neighboring region; predict chroma samples of the block based on the reconstructed luma samples of the block and the cross-component model having the model parameters; and encode the chroma samples of the block based on the predicted chroma samples.

[0006] An apparatus for video decoding is provided, the apparatus including at least one memory and one or more processors configured to obtain reconstructed luma samples of a block of a picture, select one neighboring region of the block from a plurality of candidate neighboring regions of the block, obtain reconstructed luma samples and reconstructed chroma samples of the selected neighboring region, obtain model parameters of a cross-component model that models a relationship between the reconstructed luma samples and reconstructed chroma samples of the selected neighboring region, predict chroma samples of the block based on the reconstructed luma samples of the block and the cross-component model having the model parameters, and decode the chroma samples of the block based on the predicted chroma samples.

[0007] One or more embodiments also provide a computer program comprising instructions that, when executed by one or more processors, cause the one or more processors to perform an encoding or decoding method according to any of the embodiments described herein. One or more of the present embodiments also provide a computer-readable storage medium having stored thereon instructions for encoding or decoding video according to the methods described herein.

[0008] One or more embodiments also provide a computer-readable storage medium having stored thereon video data generated according to the methods described above. One or more embodiments also provide methods and apparatus for transmitting or receiving video data generated according to the methods described herein. [Brief explanation of the drawings]

[0009] [Figure 1] 1 shows a block diagram of a system in which aspects of the present embodiments can be implemented; [Figure 2] 1 shows a block diagram of an embodiment of a video encoder. [Figure 3] 1 shows a block diagram of an embodiment of a video decoder. [Figure 4]1 shows the spatial part of the convolution filter in CCLM. [Figure 5] An example of classifying adjacent samples into two groups is shown below. [Figure 6] 1 shows the spatial part of the convolution filter in CCCM. [Figure 7] Indicates the reference region (together with its padding) used to derive the filter coefficients. [Figure 8] 1 shows an overview of the process of the CC prediction method. [Figure 9] 1 shows the spatial samples used for GL-CCCM. [Figure 10] 1 shows the neighboring regions used to derive the CC model. [Figure 11A] Here is an example of a CC linear model derived using {(L0,C0),(L1,C1),(L3,C3)}. [Figure 11B] Here is an example of a CC linear model derived using {(L1,C1),(L3,C3)}. [Figure 12] Examples of a histogram and a normalized cumulative histogram (i.e., cumulative distribution function) are shown, respectively. [Figure 13] 10 illustrates a process for selecting regions used to derive a CC model using histogram matching, according to one embodiment. [Figure 14] 10 illustrates an example of refining a selected region from a primary list of regions, according to one embodiment. [Figure 15] 1 illustrates an example of region variants derived from a primary list of three regions, according to one embodiment. [Figure 16] 1 shows example samples used to derive two models, with shaded samples used by both models, according to one embodiment. [Figure 17] An example of a reference area shape adapted to the current PU shape is shown. DETAILED DESCRIPTION OF THE INVENTION

[0010] FIG. 1 shows a block diagram of an example system in which various aspects and embodiments can be implemented. System 100 may be embodied as a device including various components described below and configured to perform one or more of the aspects described herein. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television sets, personal video recording systems, connected home appliances, and servers. The elements of system 100, alone or in combination, may be embodied in a single integrated circuit, multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 100 are distributed across multiple ICs and / or discrete components. In various embodiments, system 100 is communicatively coupled to other systems or other electronic devices, for example, via a communication bus or via dedicated input and / or output ports. In various embodiments, system 100 is configured to implement one or more of the aspects described herein.

[0011] System 100 includes at least one processor 110 configured to execute instructions loaded thereon to implement various aspects described herein, for example. Processor 110 may include embedded memory, input / output interfaces, and various other circuits as known in the art. System 100 includes at least one memory 120 (e.g., a volatile memory device and / or a non-volatile memory device). System 100 includes storage device 140, which may include non-volatile memory and / or volatile memory, including, but not limited to, EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash, magnetic disk drives, and / or optical disk drives. Storage device 140 may include, by way of non-limiting example, an internal storage device, a removable storage device, and / or a network-accessible storage device.

[0012] System 100 includes encoder / decoder module 130, which may include its own processor and memory, configured to process data to provide, for example, encoded or decoded video. Encoder / decoder module 130 represents a module that may be included in a device that performs encoding and / or decoding functions. As is known, a device may include one or both of an encoding module and a decoding module. Furthermore, encoder / decoder module 130 may be implemented as a separate element of system 100 or may be incorporated within processor 110 as a combination of hardware and software, as is known to those skilled in the art.

[0013] Program code to be loaded into processor 110 or encoder / decoder 130 to perform various aspects described herein may be stored in storage device 140 and subsequently loaded onto memory 120 for execution by processor 110. According to various embodiments, one or more of processor 110, memory 120, storage device 140, and encoder / decoder module 130 may store one or more of various items during execution of the processes described herein. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from processing equations, expressions, operations, and computational logic.

[0014] In some embodiments, memory internal to the processor 110 and / or the encoder / decoder module 130 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device may be either the processor 110 or the encoder / decoder module 130) is used for one or more of these functions. The external memory may be the memory 120 and / or the storage device 140, and may be, for example, dynamic volatile memory and / or non-volatile flash memory. In some embodiments, external non-volatile flash memory is used to store the television's operating system. In at least one embodiment, high-speed external dynamic volatile memory, such as RAM, is used as working memory for video coding and decoding operations, such as MPEG-2, HEVC, or Versatile Video Coding (VVC).

[0015] Input to the elements of system 100 may be provided via various input devices, as indicated in block 105. Such input devices include, but are not limited to, (i) an RF section that receives RF signals transmitted over the air by, for example, a broadcast station, (ii) a composite input, (iii) a USB input, and / or (iv) an HDMI input.

[0016] In various embodiments, the input devices of block 105 have associated respective input processing elements as known in the art. For example, the RF section may be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal or bandlimiting a signal to a frequency band), (ii) downconverting the selected signal, (iii) bandlimiting again to a narrower frequency band to select a signal frequency band, which in certain embodiments may be referred to as a channel (for example), (iv) demodulating the downconverted and bandlimited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section of various embodiments includes one or more elements that perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a downconverter, a demodulator, an error corrector, and a demultiplexer. The RF section may include, for example, a tuner that performs various of these functions, including downconverting received signals to a lower frequency (e.g., an intermediate frequency or a frequency near baseband) or to baseband. In one set-top box embodiment, the RF section and its associated input processing elements perform frequency selection by receiving, filtering, downconverting, and re-filtering RF signals transmitted over a wired (e.g., cable) medium to a desired frequency band. In various embodiments, the order of these (and other) elements is rearranged, some of these elements are removed, and / or other elements that perform similar or different functions are added. Adding elements may include inserting elements between existing elements, for example, inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0017] Additionally, the USB and / or HDMI terminals may include respective interface processors for connecting system 100 to other electronic devices over USB and / or HDMI connections. It should be understood that various aspects of input processing, e.g., Reed-Solomon error correction, may be performed, for example, within a separate input processing IC or within processor 110, as desired. Similarly, aspects of USB or HDMI interface processing may be implemented, as desired, within a separate interface IC or within processor 110. The demodulated, error corrected, and demultiplexed stream is provided to various processing elements, including, for example, processor 110 and encoder / decoder 130, which operates in combination with memory and storage elements to process the data stream as desired for display on an output device.

[0018] The various elements of system 100 may be provided within an integrated housing in which the various elements are interconnected and may transmit data therebetween using a suitable connection arrangement 115, e.g., an internal bus as known in the art, including an I2C bus, wiring, and a printed circuit board.

[0019] System 100 includes a communication interface 150 that enables communication with other devices over a communication channel 190. Communication interface 150 may include, but is not limited to, a transceiver configured to transmit and receive data over communication channel 190. Communication interface 150 may include, but is not limited to, a modem or a network card, and communication channel 190 may be implemented in a wired and / or wireless medium, for example.

[0020] In various embodiments, data is streamed to system 100 using a Wi-Fi network, such as IEEE 802.11. The Wi-Fi signal in these embodiments is received via communication channel 190 and communication interface 150, which are adapted for Wi-Fi communication. Communication channel 190 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to enable streaming applications and other over-the-top communications. Other embodiments provide streamed data to system 100 using a set-top box that delivers data via an HDMI connection in input block 105. Still other embodiments provide streamed data to system 100 using an RF connection in input block 105.

[0021] System 100 may provide output signals to various output devices, including display 165, speakers 175, and other peripheral devices 185. Other peripheral devices 185, in various example embodiments, include one or more of a standalone DVR, a disc player, a stereo system, a lighting system, and other devices that provide functionality based on the output of system 100. In various embodiments, control signals are communicated between system 100 and display 165, speakers 175, or other peripheral devices 185 using signaling such as AV.Link, CEC, or other communication protocols that allow control between devices with or without user intervention. Output devices may be communicatively coupled to system 100 via dedicated connections through respective interfaces 160, 170, and 180. Alternatively, output devices may be connected to system 100 using communication channel 190 via communication interface 150. Display 165 and speakers 175 may be integrated into a single unit with other components of system 100 within an electronic device, such as a television. In various embodiments, the display interface 160 includes a display driver, for example, a timing controller (T Con) chip.

[0022] Display 165 and speakers 175 may alternatively be separate from one or more of the other components, for example, if the RF portion of input 105 is part of a separate set-top box. In various embodiments in which display 165 and speakers 175 are external components, the output signal may be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0023] 2 shows an example of a block-based hybrid video encoder 200. Before being encoded, a video sequence may undergo a pre-encoding process (201), such as applying a color transformation to an input color picture (e.g., from RGB 4:4:4 to YCbCr 4:2:0) or performing a remapping of the input picture components to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata can be associated with the pre-processing and can be attached to the bitstream.

[0024] In the encoder 200, a picture is coded by the encoder elements as described below. The picture to be coded is partitioned (202) and processed, for example, in units of coding units (CUs). Each unit is coded, for example, using either intra mode or inter mode. If the unit is coded in intra mode, intra prediction is performed (260). In inter mode, motion estimation (275) and motion compensation (270) are performed. The encoder decides (205) whether to use intra mode or inter mode to code the unit, and indicates the intra / inter decision, for example, by a prediction mode flag. A prediction residual is calculated, for example, by subtracting (210) the prediction block from the original image block.

[0025] The prediction residual is then transformed (225) and quantized (230). The quantized transform coefficients, as well as other syntax elements such as motion vectors and picture partition information, are entropy coded (245) to output a bitstream. The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can bypass both the transform and quantization, i.e., the residual is coded directly without applying a transform or quantization process.

[0026] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual is combined (255) with the prediction block to reconstruct an image block. An in-loop filter (265) is applied to the reconstructed picture, performing, for example, deblocking / SAO (Sample Adaptive Offset) / ALF (Adaptive Loop Filter) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer 280.

[0027] Figure 3 shows a block diagram of an exemplary video decoder 300. In the decoder 300, the bitstream is decoded by decoder elements as described below. The video decoder 300 generally performs a decoding pass that is the inverse of the encoding pass as described in Figure 2. The encoder 200 also generally performs video decoding as part of encoding the video data.

[0028] Specifically, the decoder's input includes a video bitstream, which may be generated by the video encoder 200. First, the bitstream is entropy decoded (330) to obtain transform coefficients, prediction modes, motion vectors, and other coded information. Picture partition information indicates how the picture is partitioned. Thus, the decoder may divide the picture according to the decoded picture partition information (335). The transform coefficients are inverse quantized (340) and inverse transformed (350) to decode the prediction residual. The decoded prediction residual is combined with a prediction block (355) to reconstruct an image block. The prediction block may be obtained from intra prediction (360) or motion-compensated prediction (i.e., inter prediction) (375) (370). An in-loop filter (365) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (380). Note that for a given picture, the contents of the decoder-side reference picture buffer 380 are identical to the contents of the encoder-side reference picture buffer 280 for the same picture.

[0029] The decoded picture may further undergo a post-decoding process (385), such as an inverse color transform (e.g., YCbCr 4:2:0 to RGB 4:4:4) or an inverse remapping that performs the inverse of the remapping process performed in the pre-encoding process (201). The post-decoding process may use metadata derived in the pre-encoding process and signaled in the bitstream.

[0030] Cross-Component Linear Models (CCLM) for Intra-Prediction To reduce cross-component redundancy, a cross-component linear model (CCLM) prediction mode is used in VVC, for which chroma samples are predicted based on the reconstructed luma samples of the same CU by using a linear model as follows: pred C (i,j)=α·rec L (i,j)+β (Equation 1) In the formula, pred C (i,j) represents the predicted chroma sample in CU, and rec L (i,j) represents the downsampled reconstructed luma sample of the same CU. In VVC, CCLM parameters (α and β) are derived using up to four neighboring chroma samples and their corresponding downsampled luma samples. As shown in Figure 4, there are three CCLM modes (LM_CHROMA, MDLM_T, MDLM_L) with different positions of the neighboring chroma samples. The CCLM mode used is coded per CU.

[0031] The four adjacent luma samples at the selected location are the smaller of the two x 0 A and x 1 A and two larger values ​​x 0 B and x 1 B The corresponding chroma sample values ​​are then downsampled four times and compared to find y 0 A , y 1 A , y 0 B and y 1 B Then, X a , X b , Y a and Y b is derived as follows: X a =(x 0 A +x 1 A +1)>>1;X b =(x 0 B +x 1 B +1)>>1; Y a =(y 0 A +y 1 A +1)>>1;Y b =(y0 B +y 1 B +1)>>1 (Formula 2)

[0032] Finally, the linear model parameters α and β are obtained by: α=(Y a -Y b ) / (X a -X b ) (Formula 3) β=Y b -α·X b (Formula 4)

[0033] Multi-Model LM (MMLM) and other CCLM variants There are several variants of CCLM, which may differ in a) the location and / or number of neighboring samples used to derive the model, b) the method for deriving the linear model parameters (α, β), or c) the luma downsampling filter. For example, the Enhanced Compression Model (ECM) extends the CCLM included in VVC by adding three Multi-Model LM (MMLM) modes. In each MMLM mode, reconstructed neighboring samples are classified into two classes using a threshold that is the average of the reconstructed luma neighboring samples, as shown in Figure 5. A linear model for each class is derived using the least mean squares (LMS) method or, for example, the CCLM method described above.

[0034] Convolutional Cross-Component Model (CCCM) for Intra Prediction The convolutional cross-component model (CCCM) predicts chroma samples from reconstructed luma samples in a similar manner to that done by CCLM. As with CCLM, when chroma subsampling is used, the reconstructed luma samples may be downsampled to match the lower resolution of the chroma grid.

[0035] Also, similar to CCLM, there is the option of using a single-model or multi-model variant of CCCM. The multi-model variant uses two models: one model derived for samples above the average luma reference value and another model derived for the remaining samples (following the CCLM design spirit). In the variant, the multi-model CCCM mode can be selected for PUs for which at least 128 reference samples are available.

[0036] A pixel may include several (e.g., three) color components. For ease of notation, a luma component may be referred to as a luma sample, and a chroma component may be referred to as a chroma sample, and luma samples and chroma samples that represent the same pixel are considered to be co-located. Similarly, if an image block has several color components, the luma and chroma blocks of this image block are considered to be co-located.

[0037] CCCM uses a convolutional 7-tap filter consisting of a 5-tap plus-sign shaped spatial component, a nonlinear term, and a bias term. The input to the spatial 5-tap component of the filter consists of a center (C) luma sample co-located with the chroma sample to be predicted, and its above / north (N), below / south (S), left / west (W), and right / east (E) neighbors, as shown in Figure 6.

[0038] The nonlinear term P is expressed as a power of 2 of the center luma sample C, offset by midVal=(1<<(bitDepth-1)) and scaled to the sample value range of the content. P=(C * C+midVal)>>bitDepth

[0039] That is, for 10-bit content, midVal is 512 and P is calculated as follows: P=(C * C+512)>>10

[0040] The bias term B represents a scalar offset between the input and output (similar to the offset term in CCLM) and is set to the midpoint chroma value (512 for 10-bit content).

[0041] The output of the filter is the filter coefficient c i The chroma sample is calculated as the convolution between the input value and the chroma sample, clipped to the range of valid chroma samples. predChromaVal=c0C+c1N+c2S+c3E+c4W+c5P+c6B

[0042] filter coefficient c i is calculated by minimizing the MSE (mean square error) between the predicted and reconstructed chroma samples in the reference region. Figure 7 shows a reference region consisting of six rows / columns of chroma samples above and to the left of the PU. The reference region extends one PU wide to the right of the PU boundary and one PU high below. The reference region is adjusted to include only available samples. To support "side samples" for plus-shaped spatial filters, an additional one row and one column of samples are attached to the reference region (705), padding out any unavailable areas.

[0043] Define the following:

[0044]

number

[0045] The autocorrelation matrix is ​​subjected to LDL decomposition, and the final filter coefficients are calculated using backsubstitution. This process loosely follows the calculation of the ALF filter coefficients in ECM, but we chose LDL decomposition (i.e., alternate Cholesky decomposition) instead of Cholesky decomposition to avoid the use of square root operations. The calculation uses only integer arithmetic.

[0046] An overview of the process of predicting chroma samples using a cross-component luma model (e.g., CCLM, MMLM, CCCM) is shown in FIG. 8. Specifically, in step 810, reference samples (reconstructed luma and chroma sample values) are selected from the neighborhood of the current CU. The reconstructed luma sample values ​​may be filtered to obtain downsampled luma samples. In step 820, in the case of a multi-model, a threshold for classifying the reference samples into at least two classes is determined. In step 830, parameters of the model are derived from the reference luma and chroma sample values. In step 840, a CC model, for example, using CCLM, MMLM, or CCCM, allows deriving chroma sample prediction values ​​from the co-located (downsampled) reconstructed luma sample values ​​of the current block. In one example, when luma is downsampled by 2, the prediction of chroma sample C(x,y) uses the reconstructed luma sample at the same position L(x,y) and samples at positions L(x+1,y), L(x-1,y), L(x,y+1), L(x,y-1), as shown in Figure 6.

[0047] In the variant named GL-CCCM, the GL-CCCM filter for prediction is: predChromaVal=c0C+c1Gy+c2Gx+c3Y+c4X+c5P+c6B, where the Y and X parameters are the vertical and horizontal positions of the central luma sample, which are calculated with respect to the top-left coordinate of the block, and Gy and Gx are the vertical and horizontal gradients, respectively, calculated as follows: Gy=(2N+NW+NE)-(2S+SW+SE), Gx=(2W+NW+SW)-(2E+NE+SE), Figure 9 shows the sample locations for samples NW, NE, SW, and SE.

[0048] Selection of the location of the reference sample used to derive the CC model In another variant, the neighboring region used to select the reference sample for deriving the CC model (cross-component model) can be selected from a set of predefined regions. For example, in ECM-6.0-EE2 and ECM-7.0, the selected region from {top-left, top, left} corresponding to the modes {CHROMA_IDX, T_IDX, L_IDX}, respectively, can be signaled in the bitstream along with information indicating the single-model or multi-model mode {MDLM, MMLM}, as shown in Figure 10.

[0049] Decoder-side Intra-mode Derivation (DIMD) DIMD derives the indices of two intra-prediction modes that are likely to be the two best intra-prediction modes for predicting the current luma CB in terms of rate distortion from the gradients in the template of the decoded reference samples of the current luma CB to be coded / decoded. The current luma CB is then predicted by blending the two prediction blocks obtained by applying the two derived intra-prediction modes with the prediction block obtained by applying PLANAR. The weights involved in the blending are derived from the gradients in this template.

[0050] Template-based intra-mode derivation (TIMD) Similar to DIMD, for a current luma CB to be coded / decoded, TIMD follows a two-step process: an intra-prediction mode index derivation step involving a template of decoded reference samples of the current luma CB, and a step where the current luma CB is actually predicted.

[0051] Specifically, for a given luma CB, mode derivation via TIMD is applied in the same way on the encoder and decoder sides. For each intra prediction mode in the MPM list of this luma CB, supplemented if necessary with a default mode, this mode calculates a prediction of the luma CB template from the decoded reference samples of the template, and calculates the SATD (sum of absolute transform differences) between this prediction and the luma CB template. The two intra prediction modes with the smallest SATD are selected as TIMD modes.

[0052] As described above, a cross-component intra-prediction tool (e.g., CCLM, MMLM, or CCCM) derives a cross-component model from a set of reconstructed reference luma samples and reference chroma samples located in the vicinity of the current CU. Generally, the cross-component model describes the relationship between luma samples and reconstructed chroma samples. Using cross-component model parameters derived based on the luma samples and chroma samples of neighboring regions, the chroma samples of the current block can be predicted from the luma samples of the current block. Because the cross-component model parameters are derived in both the encoder and the decoder, these model parameters do not need to be explicitly signaled, thus reducing signaling overhead. However, the signal characteristics (statistics) of these luma reference samples may differ from those of the reconstructed luma samples in the current CU, which may affect the accuracy of the derived model.

[0053] In this specification, it is proposed to modify the process of selecting reference samples used to derive a CC model to better match the signal characteristics of the reconstructed luma samples of the current CU, for example, using at least one of the histogram of neighboring samples, the shape of the current CU, and the intra-mode direction of the current CU.

[0054] As described above, the signal characteristics of some reconstructed luma sample values ​​in the neighborhood of the current CU may be significantly different from those of the reconstructed luma sample values ​​of the current CU. This may affect the efficiency of the derived CC model for predicting chroma sample values ​​from the reconstructed CU luma sample values. For example, in FIGS. 11A and 11B, the (luma, chroma) sample values ​​of the current CU may be only (L1, C1) and (L3, C3), while the (luma, chroma) sample values ​​of the reference samples may be (L0, C0), (L1, C1), and (L3, C3). That is, because the luma value L0 is not in the current CU, the signal characteristics of the reference samples are different from those of the current CU. Whether or not to include (L0, C0) in the reference samples for deriving a CC model may result in different linear models, as shown in FIGS. 11A and 11B.

[0055] The proposed method can be applied to cross-component intra-prediction tools (CC modes), such as CCLM, MMLM, or CCCM. Generally, these methods derive a CC model using at least one of histogram matching of neighboring reconstructed luma samples, the shape of the current CU, and the intra-mode direction.

[0056] Selecting regions for deriving CC models using histogram matching In this embodiment, the selection of the neighboring region containing the reference samples used to derive the CC model is performed using histogram matching between the reconstructed luma samples of the current CU and the reconstructed luma samples of the neighboring region candidates.

[0057] FIG. 13 illustrates a process (1300) for selecting a region to be used to derive a CC model using histogram matching, according to one embodiment. In this embodiment, first, a histogram "curHist" of the reconstructed luma samples of the current block is collected (1305). Next, for each available neighboring predefined region (1360) (e.g., as shown in FIG. 10 or FIG. 15), a histogram "iHist" of region "i" is calculated (1320), and a histogram matching score with curHist is calculated (1330). Histogram matching measures the difference between two histograms. For example, it can be the sum of the differences in different bins of a normalized cumulative histogram as shown in FIG. 12. The region with the histogram that best matches curHist (1340) is selected to derive a CC model (1350).

[0058] This process can be performed at the encoder or decoder. As with other cross-component intra prediction tools, the CC model parameters and selected regions do not need to be transmitted in the bitstream, as these parameters can be derived at the decoder.

[0059] A condition (1310) checks whether a region is available. For example, a region is unavailable if all or a subset of samples are outside the current picture, or if they belong to another slice, tile, subpicture, etc. In one variant, if a region is partially unavailable, only available samples are used to derive the CC model. In another variant, only intra-coded samples are used to avoid pipeline dependencies with inter-coded CUs.

[0060] In a variant, the histogram matching score is calculated for the same range as the current reconstructed luma samples. That is, the difference is calculated only for bins that fall into the data range of the reconstructed luma samples of the current CU. For example, the range is the range [min, max] of the reconstructed luma sample values ​​of the current CU, where min is the lowest (minimum) luma value in the current CU and max is the highest (maximum) luma value in the current CU.

[0061] Reference area adjustment FIG. 14 illustrates a process (1400) for deriving region variants from a primary list of three regions, according to one embodiment. In this embodiment, neighboring regions are signaled (1405) from the primary list of regions. The signaled neighboring region shapes are adjusted / refinement using histogram matching. A set of region variants may be predefined for each primary region. For example, FIG. 15 illustrates three primary regions {top, left, top-left} and their associated variants. In one example, region variants can be obtained by adding / removing a relatively small number of samples relative to the primary sample count, which are located spatially adjacent to and / or in the same direction (e.g., top, left, or top-left) as the primary region. The size of the region variants can be scaled up or down based on the size of the primary region.

[0062] Specifically, a condition (1410) checks whether region variants are available. For each available region variant (1460), a histogram "xHist" of region variant "x" is calculated (1420), and a histogram matching score with curHist is calculated (1430). The region variant with the histogram that best matches curHist (1440) is selected as the refined region for deriving the CC model (1450).

[0063] Refining a region using region variants can be used in conjunction with other processes, for example, after the best region is selected in process 1300, region variants can also be selected.

[0064] Histogram-based mode signaling In this embodiment, a flag is signaled to indicate which method is applied: If the flag is 0, the normal method is used, which means that the default (primary) domain is used to derive the CC model.

[0065] Derivation of multi-model threshold using median As mentioned above, in the multi-model case, the threshold is derived using the average of the reference neighboring samples. In this embodiment, the threshold is calculated as the median of the selected neighboring reference samples. The median can be derived directly using a cumulative histogram. In addition, a flag may be signaled, for example, per CU or per slice, to indicate whether this method is used.

[0066] Reducing the multi-model gap To reduce discontinuities in the CC model, separate thresholds can be derived for samples below the average (Model-1) and samples above the average (Model-2). First, a preliminary threshold is derived using the average of selected neighboring samples. Then, an additional set of neighboring sample values ​​above the threshold (Model-1) is used to derive the CC model for Model-1, and an additional set of neighboring sample values ​​below the threshold (Model-2) is used to derive the CC model for Model-2. The value of the ratio (relative number of additional samples) can be, for example, 10%.

[0067] In the example shown in FIG. 16, Model-1 is calculated using reference samples below the threshold plus additional samples (shaded) above the threshold, and Model-2 is calculated using reference samples above the threshold plus additional samples (hatched) below the threshold. For each model, the number of additional samples may be determined, for example, as equal to N% of the number of samples below (or above, respectively) the threshold. The shaded samples are then used to calculate both models, while the black and white samples are used for Models 1 and 2, respectively. The dashed and solid lines represent the derived models without and with additional samples, respectively. It can be observed that the gap in threshold position between the two models is reduced by the proposed method. In addition, a flag may be signaled per CU or per slice to indicate whether this method is used.

[0068] Use of intramode direction The intra mode of the co-located luma block can be used to determine the region to be used for CC. If the mode is horizontally oriented (or close to a horizontal mode), the left region can be used. Otherwise, if the mode is vertically oriented (or close to a vertical mode), the top region can be used. Similarly, if the mode is diagonally oriented, both regions are used. The reasoning behind this method is that if the mode used is horizontally oriented, there is some correlation between the current luma block and its left region. Therefore, the CC model is expected to have the same correlation. The opposite is also true for vertically or diagonally oriented intra modes.

[0069] In one variant, DIMD can be used, i.e., the DIMD derivation mode of the co-located luma block can be used in the same way as before to determine the region. This is especially useful when the intra-mode direction is not available (the co-located luma is inter-predicted, IBC, MIP, etc.).

[0070] In another variant, chroma DIMD can be used, which is based on analyzing templates of the luma and two chroma channels. The resulting DIMD modes can be used in a similar way. Instead of DIMD, TIMD (template-based intra mode derivation) can also be used.

[0071] Considering block dimensions The block dimensions can also give clues about the area used. For a tall block (a block where the height is greater than the width), it is unlikely that the top area can be used, since the bottom sample will be far away from the top reference area. Therefore, it is expected to use the entire area, or the left area, or only the left area. Similarly, for a wide block (where the width is greater than the height), it is expected to use the top area, or the entire area, or only the top area. Finally, for a square block, all areas (top, left, or entire) can be used. Whether a block is considered tall / wide can be determined based on a width-to-height ratio threshold.

[0072] In ECM, the regions used to signal the reference region are coded together with the single / multiple model as follows:

[0073] [Table 1]

[0074] This method can be used to select regions a priori and save signaling bits. In one variant, this method can be used to derive the most likely region to be ordered first.

[0075] In one variant, this method is proposed to reduce the signaling of the region to be used. Specifically, reduced signaling is used for non-square blocks to indicate which region is to be used. If the signaling is not the top-left region, the region can be estimated. For example, for a vertically oriented block, if the signaling is not the top-left region, the left region is used because it is the most likely region (Table 2).

[0076] [Table 2]

[0077] In one variant, a separate flag for signaling single / multiple models may be used.

[0078] In another variant, the shape of the reference region can be adapted depending on the current CU shape, as shown in Examples 1 and 2 of Figure 17 for a likely square and a likely horizontally long shape (a vertically thin shape can be inferred by symmetry).

[0079] Various methods are described herein, each of which includes one or more steps or actions for achieving the described method. Unless a specific order of steps or actions is required for the successful operation of the method, the order and / or use of specific steps and / or actions may be modified or combined. Furthermore, terms such as "first," "second," and the like may be used in various embodiments to modify elements, components, steps, operations, etc., e.g., "first decode" and "second decode." The use of such terms does not imply a modified order of operations unless specifically required. Thus, in this example, the first decode need not be performed before the second decode, but could occur, for example, before, during, or during an overlapping period with the second decode.

[0080] Various methods and other aspects described herein can be used to modify modules, such as the intra prediction modules (260, 360) of video encoder 200 and decoder 300, as shown in Figures 2 and 3. Furthermore, the aspects are not limited to ECM, VVC, or HEVC, but can be applied, for example, to other standards and recommendations, and extensions of any such standards and recommendations. Unless otherwise indicated or technically excluded, the aspects described herein can be used individually or in combination.

[0081] Various numerical values ​​are used in this application. The particular values ​​are for illustrative purposes and the described aspects are not limited to these particular values.

[0082] Various implementations involve decoding. As used herein, "decoding" may encompass all or some of the processes performed on a received encoded sequence to generate a final output suitable for display, for example. In various embodiments, such processes include one or more of the processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. Whether the phrase "decoding process" is intended to refer specifically to a subset of operations or to the broader decoding process generally will be clear based on the context of the particular description and will be well understood by those skilled in the art.

[0083] Various implementations involve encoding. Similar to the discussion of "decoding," "encoding" as used herein may encompass, for example, all or part of the processes performed on an input video sequence to generate an encoded bitstream.

[0084] Implementations and aspects described herein may be realized in, for example, a method or process, an apparatus, a software program, a data stream, or a signal. Even when discussed only in the context of a single implementation (e.g., discussed only as a method), the implementation of the discussed features may also be realized in other forms (e.g., an apparatus or a program). An apparatus may be implemented in, for example, appropriate hardware, software, and firmware. The method may be performed in, for example, an apparatus, e.g., a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, e.g., computers, mobile phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate the transfer of information between end users.

[0085] References to "one embodiment" or "one embodiment," or "one implementation" or "one implementation," as well as other variations thereof, mean that a particular feature, structure, characteristic, etc. described in connection with an embodiment is included in at least one embodiment. Thus, appearances of the phrases "in one embodiment" or "in an embodiment," or "in one implementation," or "in an implementation," as well as any other variations thereof, in various places throughout this application are not necessarily all referring to the same embodiment.

[0086] Additionally, the application may refer to "determining" various information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from memory.

[0087] Additionally, the application may refer to "accessing" various information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0088] Additionally, the present application may refer to "receiving" various information. Receiving, like "accessing," is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Furthermore, "receiving" typically involves, in some manner, in operation, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0089] For example, in the case of "A / B," "A and / or B," and "at least one of A and B," it should be understood that the use of any of the following " / ," "and / or," and "at least one of" is intended to encompass the selection of only the first listed option (A), or the selection of only the second listed option (B), or the selection of both options (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such language is intended to encompass the selection of only the first listed option (A), or the selection of only the second listed option (B), or the selection of only the third listed option (C), or the selection of only the first and second listed options (A and B), or the selection of only the first and third listed options (A and C), or the selection of only the second and third listed options (B and C), or the selection of all three options (A, B, and C). This may be expanded as many times as the number of listed items, as would be apparent to one of ordinary skill in the art.

[0090] Also, as used herein, the word "signaling" refers to, among other things, indicating something to a corresponding decoder. For example, in a particular embodiment, an encoder signals which neighboring region is selected. In this way, in one embodiment, the same parameters are used on both the encoder and decoder sides. Thus, for example, an encoder can transmit specific parameters to a decoder (explicit signaling), so that the decoder can use the same specific parameters. Conversely, if the decoder already has specific parameters as well as other parameters, signaling can be used without transmission (implicit signaling), allowing the decoder to easily recognize and select the specific parameters. By avoiding the transmission of any actual functions, bit savings are realized in various embodiments. It should be appreciated that signaling can be achieved in various ways. For example, one or more syntax elements, flags, etc. are used to signal information to a corresponding decoder in various embodiments. Although this relates to the verb form of the word "signaling," the word "signaling" can also be used as a noun herein.

[0091] As will be apparent to those skilled in the art, implementations may generate a wide variety of signals formatted to carry information that may be, for example, stored or transmitted. The information may include, for example, instructions for performing a method or data generated by one of the described implementations. For example, a signal may be formatted to carry a bitstream of the described embodiments. Such a signal may be formatted, for example, as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. The signal may be transmitted over a wide variety of different wired or wireless links, as is known. The signal may be stored on a processor-readable medium.

Claims

1. 1. A method of video encoding, comprising: Obtaining reconstructed luma samples of a block of a picture; selecting a neighboring region of the block from a plurality of candidate neighboring regions of the block; obtaining reconstructed luma samples and reconstructed chroma samples of the selected adjacent region; obtaining model parameters of a cross-component model that models a relationship between the reconstructed luma samples and reconstructed chroma samples of the selected neighboring regions; predicting chroma samples of the block based on the reconstructed luma samples of the block and the cross-component model having the model parameters; encoding the chroma samples of the block based on the predicted chroma samples.

2. 1. A method of video decoding, comprising: Obtaining reconstructed luma samples of a block of a picture; selecting a neighboring region of the block from a plurality of candidate neighboring regions of the block; obtaining reconstructed luma samples and reconstructed chroma samples of the selected adjacent region; obtaining model parameters of a cross-component model that models a relationship between the reconstructed luma samples and reconstructed chroma samples of the selected neighboring regions; predicting chroma samples of the block based on the reconstructed luma samples of the block and the cross-component model having the model parameters; and decoding the chroma samples of the block based on the predicted chroma samples.

3. 3. The method of claim 1, further comprising performing histogram matching between the reconstructed luma samples of the block and reconstructed luma samples of each of the plurality of neighboring regions of the block, wherein the neighboring region having the best histogram match is selected.

4. The method according to any one of claims 1 to 3, further comprising obtaining a range of values ​​of the reconstructed samples of the block, and wherein histogram matching is performed only within said range of values.

5. The method of claim 1 , 3, or 4, further comprising encoding a signal indicating that the adjacent region has been selected.

6. The method of claim 2 , 3 , or 4 , further comprising decoding a signal indicative of the selected adjacent region.

7. The method of any one of claims 1 to 6, further comprising: refining the selected contiguous region by removing one or more samples from the selected contiguous region.

8. The cross-component model is a multi-model, and obtaining the model parameters includes: The method of any one of claims 1 to 7, comprising obtaining a threshold based on a median value of the reconstructed luma samples of the selected neighboring regions of the multi-model.

9. The method of claim 8 , wherein two models are used in the multi-model analysis, and one or more samples above or below the threshold are used to obtain model parameters for the two models.

10. The method of any one of claims 1 to 9, wherein only the left neighboring region is selected in response to the intra prediction mode of the block being horizontally oriented.

11. The method of any one of claims 1 to 9, wherein only the upper neighboring region is selected in response to the intra prediction mode of the block being horizontally oriented.

12. The method according to claim 10 or 11, wherein the intra prediction mode of the block is obtained by DIMD or TIMD.

13. The method according to any one of claims 1 to 9, wherein in response to the block being a vertically elongated block, only the left region is selected.

14. The method according to any one of claims 1 to 9, wherein in response to the block being a landscape block, only the upper region is selected.

15. 1. An apparatus for video encoding, comprising: at least one memory; and one or more processors, the one or more processors: Obtaining reconstructed luma samples of the blocks of the picture; selecting a neighboring region of the block from a plurality of candidate neighboring regions of the block; obtaining reconstructed luma samples and reconstructed chroma samples of the selected adjacent region; obtaining model parameters of a cross-component model that models a relationship between the reconstructed luma samples and reconstructed chroma samples of the selected neighboring regions; predicting chroma samples of the block based on the reconstructed luma samples of the block and the cross-component model having the model parameters; and encoding the chroma samples of the block based on the predicted chroma samples.

16. 1. An apparatus for video decoding, comprising: at least one memory; and one or more processors, the one or more processors: Obtaining reconstructed luma samples of the blocks of the picture; selecting a neighboring region of the block from a plurality of candidate neighboring regions of the block; obtaining reconstructed luma samples and reconstructed chroma samples of the selected adjacent region; obtaining model parameters of a cross-component model that models a relationship between the reconstructed luma samples and reconstructed chroma samples of the selected neighboring regions; predicting chroma samples of the block based on the reconstructed luma samples of the block and the cross-component model having the model parameters; and decoding the chroma samples of the block based on the predicted chroma samples.

17. the one or more processors:

17. The apparatus of claim 15 or 16, further configured to perform histogram matching between the reconstructed luma samples of the block and reconstructed luma samples of each of the plurality of neighboring regions of the block, wherein the neighboring region having the best histogram match is selected.

18. the one or more processors:

18. The apparatus of claim 15, further configured to obtain a range of values ​​of the reconstructed samples of the block, and wherein histogram matching is performed only within said range of values.

19. 19. The apparatus of any one of claims 15, 17, and 18, wherein the one or more processors are further configured to encode a signal indicating that the adjacent region has been selected.

20. 19. The apparatus of claim 16, 17, or 18, wherein the one or more processors are further configured to decode a signal indicative of the selected adjacent region.

21. the one or more processors:

21. The apparatus of any one of claims 15 to 20, further configured to refine the selected adjacent region by removing one or more samples from the selected adjacent region.

22. the cross-component model is multi-model, and the one or more processors:

23. The apparatus of claim 15, configured to obtain model parameters by performing thresholding based on median values ​​of the reconstructed luma samples of the selected neighboring regions of the multi-model.

23. 23. The apparatus of claim 22, wherein two models are used in the multi-model analysis, and one or more samples above or below the threshold are used to obtain model parameters for the two models.

24. The apparatus of any one of claims 15 to 24, wherein in response to the intra prediction mode of the block being horizontally oriented, only the left-neighboring region is selected.

25. The apparatus of any one of claims 15 to 23, wherein in response to the intra prediction mode of the block being horizontally oriented, only the upper-neighboring region is selected.

26. The device according to claim 24 or 25, wherein the intra prediction mode of the block is obtained by DIMD or TIMD.

27. The apparatus according to any one of claims 15 to 23, wherein in response to the block being a vertically elongated block, only the left region is selected.

28. The apparatus according to any one of claims 15 to 23, wherein in response to the block being a horizontally long block, only the upper region is selected.

29. A signal containing video data formed by carrying out a method according to any one of claims 1 and 3 to 14.

30. A computer readable storage medium having stored thereon instructions for encoding or decoding video according to the method of any one of claims 1 to 14.

Images