Method and apparatus for processing a video signal by using a cross-component linear model

Abstract

The present disclosure provides a method and apparatus for processing a video signal by using a cross-component linear model. A video signal processing method and apparatus for encoding or decoding a video signal are disclosed. The video signal processing method includes the steps of downsampling a luminance component of a reconstructed block neighboring a current block, obtaining a maximum luminance value from among the downsampled luminance components based on a first index, obtaining a second-highest maximum luminance value from among the downsampled luminance components based on a second index, obtaining a maximum average luminance value according to an average of the maximum luminance value and the second-highest maximum luminance value, obtaining a second-lowest minimum luminance value from among the downsampled luminance components based on a third index, obtaining a minimum luminance value from among the downsampled luminance components based on a fourth index, and obtaining a minimum average luminance value based on the second-lowest minimum luminance value and the minimum luminance value.

Description

[0001] This application is a divisional application of the invention patent application filed on October 14, 2019, with application number "201980066770.7" and titled "Method and apparatus for processing video signals by using a cross-component linear model". Technical Field

[0002] This disclosure relates to a method and apparatus for processing video signals, and more specifically, to a video signal processing method and apparatus for encoding and decoding video signals. Background Technology

[0003] Compression encoding refers to a series of signal processing techniques used to transmit digitized information over communication lines or to store information in a form suitable for storage media. The objects of compression encoding include objects such as speech, video, and text, and in particular, techniques used to perform compression encoding on images are called video compression. Compression encoding of video signals is performed by removing excess information, taking into account spatial, temporal, and random correlations. However, with the recent development of various media and data transmission media, there is a need for more efficient video signal processing methods and devices. Summary of the Invention

[0004] Technical issues

[0005] One aspect of this invention is to improve the compilation efficiency of video signals. Specifically, this invention improves encoding or decoding efficiency by predicting chroma components based on the luminance components of the reconstructed current block.

[0006] Technical solution

[0007] To address this problem, the present invention provides a video signal processing apparatus and a video signal processing method.

[0008] A method for processing a video signal according to this disclosure includes: downsampling the reconstructed luminance components of blocks adjacent to the current block; obtaining the maximum luminance value among the downsampled luminance components based on a first index; obtaining the second highest luminance value among the downsampled luminance components based on a second index; obtaining a maximum luminance average value based on the average of the maximum luminance value and the second highest luminance value; obtaining the second lowest luminance value among the downsampled luminance components based on a third index; obtaining the minimum luminance value among the downsampled luminance components based on a fourth index; obtaining a minimum luminance average value based on the second lowest luminance value and the minimum luminance value; and obtaining a first luminance value among the chrominance components corresponding to the downsampled luminance components based on the first index. Chromaticity value; based on the second index, obtain the second chromaticity value in the chromaticity component; based on the average of the first chromaticity value and the second chromaticity value, obtain the maximum chromaticity average value; based on the third index, obtain the third chromaticity value in the chromaticity component; based on the fourth index, obtain the fourth chromaticity value in the chromaticity component; based on the average of the third chromaticity value and the fourth chromaticity value, obtain the minimum chromaticity average value; based on the maximum luminance average value, the minimum luminance average value, the maximum chromaticity average value, and the minimum chromaticity average value, obtain elements of a function indicating the correspondence between luminance and chromaticity; and predict the chromaticity value of the current block by applying the reconstructed luminance value of the current block to the function.

[0009] In the method for processing video signals according to this disclosure, obtaining elements of a function indicating the correspondence between luminance and chrominance includes: obtaining a first difference value as the difference between the maximum average luminance and the minimum average luminance; obtaining a second difference value as the difference between the maximum average chrominance and the minimum average chrominance; obtaining a first element (α) related to the ratio of the second difference value to the first difference value; and obtaining a second element (β) based on the first element, the minimum average luminance, and the minimum average chrominance; and predicting the chrominance value of the current block includes predicting the chrominance value of the current block based on the reconstructed luminance value of the current block, the first element (α), and the second element (β).

[0010] The method for processing video signals according to this disclosure further includes downsampling the reconstructed luminance component of the current block, wherein predicting the chroma value of the current block includes: predicting the chroma value of the current block based on the following equation: (chroma value of the current block) = (((luminance value included in the downsampled luminance component of the current block) * α) >> S) + β, where S is a constant.

[0011] The method for processing video signals according to this disclosure further includes: acquiring first information indicating whether a left block adjacent to the current block is available and second information indicating whether an upper block adjacent to the current block is available; when the first information indicates that the left block is available, acquiring the downsampled luminance component based on the left block; and when the second information indicates that the upper block is available, acquiring the downsampled luminance component based on the upper block.

[0012] In the method for processing video signals according to this disclosure, the downsampled luminance component is obtained based on samples included in at least one of the left block or the top block adjacent to the current block.

[0013] In the method for processing video signals according to this disclosure, the downsampled luminance component is obtained based on a sample included in at least one of the rightmost sample of the left block or the bottommost sample of the upper block.

[0014] In the method for processing video signals according to this disclosure, the downsampled luminance component is obtained based on samples selected at fixed intervals from the rightmost sample of the left block or the bottommost sample of the upper block.

[0015] An apparatus for processing video signals according to the present disclosure includes: a processor; and a memory, wherein the processor is configured to perform the following steps based on instructions stored in the memory: downsampling reconstructed luminance components of blocks adjacent to the current block; obtaining a maximum luminance value among the downsampled luminance components based on a first index; obtaining a second maximum luminance value among the downsampled luminance components based on a second index; obtaining a maximum luminance average value based on the average of the maximum luminance value and the second maximum luminance value; obtaining a second minimum luminance value among the downsampled luminance components based on a third index; obtaining a minimum luminance value among the downsampled luminance components based on a fourth index; obtaining a minimum luminance average value based on the second minimum luminance value and the minimum luminance value; and obtaining a maximum luminance average value based on the first index. The first chromaticity value of the chromaticity component corresponding to the downsampled luminance component; the second chromaticity value of the chromaticity component is obtained based on the second index; the maximum chromaticity average value is obtained based on the average of the first chromaticity value and the second chromaticity value; the third chromaticity value of the chromaticity component is obtained based on the third index; the fourth chromaticity value of the chromaticity component is obtained based on the fourth index; the minimum chromaticity average value is obtained based on the average of the third chromaticity value and the fourth chromaticity value; the elements of a function indicating the correspondence between luminance and chromaticity are obtained based on the maximum luminance average value, the minimum luminance average value, the maximum chromaticity average value, and the minimum chromaticity average value; and the chromaticity value of the current block is predicted by applying the reconstructed luminance value of the current block to the function.

[0016] In the apparatus for processing video signals according to the present disclosure, the processor is configured to perform the following steps based on instructions stored in the memory: obtaining a first difference value as the difference between the maximum average luminance and the minimum average luminance; obtaining a second difference value as the difference between the maximum average chroma and the minimum average chroma; obtaining a first element (α) related to the ratio of the second difference value to the first difference value; obtaining a second element (β) based on the first element, the minimum average luminance, and the minimum average chroma; and predicting the chroma value of the current block based on the reconstructed luminance value of the current block, the first element (α), and the second element (β).

[0017] In the apparatus for processing video signals according to the present disclosure, the processor is configured to perform the following steps based on instructions stored in the memory: downsampling the reconstructed luminance component of the current block; and predicting the chroma value of the current block based on the following equation: (chroma value of the current block) = (((luminance value included in the downsampled luminance component of the current block) * α) >> S) + β, where S is a constant.

[0018] In the apparatus for processing video signals according to the present disclosure, the processor is configured to perform the following steps based on instructions stored in the memory: acquiring first information indicating whether a left block adjacent to the current block is available and second information indicating whether an upper block adjacent to the current block is available; when the first information indicates that the left block is available, acquiring the downsampled luminance component based on the left block; and when the second information indicates that the upper block is available, acquiring the downsampled luminance component based on the upper block.

[0019] In the apparatus for processing video signals according to the present disclosure, the downsampled luminance component is obtained based on samples included in at least one of the left block or the upper block adjacent to the current block.

[0020] In the apparatus for processing video signals according to the present disclosure, the downsampled luminance component is obtained based on a sample included in at least one of the rightmost sample of the left block or the bottommost sample of the upper block.

[0021] In the apparatus for processing video signals according to the present disclosure, the downsampled luminance component is obtained based on samples selected at fixed intervals from the rightmost sample of the left block or the bottommost sample of the upper block.

[0022] A method for encoding a video signal according to this disclosure includes: downsampling the reconstructed luminance components of blocks adjacent to the current block; obtaining the maximum luminance value among the downsampled luminance components based on a first index; obtaining the second highest luminance value among the downsampled luminance components based on a second index; obtaining a maximum luminance average value based on the average of the maximum luminance value and the second highest luminance value; obtaining the second lowest luminance value among the downsampled luminance components based on a third index; obtaining the minimum luminance value among the downsampled luminance components based on a fourth index; obtaining a minimum luminance average value based on the second lowest luminance value and the minimum luminance value; and obtaining a first luminance value among the chrominance components corresponding to the downsampled luminance components based on the first index. Chromaticity value; based on the second index, obtain the second chromaticity value in the chromaticity component; based on the average of the first chromaticity value and the second chromaticity value, obtain the maximum chromaticity average value; based on the third index, obtain the third chromaticity value in the chromaticity component; based on the fourth index, obtain the fourth chromaticity value in the chromaticity component; based on the average of the third chromaticity value and the fourth chromaticity value, obtain the minimum chromaticity average value; based on the maximum luminance average value, the minimum luminance average value, the maximum chromaticity average value, and the minimum chromaticity average value, obtain elements of a function indicating the correspondence between luminance and chromaticity; and predict the chromaticity value of the current block by applying the reconstructed luminance value of the current block to the function.

[0023] An apparatus for encoding video signals according to the present disclosure includes: a processor; and a memory, wherein the processor is configured to perform the following steps based on instructions stored in the memory: downsampling reconstructed luminance components of blocks adjacent to the current block; obtaining a maximum luminance value among the downsampled luminance components based on a first index; obtaining a second maximum luminance value among the downsampled luminance components based on a second index; obtaining a maximum luminance average value based on the average of the maximum luminance value and the second maximum luminance value; obtaining a second minimum luminance value among the downsampled luminance components based on a third index; obtaining a minimum luminance value among the downsampled luminance components based on a fourth index; obtaining a minimum luminance average value based on the second minimum luminance value and the minimum luminance value; and obtaining a maximum luminance average value based on the first index. The first chromaticity value of the chromaticity component corresponding to the downsampled luminance component; the second chromaticity value of the chromaticity component is obtained based on the second index; the maximum chromaticity average value is obtained based on the average of the first chromaticity value and the second chromaticity value; the third chromaticity value of the chromaticity component is obtained based on the third index; the fourth chromaticity value of the chromaticity component is obtained based on the fourth index; the minimum chromaticity average value is obtained based on the average of the third chromaticity value and the fourth chromaticity value; the elements of a function indicating the correspondence between luminance and chromaticity are obtained based on the maximum luminance average value, the minimum luminance average value, the maximum chromaticity average value, and the minimum chromaticity average value; and the chromaticity value of the current block is predicted by applying the reconstructed luminance value of the current block to the function.

[0024] Beneficial effects

[0025] According to embodiments of this disclosure, the compilation efficiency of video signals can be improved. According to embodiments of this disclosure, the current block can be predicted more accurately based on intra-frame prediction modes. Attached Figure Description

[0026] Figure 1 This is a block diagram schematically illustrating a video signal encoding apparatus 100 according to an embodiment of the present disclosure.

[0027] Figure 2 This is a block diagram schematically illustrating a video signal decoding apparatus according to an embodiment of the present disclosure.

[0028] Figure 3 The illustration shows an example of dividing a compiler tree unit (CTU) into compiler units (CUs) within a picture.

[0029] Figure 4 The illustration shows an embodiment of a method for sending partitions of quadtrees and multi-type trees using signals.

[0030] Figure 5The intra-frame prediction method according to embodiments of the present disclosure is illustrated in more detail.

[0031] Figure 6 The intra-frame prediction method according to embodiments of the present disclosure is illustrated in more detail.

[0032] Figure 7 The figure illustrates an inter-frame prediction method according to an embodiment of the present disclosure.

[0033] Figure 8 The method of encoder conversion of residual signals is illustrated in detail.

[0034] Figure 9 The method of obtaining the residual signal by inversely converting the conversion coefficients is illustrated in detail by the encoder and decoder.

[0035] Figure 10 The illustration shows a cross-component linear model (CCLM) method according to an embodiment of the present disclosure.

[0036] Figure 11 The illustration shows a method for determining a linear model of a CCLM according to an embodiment of the present disclosure.

[0037] Figure 12 The illustration shows a method for determining a linear model of a CCLM according to an embodiment of the present disclosure.

[0038] Figure 13 The illustration shows an example of a downsampling filter for luminance samples required when performing intra-chroma prediction by applying a cross-component linear model (CCLM).

[0039] Figure 14 The illustration shows an example of applying a CCLM filter when using MRL.

[0040] Figure 15 The illustration shows an example of applying a CCLM filter when using MRL.

[0041] Figure 16 The illustration shows a cross-component linear model (CCLM) method according to an embodiment of the present disclosure.

[0042] Figure 17 The illustration shows a method for determining the number of reference samples in the CCLM method according to an embodiment of the present disclosure.

[0043] Figure 18 The illustration shows prediction mode signaling for intra-chroma prediction according to an embodiment of the present disclosure. Detailed Implementation

[0044] Considering the functions of this disclosure, the terminology used in this specification may be currently widely used and common, but may change according to the intent, practice, or emergence of new technologies of those skilled in the art. Additionally, in some cases, terms may be arbitrarily chosen by the applicant, and in such cases, their meanings are described in the corresponding descriptive sections of this disclosure. Therefore, the terminology used in this specification should be interpreted based on its substantive meaning throughout the specification.

[0045] In this specification, some terms may be interpreted as follows. In some cases, encoding can be interpreted as either encoding or decoding. In this specification, an apparatus for generating a video signal bitstream by performing encoding (encoding) of a video signal is called an encoding apparatus or encoder, and an apparatus for performing decoding (decoding) of a video signal bitstream to reconstruct a video signal is called a decoding apparatus or decoder. Additionally, in this specification, video signal processing apparatus is used as a term encompassing both the concepts of encoder and decoder. Information is a term that includes all values, parameters, coefficients, elements, etc. In some cases, the meaning is interpreted differently, therefore this disclosure is not limited thereto. "Unit" is used to refer to the basic unit of image processing or a specific location in an image, and refers to an image region including both luminance and chrominance components. Additionally, "block" refers to an image region including a specific component among the luminance and chrominance components (i.e., Cb and Cr). However, depending on the embodiment, terms such as "unit," "block," "partition," and "region" may be used interchangeably. Furthermore, in this specification, "unit" can be used as a concept encompassing all of the compilation unit, prediction unit, and transformation unit. The image refers to a field or frame, and according to embodiments, these terms may be used interchangeably.

[0046] Figure 1 This is a schematic block diagram of a video signal encoding apparatus according to an embodiment of the present disclosure.

[0047] refer to Figure 1 The encoding device 100 disclosed herein includes a transformation unit 110, a quantization unit 115, an inverse quantization unit 120, an inverse transformation unit 125, a filtering unit 130, a prediction unit 150, and an entropy compilation unit 160.

[0048] Transform unit 110 obtains the values ​​of transform coefficients by transforming the residual signal, which is the difference between the input video signal and the prediction signal generated by prediction unit 150. For example, discrete cosine transform (DCT), discrete sine transform (DST), or wavelet transform can be used. DCT and DST perform the transform by dividing the input image signal into multiple blocks. During the transform, the coding efficiency can vary depending on the distribution and characteristics of the values ​​in the transform region. Quantization unit 115 quantizes the values ​​of the transform coefficients output from transform unit 110.

[0049] To improve coding efficiency, instead of encoding the image signal as is, a method is used that predicts the image using the region already encoded by prediction unit 150, and obtains the reconstructed image by adding the residual value between the original image and the predicted image to the predicted image. To prevent mismatches in the encoder and decoder, information that can be used in the decoder should be used when performing prediction in the encoder. For this purpose, the encoder performs the processing of the current block of reconstruction encoding again. Inverse quantization unit 120 inverse quantizes the values ​​of the transform coefficients, and inverse transform unit 125 uses the inverse quantized transform coefficient values ​​to reconstruct the residual values. Simultaneously, filtering unit 130 performs filtering operations to improve the quality of the reconstructed image and improve coding efficiency. For example, it may include a deblocking filter, sample adaptive offset (SAO), and adaptive loop filter. The filtered image is output or stored in decoded image buffer (DPB) 156 for use as a reference image.

[0050] To improve compilation efficiency, instead of compiling the image signal as is, a method is used to predict the image via prediction unit 150 by using a compiled region and adding the residual value between the original image and the predicted image to the predicted image, thereby obtaining a reconstructed image. Intra-frame prediction unit 152 performs intra-frame prediction within the current image, and inter-frame prediction unit 154 predicts the current image using a reference image stored in decoded image buffer 156. Intra-frame prediction unit 152 performs intra-frame prediction from the reconstructed region in the current image and transmits intra-frame compilation information to entropy compilation unit 160. Inter-frame prediction unit 154 may include motion estimation unit 154a and motion compensation unit 154b. Motion estimation unit 154a obtains the motion vector value of the current region by referencing a specific reconstructed region. Motion estimation unit 154a transmits the position information of the reference region (reference frame, motion vector, etc.) to entropy compilation unit 160 so that the position information is included in the bitstream. Motion compensation unit 154b performs inter-frame motion compensation using the motion vector value transmitted from motion estimation unit 154a.

[0051] Prediction unit 150 includes intra-frame prediction unit 152 and inter-frame prediction unit 154. Intra-frame prediction unit 152 performs intra-frame prediction within the current image, and inter-frame prediction unit 154 performs inter-frame prediction to predict the current image using a reference image stored in DBP 156. Intra-frame prediction unit 152 performs intra-frame prediction from reconstructed samples in the current image and transmits intra-frame coding information to entropy compilation unit 160. Intra-frame coding information may include at least one of intra-frame prediction mode, most probable mode (MPM) flag, and MPM index. Intra-frame coding information may include information about the reference sample. Inter-frame prediction unit 154 may include motion estimation unit 154a and motion compensation unit 154b. Motion estimation unit 154a obtains motion vector values ​​for the current region by referencing a specific region of the reconstructed reference image. Motion estimation unit 154a transmits a set of motion information (reference image index, motion vector information, etc.) for the reference region to entropy compilation unit 160. The motion compensation unit 154b performs motion compensation by using motion vector values ​​passed from the motion estimation unit 154a. The inter-frame prediction unit 154 passes inter-frame coding information, including motion information about the reference region, to the entropy compilation unit 160.

[0052] According to another embodiment, prediction unit 150 may include an intra-block copy (BC) prediction unit (not shown). The intra-BC prediction unit performs intra-BC prediction based on reconstructed samples in the current image and transmits intra-BC coding information to entropy compilation unit 160. The intra-BC prediction unit obtains a specific region in the reference current image, indicating the block vector values ​​of the reference region used to predict the current region. The intra-BC prediction unit can use the obtained block vector values ​​to perform intra-BC prediction. The intra-BC prediction unit transmits intra-BC coding information to entropy compilation unit 160. The intra-BC coding information may include block vector information.

[0053] When performing the image prediction described above, the transformation unit 110 transforms the residual values ​​between the original image and the predicted image to obtain transformation coefficient values. In this case, the transformation can be performed on a unit of specific blocks within the image, and the size of the specific blocks can be changed within a preset range. The quantization unit 115 quantizes the transformation coefficient values ​​generated in the transformation unit 110 and sends them to the entropy compilation unit 160.

[0054] Entropy compilation unit 160 performs entropy compilation on information indicating quantized transform coefficients, intra-frame coding information, inter-frame coding information, etc., to generate a video signal bitstream. In entropy compilation unit 160, variable-length compilation (VLC) schemes, arithmetic compilation schemes, etc., can be used. Variable-length compilation (VLC) schemes involve transforming input symbols into continuous codewords, and the length of the codewords can be variable. For example, frequently occurring symbols are represented by short codewords, while rarely occurring symbols are represented by long codewords. Context-based adaptive variable-length compilation (CAVLC) can be used as a variable-length compilation scheme. Arithmetic compilation transforms continuous data symbols into single prime numbers, where arithmetic compilation obtains the optimal number of bits required to represent each symbol. Context-based adaptive binary arithmetic compilation (CABAC) can be used as arithmetic compilation. For example, entropy compilation unit 160 can binarize information indicating quantized transform coefficients. Entropy compilation unit 160 can generate a bitstream by arithmetically compiling binary information.

[0055] The generated bitstream is encapsulated using Network Abstraction Layer (NAL) units as the basic unit. Each NAL unit comprises an integer number of compiled tree units. To decode the bitstream in the video decoder, the bitstream must first be separated into NAL units, and then each separated NAL unit must be decoded. Simultaneously, the information required for decoding the video signal bitstream can be sent via Raw Byte Sequence Payloads (RBSPs) of higher-level sets such as Picture Parameter Sets (PPS), Sequence Parameter Sets (SPS), and Video Parameter Sets (VPS).

[0056] at the same time, Figure 1 The block diagram illustrates an encoding device 100 according to an embodiment of the present invention, and the separately displayed blocks logically distinguish and illustrate the elements of the encoding device 100. Therefore, depending on the device design, the elements of the encoding device 100 can be mounted as one or more chips. According to an embodiment, the operation of each element of the encoding device 100 can be performed by a processor (not shown).

[0057] Figure 2 This is a schematic block diagram of a video signal decoding apparatus 200 according to an embodiment of the present disclosure. (See reference...) Figure 2 The decoding device 200 of the present invention includes an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 225, a filtering unit 230, and a prediction unit 250.

[0058] Entropy decoding unit 210 performs entropy decoding on the video signal bitstream to extract transform coefficient information, intra-frame coding information, inter-frame coding information, etc., for each region. For example, entropy decoding unit 210 can obtain binarized codes for transform coefficient information of a specific region from the video signal bitstream. Entropy decoding unit 210 obtains quantized transform coefficients by inverse binarizing the binary code. Inverse quantization unit 220 inverse quantizes the quantized transform coefficients, and inverse transform unit 225 reconstructs the residual value using the inverse quantized transform coefficients. Video signal processing device 200 reconstructs the original pixel value by adding the residual value obtained by inverse transform unit 225 to the predicted value obtained by prediction unit 250.

[0059] Simultaneously, the filtering unit 230 performs filtering on the image to improve image quality. This may include a deblocking filter to reduce block distortion and / or an adaptive loop filter to remove distortion from the entire image. The filtered image is output or stored in the DPB 256 as a reference image for the next image.

[0060] Prediction unit 250 includes intra-frame prediction unit 252 and inter-frame prediction unit 254. Prediction unit 250 generates a prediction picture using the coding type decoded by the entropy decoding unit 210 described above, the transform coefficients of each region, and the intra / inter-frame coding information. To reconstruct the current block in which decoding is performed, the current picture or the decoded regions of other pictures including the current block can be used. In reconstruction, only the current picture, i.e., the picture (or tile / slice) that only performs intra-frame prediction or intra-frame BC prediction, is referred to as an intra-frame picture or I-picture (or tile / slice), and the picture (or tile / slice) that can perform all intra-frame prediction, inter-frame prediction, and intra-frame BC prediction is referred to as an inter-frame picture (or tile / slice). To predict the sample value of each block within an inter-frame image (or tile / slice), an image (or tile / slice) using at most one motion vector and a reference image index is called a prediction image or P-image (or tile / slice), and an image (or tile / slice) using at most two motion vectors and a reference image index is called a bidirectional prediction image or B-image (or tile / slice). In other words, a P-image (or tile / slice) uses at most one set of motion information to predict each block, and a B-image (or tile / slice) uses at most two sets of motion information to predict each block. Here, the set of motion information includes one or more motion vectors and a reference image index.

[0061] Intra-prediction unit 252 generates prediction blocks using intra-coding information and reconstructed samples in the current image. As described above, the intra-coding information may include at least one of intra-prediction mode, most probable mode (MPM) flag, and MPM index. Intra-prediction unit 252 predicts sample values ​​for the current block by using reconstructed samples located to the left and / or above the current block as reference samples. In this disclosure, the reconstructed samples, reference samples, and samples of the current block can represent pixels. Moreover, sample values ​​can represent pixel values.

[0062] According to an embodiment, the reference sample may be a sample included in the neighboring blocks of the current block. For example, the reference sample may be a sample adjacent to the left boundary of the current block and / or a sample adjacent to the top boundary. Moreover, the reference sample may be a sample among the samples of the neighboring blocks of the current block located on a line within a predetermined distance from the left boundary of the current block and / or a sample located on a line within a predetermined distance from the top boundary of the current block. In this case, the neighboring blocks of the current block may include the left (L) block, the top (A) block, the bottom left (BL) block, the top right (AR) block, or the top left (AL) block.

[0063] Inter-frame prediction unit 254 generates prediction blocks using reference images and inter-frame coding information stored in DPB 256. The inter-frame coding information may include a set of motion information (reference image index, motion vector information, etc.) for the current block used as the reference block. Inter-frame prediction may include L0 prediction, L1 prediction, and bidirectional prediction. L0 prediction means predicting using a reference image included in the L0 image list, while L1 prediction means predicting using a reference image included in the L1 image list. For this purpose, a set of motion information (e.g., motion vectors and reference image index) may be required. In the bidirectional prediction method, up to two reference regions can be used, and the two reference regions may exist in the same reference image or in different images. That is, in the bidirectional prediction method, up to two sets of motion information (e.g., motion vectors and reference image indexes) can be used, and the two motion vectors may correspond to the same reference image index or different reference image indices. In this case, in terms of time, reference images can be displayed (or output) before and after the current image. According to an embodiment, the two reference regions used in the bidirectional prediction scheme may be regions selected from image list L0 and image list L1, respectively.

[0064] The inter-frame prediction unit 254 can obtain a reference block for the current block using motion vectors and a reference image index. The reference block is located in the reference image corresponding to the reference image index. Furthermore, the sample value of the block specified by the motion vector, or its interpolated value, can be used as a predictor for the current block. For motion prediction with sub-pellet unit pixel accuracy, for example, an 8-tap interpolation filter for the luma signal and a 4-tap interpolation filter for the chroma signal can be used. However, the interpolation filter for motion prediction at the sub-pellet level is not limited to this. In this way, the inter-frame prediction unit 254 performs motion compensation to predict the texture of the current unit based on a motion image previously reconstructed using motion information. In such a case, the inter-frame prediction unit can use a set of motion information.

[0065] According to another embodiment, prediction unit 250 may include an intra-frame BC prediction unit (not shown). The intra-frame BC prediction unit can reconstruct a current region by referencing a specific region including reconstructed samples within the current image. The intra-frame BC prediction unit obtains intra-frame BC coding information of the current region from entropy decoding unit 210. The intra-frame BC prediction unit obtains block vector values ​​indicating a specific region in the current image. The intra-frame BC prediction unit can perform intra-frame BC prediction using the obtained block vector values. The intra-frame BC coding information may include block vector information.

[0066] The reconstructed video image is generated by adding the predicted value output from the intra-frame prediction unit 252 or the inter-frame prediction unit 254 to the residual value output from the inverse transform unit 225. That is, the video signal decoding device 200 uses the predicted block generated by the prediction unit 250 and the residual obtained from the inverse transform unit 225 to reconstruct the current block.

[0067] at the same time, Figure 2 The block diagram illustrates a decoding apparatus 200 according to an embodiment of the present disclosure, and the separately displayed blocks logically distinguish and illustrate the elements of the decoding apparatus 200. Therefore, depending on the device design, the elements of the decoding apparatus 200 can be mounted as one or more chips. According to an embodiment, the operation of each element of the decoding apparatus 200 can be performed by a processor (not shown).

[0068] Figure 3 The illustration shows an example where a compiler tree unit (CTU) is divided into compiler units (CUs) in the image.

[0069] During the compilation of a video signal, an image can be segmented into a series of compilation tree units (CTUs). A compilation tree unit consists of N×N blocks of luminance samples and two blocks of corresponding chrominance samples. A compilation tree unit can be segmented into multiple compilation units. A compilation tree unit may not be segmented and may be a leaf node. In this case, the compilation tree unit itself can be a compilation unit. A compilation unit refers to the basic unit used to process images during the aforementioned video signal processing, i.e., intra / inter-frame prediction, transform, quantization, and / or entropy compilation. The size and shape of a compilation unit in an image may not be constant. Compilation units can have square or rectangular shapes. Rectangular compilation units (or rectangular blocks) include vertical compilation units (or vertical blocks) and horizontal compilation units (or horizontal blocks). In this specification, a vertical block is a block whose height is greater than its width, and a horizontal block is a block whose width is greater than its height. Furthermore, in this specification, a non-square block may refer to a rectangular block, but this disclosure is not limited thereto.

[0070] refer to Figure 3 First, the compiler tree unit is divided into a quadtree (QT) structure. That is, in a quadtree structure, a node of size 2×2N can be divided into four nodes of size N×N. In this specification, a quadtree can also be referred to as a quaternion tree. Quadtree partitioning can be performed recursively, and not all nodes need to be partitioned at the same depth.

[0071] Simultaneously, the leaf nodes of the aforementioned quadtree can be further divided into multi-type tree (MTT) structures. According to embodiments of this disclosure, in a multi-type tree structure, a node can be divided into a horizontally or vertically partitioned binary or ternary tree structure. That is, in a multi-type tree structure, there are four partitioning structures, such as vertical binary partitioning, horizontal binary partitioning, vertical ternary partitioning, and horizontal ternary partitioning. According to embodiments of this disclosure, in each tree structure, the width and height of the node can both be powers of 2. For example, in a binary tree (BT) structure, a node of size 2×2N can be partitioned into two NX2N nodes through vertical binary partitioning, and further partitioned into two 2NXN nodes through horizontal binary partitioning. Additionally, in a ternary tree (TT) structure, a node of size 2NX2N is partitioned into (N / 2)×2N, NX2N, and (N / 2)×2N nodes through vertical ternary partitioning, and further partitioned into 2NX(N / 2), 2NXN, and 2NX(N / 2) nodes through horizontal ternary partitioning. This multi-type tree split can be performed recursively.

[0072] Leaf nodes of multi-type trees can be compilation units. If no partitioning for a compilation unit is specified or the compilation unit is small for the maximum transform length, the compilation unit can be used as the unit for prediction and transformation without further partitioning. Alternatively, at least one of the following parameters for quadtrees and multi-type trees can be predefined or sent through RBSPs of higher-level collections such as PPS, SPS, VPS, etc.: 1) CTU size: the size of the root node of the quadtree, 2) Minimum QT size MinQtSize: the minimum allowed QT leaf node size, 3) Maximum BT size MaxBtSize: the maximum allowed BT root node size, 4) Maximum TT size MaxTtSize: the maximum allowed TT root node size, 5) Maximum MTT depth MaxMttDepth: the maximum allowed depth of the MTT derived from the leaf nodes of the QT, 6) Minimum BT size MinBtSize: the minimum allowed BT leaf node size, 7) Minimum TT size MinTtSize: the minimum allowed TT leaf node size.

[0073] Figure 4 An embodiment of a method for signaling the segmentation of quadtrees and multi-type trees is shown. Preset flags can be used to signal the segmentation of the aforementioned quadtrees and multi-type trees. Reference Figure 4 At least one of the following flags can be used: "qt_split_flag" indicating whether to split a quadtree node, "mtt_split_flag" indicating whether to split a multi-type tree node, "mtt_split_vertical_flag" indicating the split direction of a multi-type tree node, or "mtt_split_binary_flag" indicating the split type of a multi-type tree node.

[0074] According to embodiments of this disclosure, the compiler tree unit is the root node of a quadtree and can be initially partitioned into a quadtree structure. In the quadtree structure, a "qt_split_flag" signal is sent to each node "QT_node". If the value of "qt_split_flag" is 1, the node is partitioned into four square nodes, and if the value of "qt_split_flag" is 0, the corresponding node becomes a leaf node "QT_leaf_node" of the quadtree.

[0075] Each quadtree leaf node "QT_leaf_node" can be further segmented into a multi-type tree structure. In the multi-type tree structure, "mtt_split_flag" is signaled for each node "MTT_node". When the value of "mtt_split_flag" is 1, the corresponding node is split into multiple rectangular nodes, and when the value of "mtt_split_flag" is 0, the corresponding node is a leaf node "MTT_leaf_node" in the multi-type tree. When the multi-type tree node "MTT_node" is split into multiple rectangular nodes (i.e., when the value of "mtt_split_flag" is 1), the "mtt_split_vertical_flag" and "mtt_split_binary_flag" of the node "MTT_node" can be additionally signaled. When the value of "mtt_split_vertical_flag" is 1, it indicates the vertical split of the node "MTT_node", and when the value of "mtt_split_vertical_flag" is 0, it indicates the horizontal split of the node "MTT_node". Additionally, when the value of "mtt_split_binary_flag" is 1, the node "MTT_node" is split into 2 rectangular nodes, and when the value of "mtt_split_binary_flag" is 0, the node "MTT_node" is split into 3 rectangular nodes.

[0076] Image prediction (motion compensation) for encoding is performed on compilation units that are no longer divided (i.e., leaf nodes of the compilation unit tree). In the following text, the basic unit used to perform the prediction will be referred to as a "prediction unit" or "prediction block".

[0077] In the following text, the term "unit" as used herein may be used in place of a prediction unit, which is the basic unit used to perform prediction. However, this disclosure is not limited thereto, and "unit" may be understood to broadly encompass the concept of a compilation unit.

[0078] Figure 5 The intra-frame prediction method according to embodiments of the present disclosure is illustrated in more detail. Figure 6 The intra-frame prediction method according to embodiments of the present disclosure is illustrated in more detail.

[0079] As described above, the intra-frame prediction unit predicts the sample value of the current block by using the reconstructed sample located to the left and / or above the current block as a reference sample.

[0080] first, Figure 5An embodiment of reference samples for prediction of the current block in intra-frame prediction mode is shown. According to the embodiment, the reference samples may be samples adjacent to the left boundary and / or the top boundary of the current block. Figure 5 As shown, when the size of the current block is WXH and a single reference line adjacent to the current block is used for intra-frame prediction, the reference samples can be configured using the maximum 2W+2H+1 neighboring samples located to the left and above the current block.

[0081] When at least some samples to be used as reference samples have not yet been reconstructed, the intra-prediction unit can obtain reference samples by performing a reference sample padding process. The intra-prediction unit can perform a reference sample filtering process to reduce errors in intra-prediction. That is, filtering can be performed on neighboring samples and / or reference samples obtained through the reference sample padding process to obtain filtered reference samples. The intra-prediction unit predicts samples for the current block using the reference samples obtained as described above. The intra-prediction unit predicts samples for the current block using either unfiltered or filtered reference samples. In this disclosure, neighboring samples can include samples on at least one reference line. For example, neighboring samples can include neighboring samples on lines adjacent to the boundary of the current block.

[0082] Next, Figure 6 An embodiment of a prediction mode for intra-frame prediction is illustrated. For intra-frame prediction, intra-frame prediction mode information indicating the direction of intra-frame prediction can be transmitted via signaling. The intra-frame prediction mode information indicates one of a plurality of intra-frame prediction modes included in the set of intra-frame prediction modes. When the current block is an intra-frame prediction block, the decoder receives the intra-frame prediction mode information of the current block from the bitstream. The decoder's intra-frame prediction unit performs intra-frame prediction on the current block based on the extracted intra-frame prediction mode information.

[0083] According to embodiments of this disclosure, the intra-prediction mode set may include all intra-prediction modes used in intra-prediction (e.g., a total of 67 intra-prediction modes). More specifically, the intra-prediction mode set may include planar modes, DC modes, and multiple (e.g., 65) angle modes (i.e., directional modes). Each intra-prediction mode can be indicated by a preset index (i.e., an intra-prediction mode index). For example, as... Figure 6As shown, intra-prediction mode index 0 indicates a planar mode, while intra-prediction mode index 1 indicates a DC mode. Furthermore, intra-prediction mode indices 2 through 66 can each indicate different angle modes. Each angle mode indicates an angle that differs from the others within a preset angle range. For example, an angle mode can indicate an angle within a clockwise angle range of 45 degrees and -135 degrees (i.e., a first angle range). An angle mode can be defined based on the 12 o'clock direction. In this case, intra-prediction mode index 2 indicates a horizontal diagonal (HDIA) mode, intra-prediction mode index 18 indicates a horizontal (horizontal, HOR) mode, intra-prediction mode index 34 indicates a diagonal (DIA) mode, intra-prediction mode index 50 indicates a vertical (VER) mode, and intra-prediction mode index 66 indicates a vertical diagonal (VDIA) mode.

[0084] In the following text, reference will be made to Figure 7 An inter-frame prediction method according to embodiments of the present disclosure is described. In this disclosure, the inter-frame prediction method may include a general inter-frame prediction method optimized for translational motion, and an inter-frame prediction method based on an affine model. The motion vector may include at least one of a general motion vector for motion compensation according to the general inter-frame prediction method and a control point motion vector for affine motion compensation.

[0085] Figure 7 An inter-frame prediction method according to an embodiment of the present disclosure is illustrated. As described above, the decoder can predict the current block by referring to a reconstructed sample of another decoded image. (Reference) Figure 7 The decoder obtains reference block 702 in reference image 720 based on the motion information set of current block 701. The motion information set may include a reference image index and a motion vector 703. The reference image index indicates a reference image 720 in the reference image list that includes a reference block for inter-frame prediction of the current block. According to an embodiment, the reference image list may include at least one of the aforementioned L0 image list and L1 image list. The motion vector 703 represents the offset between the coordinate values ​​of the current block 701 in the current image 710 and the coordinate values ​​of the reference block 702 in the reference image 720. Based on the sample values ​​of reference block 702, the decoder obtains a predictor for the current block 701 and reconstructs the current block 701 using the predictor.

[0086] Specifically, the encoder can obtain the aforementioned reference block by searching for blocks similar to the current block from images with an earlier reconstruction order. For example, the encoder can search for a reference block where the sum of the differences between the current block and the sample values ​​is minimized within a pre-configured search region. To measure the similarity between samples in the current block and the reference block, at least one of the sum of absolute differences (SAD) or sum of Hadamard transform differences (SATD) can be used. SAD can be a value obtained by summing all the absolute values ​​of the individual differences between the sample values ​​included in the two blocks. SATD can be a value obtained by adding all the absolute values ​​of the Hadamard transform coefficients obtained by performing a Hadamard transform on the differences between the sample values ​​included in the two blocks.

[0087] One or more reference regions can be used to predict the current block. As described above, two or more reference regions can be used for inter-frame prediction of the current block via a bidirectional prediction scheme. According to an embodiment, the decoder can obtain two reference blocks based on two sets of motion information of the current block. The decoder can obtain a first predictor and a second predictor for the current block based on the sample values ​​of the two corresponding obtained reference blocks. The decoder can reconstruct the current block by using the first predictor and the second predictor. For example, the decoder can reconstruct the current block based on the average of each sample from the first predictor and the second predictor.

[0088] As described above, for motion compensation of the current block, one or more sets of motion information can be signaled. The similarity between the motion information sets used for motion compensation of each of multiple blocks can be used. For example, the predicted motion information set for the current block can be derived from the motion information set used to predict one of the other samples previously reconstructed. Therefore, the encoder and decoder can reduce signaling overhead.

[0089] For example, there may be multiple candidate blocks that have already been predicted based on motion information sets that are the same as or similar to the current block's motion information set. The decoder can generate a merged candidate list based on these multiple candidate blocks. The merged candidate list can include candidates corresponding to samples from samples reconstructed before the current block that may have already been predicted based on motion information sets related to the current block's motion information set. The encoder and decoder can configure the merged candidate list for the current block according to predetermined rules. The merged candidate lists configured by each of the encoder and decoder can be identical to each other. For example, the encoder and decoder can configure the merged candidate list for the current block based on the current block's position within the current image. See below for further details. Figure 9 This disclosure describes a method for configuring a list of merge candidates for the current block using an encoder and decoder. In this disclosure, the position of a specific block indicates its relative position to the top-left sample of that specific block within an image that includes it.

[0090] Meanwhile, to improve compilation efficiency, the transform coefficients obtained by quantizing the residual signal can be compiled instead of compiling the residual as is. As mentioned above, the transform unit can obtain transform coefficient values ​​by transforming the residual signal. In this case, the residual signal of a specific block can be distributed across the entire region of the current block. Therefore, by frequency domain transformation of the residual signal, energy can be concentrated in the low-frequency region, thereby improving compilation efficiency. The methods for transforming or inversely transforming the residual signal will be described in detail below.

[0091] Figure 8 This diagram specifically illustrates a method for transforming residual signals using an encoder. As described above, residual signals in the spatial domain can be transformed into the frequency domain. The encoder can obtain transform coefficients by transforming the obtained residual signals. First, the encoder can obtain at least one residual block comprising residual signals for the current block. The residual block can be the current block or one of the blocks partitioned from the current block. In this disclosure, the residual block can be referred to as a residual array or residual matrix, which includes residual samples of the current block. In this disclosure, the residual block can represent a transform unit or a block having the same size as the transform block.

[0092] Next, the encoder can transform the residual block using a transform kernel. The transform kernel used to transform the residual block can be a transform kernel with separable characteristics of vertical and horizontal transforms. In this case, the transform for the residual block can be performed as a vertical transform and a horizontal transform, respectively. For example, the encoder can perform a vertical transform by applying the transform kernel in the vertical direction of the residual block. The encoder can perform a horizontal transform by applying the transform kernel in the horizontal direction of the residual block. In this disclosure, the transform kernel can be used as a term referring to a set of parameters used for transforming the residual signal, such as a transform matrix, transform array, transform function, and transform. According to embodiments, the transform kernel can be any one of a plurality of available kernels. Transform kernels based on different transform types can be used for each of the vertical and horizontal transforms.

[0093] The encoder can pass a transform block transformed from the residual block to the quantization unit and quantize the transform block. The transform block can include multiple transform coefficients. Specifically, the transform block can include multiple transform coefficients arranged in two dimensions. Similar to the case of the residual block, the size of the transform block can be the same as the size of the current block or a block partitioned from the current block. The transform coefficients passed to the quantization unit can be represented as quantized values.

[0094] The encoder can perform an additional transformation before quantizing the transform coefficients. For example... Figure 8As shown, the above transformation method can be referred to as the primary transformation, and the additional transformation can be referred to as the secondary transformation. The secondary transformation is optional for each residual block. According to an embodiment, the encoder can improve compilation efficiency by performing a secondary transformation on regions where it is difficult to concentrate energy in the low-frequency region using only the primary transformation. For example, a secondary transformation can be added to blocks where the residual values ​​exhibit large values ​​in directions other than the horizontal or vertical directions of the residual block. Compared to the residual values ​​of inter-frame prediction blocks, the residual values ​​of intra-frame prediction blocks have a higher transformation probability in directions other than the horizontal or vertical directions. Therefore, the encoder can additionally perform a secondary transformation on the residual signals of intra-frame prediction blocks. The encoder can omit the secondary transformation for the residual signals used in inter-frame prediction blocks.

[0095] As another example, whether to perform a secondary transformation may depend on the size of the current block or the size of the residual block. Transform kernels of different sizes may be used, depending on the size of the current block or the size of the residual block. For example, an 8×8 secondary transformation may be applied to blocks where the shorter side between the width and height is shorter than a first pre-configured length. Furthermore, a 4×4 secondary transformation may be applied to blocks where the shorter side between the width and height is longer than a second pre-configured length. Here, the first pre-configured length may be a larger value than the second pre-configured length, but this disclosure is not limited thereto. Unlike the primary transformation, the secondary transformation may not be performed as a separate vertical and horizontal transformation. This secondary transformation may be referred to as a low-frequency band inseparable transformation (LFNST).

[0096] In the case of video signals in a specific region, even if a frequency transformation is performed due to a sudden change in brightness, the energy in the high-frequency band will not be reduced. Therefore, the degradation may be attributed to the compression performance of quantization. When performing a transformation on a region where residual values ​​are scarce, encoding and decoding times may be unnecessarily increased. Therefore, the transformation of the residual signal in a specific region can be omitted. Whether to perform a transformation on the residual signal in a specific region can be determined by syntax elements related to the transformation of that specific region. For example, syntax elements can include transform skip information. The transform skip information can be a transform skip flag. If the transform skip information for the residual block indicates a transform skip, then no transformation is performed on the residual block. In this case, the encoder can immediately quantize the residual signal for which the corresponding region has not been transformed. This can be achieved via... Figure 1 The transformation unit performs the reference Figure 8 The operation of the encoder is described.

[0097] The aforementioned transform-related syntax elements can be information parsed from the video signal bitstream. The decoder can perform entropy decoding on the video signal bitstream to obtain the transform-related syntax elements. The encoder can perform entropy compilation on the transform-related syntax elements to generate the video signal bitstream.

[0098] Figure 9 This diagram illustrates a method for obtaining a residual signal by performing an inverse transform on the transform coefficients using an encoder and a decoder. For ease of description, the inverse transform operation will be described via an inverse transform unit in each of the encoder and decoder. The inverse transform unit can obtain the residual signal by performing an inverse transform on the inverse-quantized transform coefficients. First, the inverse transform unit can detect whether an inverse transform is to be performed on a specific region from the transform-related syntax elements of the region. According to an embodiment, when the transform-related syntax element for a specific transform block indicates that the transform is skipped, the transform of the transform block can be omitted. In this case, the aforementioned first and second inverse transforms of the transform block can be omitted. The inverse-quantized transform coefficients can be used as the residual signal. For example, the decoder can reconstruct the current block by using the inverse-quantized transform coefficients as the residual signal.

[0099] According to another embodiment, the transform-related syntax elements for a specific transform block may not indicate transform skipping. In this case, the inverse transform unit can determine whether to perform a second inverse transform on the second transform. For example, when the transform block is a transform block of an intra-prediction block, a second inverse transform can be performed on the transform block. The second transform kernel for the transform block can be determined based on the intra-prediction mode corresponding to the transform block. As another example, whether to perform a second inverse transform can be determined based on the size of the transform block. The second inverse transform can be performed after inverse quantization and before the first inverse transform.

[0100] The inverse transform unit can perform a primary inverse transform on the inverse-quantized transform coefficients or the transform coefficients of a secondary inverse transform. In the case of the primary inverse transform, as with the primary transform, the vertical and horizontal transforms can be performed separately. For example, the inverse transform unit can obtain a residual block by performing a vertical and horizontal inverse transform on the transform block. The inverse transform unit can perform an inverse transform on the transform block based on a transform kernel used to transform the transform block. For example, the encoder can explicitly or implicitly signal information indicating which of a plurality of available transform kernels will be applied to the current transform block. The decoder can select the transform kernel to be used for the inverse transform of the transform block from among the plurality of available transform kernels by using the information indicating the transform kernel signaled. The inverse transform unit can reconstruct the current block using the residual signal obtained by performing an inverse transform on the transform coefficients.

[0101] Figure 10 The illustration shows a cross-component linear model (CCLM) method according to an embodiment of the present disclosure.

[0102] The minimum and maximum values ​​of luminance are obtained using the reconstructed luminance and chrominance values ​​adjacent to the current block, and equation (1) is derived using these two values ​​from equations (2) and (3). Finally, the chrominance is predicted using the reconstructed luminance values ​​of the current block via equation (4). When the image format is 4:2:0, the number of luminance samples is adjusted to the number of chrominance samples by applying a downsampling filter.

[0103] More specifically, the encoder or decoder may perform a step of downsampling the luminance component of the reconstructed block adjacent to the current block. The reconstructed block adjacent to the current block may be at least one of the block above or to the left of the current block.

[0104] In a YCbCr image system, subsampling types can include 8:4:4, 4:4:4, 4:2:2, 4:1:1, and 4:2:0. In the 4:4:4 case, the luma component and the two chromaticity components can have the same number of samples, but in the 4:2:2 or 4:2:0 cases, the number of samples for the luma component can be greater than the number of samples for the chromaticity components. Therefore, the encoder or decoder can downsample the luma component so that the number of samples for the luma component is appropriate for the number of samples for the chromaticity components. That is, the encoder or decoder can establish a one-to-one correspondence between the luma and chromaticity components and derive a linear equation between them. For example, the linear equation could be... Figure 10 Equation (1).

[0105] The encoder or decoder can reconstruct at least one of the previous block or the left block of the current block earlier than the current block. The encoder or decoder can predict the value of the chroma sample of the current block based on the sample values ​​of the reconstructed previous or left block.

[0106] Figure 10 The vertical axis represents the chromaticity values ​​of the pixels included in the upper or left block of the current block. The horizontal axis represents the luminance values ​​of the pixels included in the upper or left block of the current block.

[0107] The chromaticity values ​​in this disclosure can indicate sample values ​​of the chromaticity components of the pixels included in the block. Similarly, the luminance values ​​in this disclosure can indicate sample values ​​of the luminance components of the pixels included in the block.

[0108] An encoder or decoder can acquire two pixels from at least one of the upper or left blocks to obtain equation (2). For example, the encoder or decoder can acquire pixel A and pixel B. The encoder or decoder can acquire the x-axis as the luminance component of point pixel A. A And obtain the y-value as the chromaticity component of pixel A. A The encoder or decoder can obtain the x-axis as the luminance component of pixel B. BAnd obtain the y-value as the chromaticity component of pixel B. B .

[0109] The encoder or decoder can derive equations (1) to (4) using the luminance and chrominance components of pixel A and pixel B. Furthermore, the encoder or decoder can predict the chrominance component of the current block by applying equation (4) to the reconstructed luminance component of the current block.

[0110] Figure 11 The illustration shows a method for determining a linear model of a CCLM according to an embodiment of the present disclosure.

[0111] Unlike Figure 10 The method, in Figure 11 In China, based on Figure 11 F (which is the maximum value of brightness) Figure 11 C) and the second highest value ( Figure 11 The average value of D) and Figure 11 E (which is the minimum value of brightness) Figure 11 A) and the second lowest minimum value ( Figure 11 The average value of B) is used Figure 10 Equations (2) and (3) are used to derive equation (1). This can mitigate the distortion that the maximum / minimum values ​​may have. Although Figure 11 An example of averaging two sample values ​​for the maximum / minimum and the second-highest / second-lowest values ​​has been described, but it is also possible to use the average within a predetermined number of maximum and minimum value groups, and to average values ​​over a specific range of values.

[0112] More specifically, the encoder or decoder can perform a step of downsampling the luma component of the reconstructed block adjacent to the current block. The adjacent reconstructed block can be the block above or to the left of the current block. The encoder or decoder can downsample the luma component of the adjacent block so that the number of samples for the luma component is appropriate for the number of samples for the chroma component. That is, the encoder or decoder can make a one-to-one correspondence between the luma component and the chroma component and derive a linear equation for the luma component and the chroma component.

[0113] The encoder or decoder can perform the step of obtaining the maximum luminance value among the downsampled luminance components based on a first index. For example, the maximum luminance value is obtained as shown in the following equation.

[0114] Maximum brightness value = downsampled brightness component [first index]

[0115] In this disclosure, the luminance value can indicate a sample value of the luminance component of a pixel included in the block. The first index can correspond to Figure 11C. The encoder or decoder can obtain the first index based on a predetermined process. The first index may be an index indicating the maximum luminance value among the downsampled luminance components. For example, refer to Figure 11 The encoder or decoder can obtain Cx in the downsampled luminance component based on the first index.

[0116] The encoder or decoder can obtain downsampled luminance components from the reconstructed upper or left block. The encoder or decoder can use only some of the downsampled luminance components. For example, downsampled luminance components can be obtained based on samples included in at least one of the rightmost sample of the left block or the bottommost sample of the upper block. Alternatively, downsampled luminance components can be obtained based on samples selected at fixed intervals from the rightmost sample of the left block or the bottommost sample of the upper block.

[0117] The encoder or decoder can perform the step of obtaining the second highest luminance value among the downsampled luminance components based on the second index. For example, the second highest luminance value is obtained as shown in the following equation.

[0118] Second highest brightness value = downsampled brightness component [Second Index]

[0119] The second index can correspond to Figure 11 The D. encoder or decoder can obtain the second index based on a predetermined process. The second index can be an index indicating the second highest luminance value among the downsampled luminance components. For example, referencing Figure 11 The encoder or decoder can obtain the Dx in the downsampled luminance component based on the second index.

[0120] The encoder or decoder can perform a step to obtain the maximum brightness average value based on the average of the maximum brightness value and the second highest brightness value. The maximum brightness average value can correspond to... Figure 11 The brightness value F.

[0121] The encoder or decoder can obtain the average maximum brightness value based on the formula "(maximum brightness value + second highest brightness value) >> 1" or "(maximum brightness value + second highest brightness value + 1) >> 1". The average value can be an integer. Alternatively, the encoder or decoder can obtain the average value obtained without rounding to the nearest decimal point based on "(maximum brightness value + second highest brightness value) >> 1". Furthermore, the encoder or decoder can obtain the average value obtained by rounding to the nearest decimal point based on "(maximum brightness value + second highest brightness value + 1) >> 1".

[0122] The encoder or decoder can perform the step of obtaining the second lowest luminance value in the downsampled luminance component based on a third index. For example, the second lowest luminance value is obtained as shown in the following equation.

[0123] Second lowest brightness value = downsampled brightness component [Third Index]

[0124] The third index can correspond to Figure 11 B. The encoder or decoder can obtain the third index based on a predetermined process. The third index can be an index indicating the second lowest luminance value among the downsampled luminance components. For example, referencing Figure 11 The encoder or decoder can obtain Bx from the downsampled luminance component based on the third index.

[0125] The encoder or decoder can perform the step of obtaining the minimum luminance value among the downsampled luminance components based on the fourth index. For example, the minimum luminance value is obtained as shown in the following equation.

[0126] Maximum brightness value = downsampled brightness component [Fourth Index]

[0127] The fourth index can correspond to Figure 11 A. The encoder or decoder can obtain the fourth index based on a predetermined process. The fourth index can be an index indicating the minimum luminance value among the downsampled luminance components. For example, refer to... Figure 11 The encoder or decoder can obtain Ax from the downsampled luminance component based on the fourth index.

[0128] The encoder or decoder can perform the step of obtaining the minimum brightness average value based on the second lowest brightness value and the minimum brightness value. The minimum brightness average value can correspond to... Figure 11 The brightness value of E in the equation. The encoder or decoder can obtain the average minimum brightness value based on the formula "(second lowest brightness value + minimum brightness value) >> 1" or "(second lowest brightness value + minimum brightness value + 1) >> 1". Since the method for obtaining the average value has already been described, its description will be omitted.

[0129] The method for obtaining the first to fourth indices is described below. The encoder or decoder can configure the first to fourth indices as initial values ​​to obtain them. The third to fourth indices can have different values. The encoder or decoder can obtain the third value of the downsampled luminance component based on the third index and the fourth value of the downsampled luminance component based on the fourth index. If the fourth value is greater than the third value, the encoder or decoder can swap the values ​​of the third and fourth indices. That is, the encoder or decoder can determine that the swapped value of the third index is the same as the original value of the fourth index, and the swapped value of the fourth index is the same as the original value of the third index.

[0130] Furthermore, after the above process, the encoder or decoder can obtain a first value in the downsampled luminance component based on a first index, and a second value in the downsampled luminance component based on a second index. When the second value is greater than the first value, the encoder or decoder can swap the values ​​of the first index and the second index. That is, the encoder or decoder can determine that the value of the swapped first index is the value of the original second index, and the value of the swapped second index is the value of the original first index.

[0131] Furthermore, after the above process, the encoder or decoder can obtain a first value in the downsampled luminance component based on a first index, and a fourth value in the downsampled luminance component based on a fourth index. When the fourth value is greater than the first value, the encoder or decoder can swap the values ​​of the first and third indices and swap the values ​​of the second and fourth indices. That is, the encoder or decoder can determine that the value of the first index after the swap is the value of the third index before the swap, the value of the third index after the swap is the value of the first index before the swap, the value of the second index after the swap is the value of the fourth index before the swap, and the value of the fourth index after the swap is the value of the second index before the swap.

[0132] Furthermore, after the above process, the encoder or decoder can obtain the second value of the downsampled luminance component based on the second index, and the third value of the downsampled luminance component based on the third index. When the third value is greater than the second value, the encoder or decoder can swap the values ​​of the second and third indices. That is, the encoder or decoder can determine that the value of the swapped second index is the value of the original third index, and the value of the swapped third index is the value of the original second index.

[0133] The process of determining the first to fourth indices has been described above. However, the process performed by the encoder or decoder is not limited to the above. The encoder or decoder can use various methods to arrange the indices and luminance values ​​such that the first to fourth indices sequentially indicate the maximum, second-highest, second-lowest, and minimum luminance values ​​among the sampled luminance components.

[0134] The encoder or decoder may perform the step of obtaining a first chromaticity value from the chromaticity component corresponding to the downsampled luminance component based on a first index. For example, the first chromaticity value is obtained as shown in the following equation.

[0135] First chromaticity value = chromaticity component [first index]

[0136] As described above, in this disclosure, the chromaticity value can indicate the sample value of the chromaticity component of the pixel included in the block. Furthermore, the first index can correspond to... Figure 11 A. Reference Figure 11The encoder or decoder can obtain the Cy component in the chromaticity component based on the first index.

[0137] The first index may indicate the maximum luminance value among the downsampled luminance components. However, the first chrominance value obtained based on the first index in the corresponding chrominance component may or may not be the maximum value of the chrominance component. That is, the first index does not indicate that the first chrominance value is the maximum value of the chrominance component.

[0138] The encoder or decoder can perform the step of obtaining a second chromaticity value from the chromaticity components based on a second index. For example, the second chromaticity value is obtained as shown in the following equation.

[0139] Second chromaticity value = chromaticity component [second index]

[0140] As mentioned above, the second index can correspond to Figure 11 D. Reference Figure 11 The encoder or decoder can obtain Dy from the chroma component based on the second index.

[0141] The second index can be an index indicating the second highest luminance value among the downsampled luminance components. However, among the chrominance components corresponding to the downsampled luminance components, the second chrominance value obtained based on the second index may or may not be the second highest value of the chrominance component. That is, the second index does not indicate that the second chrominance value is the second highest value of the chrominance component. For example, the second chrominance value can be greater than, equal to, or less than the first chrominance value.

[0142] The encoder or decoder can perform the step of obtaining the maximum chromaticity average value based on the average of the first chromaticity value and the second chromaticity value. The minimum chromaticity average value can correspond to... Figure 11 The chromaticity value of F in the equation.

[0143] The encoder or decoder can obtain the maximum chromaticity average value based on the formula "(first chromaticity value + second chromaticity value) >> 1" or "(first chromaticity value + second chromaticity value + 1) >> 1". Since the method for obtaining the average value has already been described, its description will be omitted.

[0144] An encoder or decoder can perform the step of retrieving the third chromaticity value from the chromaticity components based on the third index. For example, the third chromaticity value is retrieved as shown in the following equation.

[0145] Third chromaticity value = chromaticity component [third index]

[0146] As mentioned above, the third index can correspond to Figure 11 B. Reference Figure 11 The encoder or decoder can obtain the By in the chroma component based on the third index.

[0147] The third index can indicate the second lowest luminance value among the downsampled luminance components. However, among the chrominance components corresponding to the downsampled luminance components, the third chrominance value obtained based on the third index may or may not be the second lowest value of the chrominance component. That is, the third index does not indicate that the third chrominance value is the second lowest value of the chrominance component.

[0148] An encoder or decoder can perform the step of obtaining the fourth chromaticity value in the chromaticity component based on the fourth index. For example, the fourth chromaticity value is obtained as shown in the following equation.

[0149] Fourth chromaticity value = chromaticity component [fourth index]

[0150] As mentioned above, the fourth index can correspond to Figure 11 A. Reference Figure 11 The encoder or decoder can obtain Ay from the chromaticity component based on the fourth index.

[0151] The fourth index can be an index indicating the minimum luminance value among the downsampled luminance components. However, among the chrominance components corresponding to the downsampled luminance components, the fourth chrominance value obtained based on the fourth index may or may not be the minimum value of the chrominance component. That is, the fourth index does not indicate that the fourth chrominance value is the minimum value of the chrominance component. For example, the fourth chrominance value can be greater than, equal to, or less than the third chrominance value.

[0152] The encoder or decoder can perform the step of obtaining a minimum chromaticity average based on the average of the third and fourth chromaticity values. The minimum chromaticity average can correspond to... Figure 11 The chromaticity value of E in the equation.

[0153] The encoder or decoder can obtain the minimum chromaticity average value based on the formula "(third chromaticity value + fourth chromaticity value) >> 1" or "(third chromaticity value + fourth chromaticity value + 1) >> 1". Since the method for obtaining the average value has already been described, its description will be omitted.

[0154] An encoder or decoder can perform the step of obtaining elements of a function indicating the correspondence between luminance and chrominance based on the maximum average luminance, minimum average luminance, maximum average chrominance, and minimum average chrominance. The function indicating the correspondence between luminance and chrominance can be... Figure 10 Equation (1) or equation (4).

[0155] In equation (1), x can be obtained based on the reconstructed luminance component of the current block. To make the number of samples of the luminance component included in the current block suitable for the number of samples of the chrominance component included in the current block, the encoder or decoder can downsample the reconstructed luminance component of the current block. The encoder or decoder can predict the chrominance component (Y) by applying the downsampled luminance component (X) of the current block to equation (1).

[0156] In equation (4), recL can represent the downsampled luminance component of the current block. S can be a predetermined constant, or it can be a value obtained based on the maximum luminance average, minimum luminance average, maximum chromaticity average, and minimum chromaticity average. C The predicted chromaticity components can be represented. In equation (4), α and β can be derived from equations (2) and (3).

[0157] In equations (2) and (3), y B It can indicate the maximum average chromaticity. A It can indicate the minimum average chromaticity. x B It can indicate the average value of maximum brightness. A It can indicate the minimum average brightness.

[0158] To obtain elements of a function indicating the correspondence between luminance and chrominance, the encoder or decoder may perform a step of obtaining a first difference value as the difference between the maximum average luminance and the minimum average luminance. Furthermore, the encoder or decoder may perform a step of obtaining a second difference value as the difference between the maximum average chrominance and the minimum average chrominance.

[0159] The encoder or decoder may perform the step of obtaining a first element (α) related to the ratio of the second difference value to the first difference value. For example, the encoder or decoder may obtain the first element (α) based on equation (2).

[0160] The encoder or decoder can perform the step of obtaining the second element (β) based on the first element, the minimum luminance average value, and the minimum chrominance average value. For example, the encoder or decoder can obtain the second element (β) based on equation (3).

[0161] The encoder or decoder may perform the step of predicting the chromaticity value of the current block based on the reconstructed luminance value, the first element (α), and the second element (β). The encoder or decoder may predict the chromaticity value of the current block based on equation (1) or equation (4). The luminance value may be a sample value included in the luminance component of the current block, and the chromaticity value may be a sample value included in the predicted chromaticity component of the current block.

[0162] An encoder or a decoder may perform a step of predicting a chrominance value of a current block by applying a reconstructed luma value of the current block to a function. As described above, the encoder or the decoder may reconstruct a luma component of the current block. In addition, the encoder or the decoder may make the number of samples of the luma component included in the current block suitable for the number of samples of the chrominance component. That is, the encoder or the decoder may perform a step of downsampling the reconstructed luma component of the current block. Additionally, the encoder or the decoder may predict the chrominance component by applying the downsampled luma component to Equation (1) or Equation (4). More specifically, the encoder or the decoder may perform a step of predicting a chrominance value of the current block based on the following equation. The following equation describes Equation (4) in detail.

[0163] (Chrominance value of the current block) = ((((Luminance value included in the downsampled luma component of the current block) * α) >> S) + β);

[0164] As described above, S may be a predetermined constant. However, it is not limited thereto, and S may be a value obtained based on the maximum luma average value, the minimum luma average value, the maximum chrominance average value, or the minimum chrominance average value.

[0165] However, it is not limited thereto, and the encoder or the decoder may perform a step of predicting a chrominance value based on the following equation obtained by modifying Equation (4).

[0166] (Chrominance value of the current block) = Clip1C((((Luminance value included in the downsampled luma component of the current block) * α) >> S) + β)

[0167] Clip1C may be given as follows.

[0168] Clip1C(x) = Clip3(0, (1 << BitDepthC) - 1, x)

[0169] Clip3(x, y, z) =

[0170] x; when z < x

[0171] y; when z > y

[0172] z; otherwise

[0173] BitDepthC represents the bit depth of the chrominance component.

[0174] Figure 12 The figure illustrates a method of determining a linear model of CCLM according to an embodiment of the present disclosure.

[0175] A method of determining a linear model of CCLM is described. It may be implemented using the maximum value of luminance ( Figure 12 's C) and the minimum value ( Figure 12Linear model 1 of A) and using the second highest maximum value ( Figure 12 D) and the second lowest minimum ( Figure 12 Linear Model 2 (B) can be replaced by Linear Model 1. As mentioned above, linear models can be implemented and used based on the maximum / minimum values ​​of the characteristic ranking. Figure 11 and Figure 12 In this model, chroma samples can be used instead of luminance samples to implement a linear model.

[0176] In addition, Figure 8 In the luminance block to which MRL has been applied, when CCLM is applied, the corresponding luminance sample is used as the luminance sample required by the linear model. In the case of chrominance, when the MRL index is 3, the chrominance sample corresponding to MRL index 0, the chrominance sample corresponding to MRL index 3, and the chrominance sample corresponding to the average / weighted average / fixed MRL index to which the average value / weighted value is applied can be used. In the chrominance prediction method, the prediction mode information of the luminance at a specific location can be obtained using the luminance prediction method. In this case, the MRL index is also stored, just like the luminance mode. Furthermore, chrominance can be operated in the same way as the prediction of luminance using MRL based on multiple reference samples. The MRL index for chrominance can be sent separately as a signal. Alternatively, the MRL index for luminance can be used.

[0177] It is possible Figures 10 to 12 As shown, based on the reconstructed luminance and chrominance, the mapping relationship between luminance and chrominance for chrominance prediction is configured through equations (1) to (4) and various linear regression methods. In this case, although the mapping relationship between the right adjacent luminance and chrominance samples of the reconstruction is usually used, other adjacent luminance and chrominance samples can be considered to derive the mapping relationship between luminance and chrominance for prediction depending on the MRL application. This method can be applied by increasing the number of samples by adding adjacent samples added by MRL to the mapping between the reconstructed exactly adjacent luminance and chrominance samples. Alternatively, the individual linear mapping relationships can be configured by applying separate mapping equations (e.g., equations (1) to (4)) between the adjacent samples added by MRL and the exactly adjacent samples to the exactly adjacent reconstructed luminance-chrominance samples (e.g., MRL0) and the reconstructed luminance-chrominance samples added by MRL (e.g., MRL3), and the comprehensive parameters can be derived by applying the average or weighted average of the parameters (such as the derived α / β, etc.), which can be used as the chrominance prediction equation.

[0178] Figure 13 The illustration shows an example of a downsampling filter for the luminance samples required when performing chroma intra-frame prediction by applying a cross-component linear model (CCLM).

[0179] When an encoder or decoder performs prediction by applying CCLM to the chroma components of a 4×4 prediction block, in the case of an image format of 4:2:0, the luminance samples should be downsampled in the form of 2x2 chroma blocks. To detect the relationship between the luminance and chroma of the current block's neighboring blocks, the encoder or decoder can downsample the luminance components of the neighboring blocks, and the processing of this is described below.

[0180] The encoder or decoder can perform the steps of obtaining first information (availL) indicating whether the left block adjacent to the current block is available and second information (availT) indicating whether the upper block adjacent to the current block is available.

[0181] When the information indicating whether the left block adjacent to the current block is available (availL) indicates that the left block is available, the downsampled adjacent left luminance component pLeftDsY[y] is derived as follows (where y = 0..nTbH-1).

[0182] pLeftDsY[y]=(pY[-1][2*y]+pY[-1][2*y+1]+2*pY[-2][2*y]+2*pY[-2][2*y+1]+pY[-3][2*y]+pY[-3][2*y+1]+4)>>3 Equation (1)

[0183] That is, when the first information indicates that the left block is available, the encoder or decoder can perform the step of obtaining the downsampled luminance component based on the left block.

[0184] When the information indicating whether the upper block adjacent to the current block is available (availT) indicates that the upper block is available, the downsampled adjacent upper luminance component pTopDsY[x] can be derived as follows (where x = 0..nTbH-1).

[0185] pTopDsY[x]=(pY[2*x-1][-2]+pY[2*x-1][-1]+2*pY[2*x][-2]+2*pY[2*x][-1]+pY[2*x+1][-2]+pY[2*x+1][-1]+4)>>3 Equation (2)

[0186] That is, when the second information indicates that the upper block is available, the encoder or decoder can perform the step of obtaining the downsampled luminance component based on the upper block.

[0187] If availTL is true, then pTopDsY[0] is derived as follows:

[0188] pTopDsY[0]=(pY[-1][-2]+pY[-1][-1]+2*pY[0][-2]+2*pY[0][-1]+pY[1][-2]+pY[1][-1]+4)>>3 Equation (3)

[0189] Furthermore, when the first information indicates that the left block is available and the second information indicates that the upper block is available, the encoder or decoder can obtain the downsampled luminance component based on both the left block and the upper block.

[0190] Otherwise, derive pTopDsY[0] as follows:

[0191] pTopDsY[0]=(pY[0][-2]+pY[0][-1]+1)>>1 Equation (4)

[0192] Figure 13 Figure b illustrates that when applying MRL, different downsampling filters of CCLM should be applied to the MRL index. When the MRL index is 3, equation (5) is derived by shifting the X-axis by -1 in equation (1).

[0193] pLeftDsY[y]=(pY[-2][2*y]+pY[-2][2*y+1]+2*pY[-3][2*y]+2*pY[-3][2*y+1]+pY[-4][2*y]+pY[-4][2*y+1]+4)>>3 Equation (5)

[0194] Equation (6) is derived by shifting the Y-axis by -1 in equation (2).

[0195] pTopDsY[x]=(pY[2*x-1][-4]+pY[2*x-1][-3]+2*pY[2*x][-4]+2*pY[2*x][-3]+pY[2*x+1][-4]+pY[2*x+1][-3]+4)>>3 Equation (6)

[0196] Equations (7) and (8) are derived by applying the same method to equations (3) and (4).

[0197] pTopDsY[0]=(pY[-1][-4]+pY[-1][-3]+2*pY[0][-4]+2*pY[0][-3]+pY[1][-4]+pY[1][-3]+4)>>3 Equation (7)

[0198] pTopDsY[0]=(pY[0][-4]+pY[0][-3]+1)>> 1 Equation (8)

[0199] Figure 14The illustration shows an example of applying a CCLM filter when using MRL.

[0200] In order to apply, such as Figure 14 The CCLM filter shown in a can be transformed by the encoder and decoder into equations (9), (10), (11) and (12) and used as a downsampling filter.

[0201] pLeftDsY[y]=(pY[-1][2*y]+pY[-1][2*y+1]+2*pY[-3][2*y]+2*pY[-3][2*y+1]+pY[-4][2*y]+pY[-4][2*y+1]+4)>>3 Equation (9)

[0202] pTopDsY[x]=(pY[2*x-1][-4]+pY[2*x-1][-1]+2*pY[2*x][-4]+2*pY[2*x][-1]+pY[2*x+1][-4]+pY[2*x+1][-1]+4)>>3 Equation (10)

[0203] pTopDsY[0]=(pY[-1][-4]+pY[-1][-1]+2*pY[0][-4]+2*pY[0][-1]+pY[1][-4]+pY[1][-1]+4)>>3 Equation (11)

[0204] pTopDsY[0]=(pY[0][-4]+pY[0][-1]+1)>>1 Equation (12)

[0205] In order to apply, such as Figure 14 The CCLM filter shown in b can be used by the encoder and decoder to change equations (1), (2), (3) and (4) to equations (13), (14), (15) and (16) and to use a downsampling filter.

[0206] pLeftDsY[y]=(pY[-1][2*y]+pY[-1][2*y+1]+2*pY[-2][2*y]+2*pY[-2][2*y+1]+pY[-4][2*y]+pY[-4][2*y+1]+4)>>3 Equation (13)

[0207] Equation (14) is the same as equation (10).

[0208] Equation (15) is the same as equation (11).

[0209] Equation (16) is the same as equation (12).

[0210] Figure 15 The illustration shows an example of applying a CCLM filter when using MRL.

[0211] In order to apply, such as Figure 15 The CCLM filter shown in a can be transformed into equations (1), (2), (3) and (4) by the encoder and decoder, and a downsampling filter can be used.

[0212] pLeftDsY[y]=(pY[-1][2*y]+pY[-1][2*y+1]+pY[-3][2*y]+pY[-3][2*y+1]+2*pY[-4][2*y]+2*pY[-4][2*y+1]+4)>>3 Equation (17)

[0213] Equation (18) is the same as equation (10).

[0214] Equation (19) is the same as equation (11).

[0215] Equation (20) is the same as equation (12).

[0216] In order to apply, such as Figure 15 The CCLM filter shown in b can be transformed into equations (21) to (24) by the encoder and decoder, and a downsampling filter can be used.

[0217] pLeftDsY[y]=(pY[-1][2*y]+pY[-1][2*y+1]+pY[-2][2*y]+pY[-2][2*y+1]+2*pY[-4][2*y]+2*pY[-4][2*y+1]+4)>>3 Equation (21)

[0218] Equation (22) is the same as equation (10).

[0219] Equation (23) is the same as equation (11).

[0220] Equation (24) is the same as equation (12).

[0221] These methods correspond to the case where the MRL is one of 0, 1, and 3 (i.e., MRL index = 3), but a regular filter can be applied when using other MRL indices (such as 0 and 1). When the MRL index is 2, the regular filter coefficients can be used and applied in the regular method, using (-1, x-axis sample) and (-3, x-axis sample) on the y-axis in the equation applied to the upper side. Furthermore, on the left side, the weight values ​​1, 2, and 1 corresponding to -1, -2, and -3 on the x-axis can be changed to 1, 1, and 2 and applied.

[0222] Back Figure 10 The encoder or decoder uses the reconstructed luminance and chrominance of the adjacent blocks to find the minimum and maximum values ​​of luminance, and uses these two values ​​to derive equation (1) from equations (2) and (3). Finally, the chrominance is predicted using the reconstructed luminance values ​​of the current block via equation (4).

[0223] When the image format is 4:2:0, a downsampling filter is applied to adjust the number of luminance samples to match the number of chrominance samples. When applying... Figure 10 Equation (1), in Figure 13 When all upper and left reference samples in 'a' are valid, the minimum luminance value (MinLuma), maximum luminance value (MaxLuma), chromaticity value for minimum luminance (ChromaForMinLuma), and chromaticity value for maximum luminance (ChromaForMaxLuma) should be derived. (See reference...) Figure 11 The encoder or decoder can derive the maximum luminance average, minimum luminance average, maximum chromaticity average, and minimum chromaticity average, and apply equation (1). The minimum luminance value (MinLuma) can correspond to the minimum luminance average, the maximum luminance value (MaxLuma) can correspond to the maximum luminance average, the chromaticity value for the minimum luminance (ChromaForMinLuma) can correspond to the minimum chromaticity average, and the chromaticity value for the maximum luminance (ChromaForMaxLuma) can correspond to the maximum chromaticity average.

[0224] In this scenario, the value obtained first on the top side serves as a reference value when identifying the left sample and is compared with the left sample value. After identifying all samples on the top and left sides, the values ​​of MinLuma, MaxLuma, ChromaForMinLuma, and ChromaForMaxLuma are determined. The pseudocode used to derive MinLuma, MaxLuma, ChromaForMinLuma, and ChromaForMaxLuma is described below.

[0225] 1. Derived variables MinLuma, MaxLuma, ChromaForMinLuma, and ChromaForMaxLuma as follows:

[0226] Set the variable MinLuma to equal 1 << (BitDepthY) + 1 and the variable MaxLuma to equal -1, changing 1 << (BitDepthY) + 1 to (1 < <BitDepthY)+1。

[0227] – If availT is true, then derive the variables MinLuma, MaxLuma, ChromaForMinLuma, and ChromaForMaxLuma as follows, where x = 0..nS–1:

[0228] – If (MinLuma > pTopDsY[x*xS]), then the following applies:

[0229] MinLuma = pTopDsY[x*xS]

[0230] ChromaForMinLuma = p[x*xS][-1]

[0231] – If (MaxC < pTopDsY[x*xS]), then the following applies:

[0232] MaxLuma = pTopDsY[x*xS]

[0233] ChromaForMaxLuma = p[x*xS][-1]

[0234] – If availL is true, then derive the variables MinLuma, MaxLuma, ChromaForMinLuma, and ChromaForMaxLuma as follows, where y = 0..nS-1:

[0235] – If (MinLuma > pTopDsY[y*yS]), then the following applies:

[0236] Change (MinLuma > pTopDsY[y*yS]) to (MinLuma > pLeftDsY[y*yS]).

[0237] Change MinLuma = pTopDsY[y*yS]:MinLuma = pTopDsY[y*yS] to MinLuma = pLeftDsY[y*yS].

[0238] ChromaForMinLuma = p[-1][y*yS]

[0239] – If (MaxLuma < pTopDsY[y*yS]), then the following applies:

[0240] Change (MaxLuma < pTopDsY[y*yS]) to (MaxLuma < pLeftDsY[y*yS]).

[0241] Change MaxLuma=pTopDsY[y*yS]:MaxLuma=pTopDsY[y*yS] to MaxLuma=pLeftDsY[y*yS].

[0242] ChromaForMaxLuma=p[-1][y*yS]

[0243] Figure 16 The illustration shows a cross-component linear model (CCLM) method according to an embodiment of the present disclosure.

[0244] For reference Figure 10 The minimum and maximum values ​​of the luminance are obtained using the reconstructed luminance and chrominance adjacent to the current prediction block, and equation (1) is derived from equations (2) and (3) using these two values. Finally, the chrominance is predicted using the reconstructed luminance value of the current block via equation (4).

[0245] refer to Figure 16 When the image format is 4:2:0, the number of luminance samples can be greater than the number of chrominance samples. Therefore, the encoder or decoder can adjust the number of luminance samples to match the number of chrominance samples by applying a downsampling filter to the luminance samples. (Reference) Figure 10 When both the upper and left reference samples are valid, the minimum luminance value (MinLuma), the maximum luminance value (MaxLuma), the chromaticity value for the minimum luminance (ChromaForMinLuma), and the chromaticity value for the maximum luminance (ChromaForMaxLuma) should be derived. (See reference...) Figure 11 The encoder or decoder can derive the maximum luminance average, minimum luminance average, maximum chromaticity average, and minimum chromaticity average, and apply equation (1). The minimum luminance value (MinLuma) can correspond to the minimum luminance average, the maximum luminance value (MaxLuma) can correspond to the maximum luminance average, the chromaticity value for the minimum luminance (ChromaForMinLuma) can correspond to the minimum chromaticity average, and the chromaticity value for the maximum luminance (ChromaForMaxLuma) can correspond to the maximum chromaticity average.

[0246] In this scenario, the value obtained first on the top side serves as a reference value when identifying the left sample and is compared with it. After identifying all samples on the top and left sides, the values ​​of MinLuma, MaxLuma, ChromaForMinLuma, and ChromaForMaxLuma are determined.

[0247] The encoder or decoder can obtain MinLuma, MaxLuma, ChromaForMinLuma, and ChromaForMaxLuma through the following pseudocode.

[0248] Derive the variables MinLuma, MaxLuma, ChromaForMinLuma, and ChromaForMaxLuma as follows:

[0249] Set the variable MinLuma to be equal to 1<<(BitDepthY)+1, and set the variable MaxLuma to be equal to -1, changing 1<<(BitDepthY)+1 to (1<<BitDepthY)+1.

[0250] If availT is true, then the variables MinLuma, MaxLuma, ChromaForMinLuma, and ChromaForMaxLuma can be derived as follows, where x = 0..nS-1:

[0251] If (MinLuma > pTopDsY[x*xS]), then the following applies:

[0252] MinLuma = pTopDsY[x*xS]

[0253] ChromaForMinLuma = p[x*xS][-1]

[0254] If (MaxC < pTopDsY[x*xS]), then the following applies:

[0255] MaxLuma = pTopDsY[x*xS]

[0256] ChromaForMaxLuma = p[x*xS][-1]

[0257] If availL is true, then the variables MinLuma, MaxLuma, ChromaForMinLuma, and ChromaForMaxLuma can be derived as follows, where y = 0..nS-1:

[0258] If (MinLuma > pTopDsY[y*yS]), then the following applies:

[0259] MinLuma = pLeftDsY[y*yS]

[0260] ChromaForMinLuma = p[-1][y*yS]

[0261] If (MaxLuma < pTopDsY[y * yS]), then the following applies:

[0262] MaxLuma = pLeftDsY[y * yS]

[0263] ChromaForMaxLuma = p[-1][y * yS]

[0264] Figure 17 FIG. illustrates a method for determining the number of reference samples in the CCLM method according to an embodiment of the present disclosure.

[0265] Reference Figure 17 A, in the case of a block that is not a square block in the conventional CCLM, the encoder or decoder uses the samples corresponding to the number of the short side as the reference samples for the long side. Instead of this method, the encoder or decoder can use all the samples of the long side as the reference samples for the long side. The encoder or decoder can use all the downsampled reference samples and apply Figure 17 the method of

[0266] Reference Figure 17 B, when the chroma block size is W × H, the encoder or decoder can use only some samples, that is, some of the 2W samples (left figure) and 2H samples (right figure), instead of all, as the CCLM reference samples.

[0267] For example, the encoder or decoder can obtain the downsampled luminance component based on the pixels (samples) included in at least one of the left block or the upper block adjacent to the current block.

[0268] In addition, referring to Figure 17 , the encoder or decoder can obtain the downsampled luminance component based on the samples included in at least one of the rightmost sample 1120 of the left block or the lowermost sample 1110 of the upper block. The encoder or decoder can obtain the downsampled luminance component based on some of the rightmost sample 1120 of the left block or the lowermost sample 1110 of the upper block.

[0269] The number of some samples can be 1 / 2 or 1 / 4 of the total number of samples 1110 and 1120. In this case, when half of the samples are configured, the samples placed at a fixed interval can be used. Alternatively, it is applied only to blocks having a predetermined size or larger. That is, the encoder or decoder can obtain the downsampled luminance component based on the samples selected at a fixed interval from the rightmost sample of the left block or the lowermost sample of the upper block.

[0270] The length of the side of the current block to which CCLM is applied can be one of 16, 32, 62, and 128. For example, CCLM can be applied when using samples contained in the upper block and the width of the current block is 32 samples or more. CCLM can also be applied when using samples contained in the left block and the height of the current block is 32 or more.

[0271] exist Figure 17 In C, when the chroma block size is W×H, the encoder or decoder can use all 2W+H samples as CCLM reference samples to calculate α and β as defined in CCLM. Alternatively, the encoder or decoder can calculate α and β using all samples in the short side and only the samples in the long side that correspond to the samples in the short side.

[0272] In this scenario, the encoder or decoder can use samples only at fixed intervals.

[0273] Figure 18 The illustration shows prediction mode signaling for intra-chroma prediction according to an embodiment of the present disclosure.

[0274] When the variable `sps_cclm_enabled_flag` equals 1, CCLM-related modes are enabled. The number of modes can be up to 3. Figure 18 Tables 1 and 2 show the intra_chroma_pred_mode that indicates the intra-chroma prediction mode.

[0275] [Table 1] can show the chroma modes based on index information 0 to 7. Similar to the information in [Table 2], the information in [Table 1] can be represented in binary form before encoding the input, in which case the amount of information can vary depending on each usage mode. For example, frequently used modes have high priority and less information.

[0276] In [Table 2], a value of 4 corresponds to CCLM, a value of 5 corresponds to MDLM_L mode, and a value of 6 corresponds to MDLM_T. When the value of intra_chroma_pred_mode is 5 or 6, the coding efficiency may increase depending on the order if the encoder or decoder uses context modeling. Therefore, the encoder or decoder prioritizes MDLM_T over MDLM_L in [Table 2]. Alternatively, priority can be determined based on the chroma size of the current predicted block. MDLM_T may have higher priority than MDLM_L when W (width) > H (height), while MDLM_L may have higher priority than MDLM_T when W < H. MDLM_T can be a CCLM mode that only uses upsampled samples, such as... Figure 17As shown in the left figure (b), MDLM_L can be one of the CCLM modes that only uses the left sample, such as Figure 17 As shown in Figure B on the right.

[0277] Some aspects of this disclosure have already been described in the decoder section, but the same operations can be performed in the encoder. For example, an encoding apparatus including a processor and memory can perform the following steps: downsampling the reconstructed luminance components of blocks adjacent to the current block; obtaining the maximum luminance value among the downsampled luminance components based on a first index; obtaining the second highest luminance value among the downsampled luminance components based on a second index; obtaining a maximum luminance average value based on the average of the maximum luminance value and the second highest luminance value; obtaining the second lowest luminance value among the downsampled luminance components based on a third index; obtaining the minimum luminance value among the downsampled luminance components based on a fourth index; obtaining a minimum luminance average value based on the second lowest luminance value and the minimum luminance value; and obtaining the chrominance component corresponding to the downsampled luminance components based on the first index. The process involves: obtaining a first chromaticity value; obtaining a second chromaticity value within the chromaticity component based on the second index; obtaining a maximum chromaticity average value based on the average of the first and second chromaticity values; obtaining a third chromaticity value within the chromaticity component based on the third index; obtaining a fourth chromaticity value within the chromaticity component based on the fourth index; obtaining a minimum chromaticity average value based on the average of the third and fourth chromaticity values; obtaining elements of a function indicating the correspondence between luminance and chromaticity based on the maximum luminance average value, the minimum luminance average value, the maximum chromaticity average value, and the minimum chromaticity average value; and predicting the chromaticity value of the current block by applying the reconstructed luminance value of the current block to the function.

[0278] The above-described embodiments of the present invention can be implemented by various means. For example, the embodiments of the present invention can be implemented by hardware, firmware, software, or a combination thereof.

[0279] In the case of hardware implementation, the method according to the embodiments of the present invention can be implemented by one or more of the following: application-specific integrated circuit (ASIC), digital signal processor (DSP), digital signal processing device (DSPD), programmable logic device (PLD), field-programmable gate array (FPGA), processor, controller, microcontroller, microprocessor, etc.

[0280] When implemented via firmware or software, the method according to embodiments of the invention can be implemented in the form of modules, processes, or functions that perform the above-described functions or operations. Software code can be stored in memory and driven by a processor. The memory can be located inside or outside the processor and can exchange data with the processor in various known ways.

[0281] Some embodiments may also be implemented in the form of a recording medium including computer-executable instructions, such as a program module executed by a computer. A computer-readable medium can be any available medium accessible to a computer and can include all volatile, non-volatile, removable, and non-removable media. Additionally, a computer-readable medium can include computer storage media and communication media. Computer storage media includes all volatile, non-volatile, removable, and non-removable media implemented in any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data. Typically, communication media includes computer-readable instructions, other data modulated data signals (such as data structures or program modules), or other transmission mechanisms, and includes any information transmission medium.

[0282] The above description of the present invention is for illustrative purposes only, and it will be understood that those skilled in the art to which this invention pertains can make changes to the invention without altering its technical concept or essential characteristics, and that the invention can be readily modified in other specific forms. Therefore, the above embodiments are illustrative and not limiting in any way. For example, each component described as a single entity can be distributed and implemented, and similarly, components described as distributed can also be implemented in an associated manner.

[0283] The scope of this invention is defined by the appended claims rather than the foregoing detailed description, and all changes or modifications derived from the meaning and scope of the appended claims and their equivalents shall be construed as being included within the scope of this invention.

Claims

1. A method for processing video signals by a device, the method comprising: The luminance samples of at least one luminance block adjacent to the current luminance block are downsampled, wherein the downsampled luminance samples are obtained by applying a filter to a plurality of samples included in at least three vertical sample lines of the left luminance block adjacent to the current luminance block or in at least two horizontal sample lines of the upper luminance block adjacent to the current luminance block. The maximum brightness average value is obtained based on the maximum brightness value and the second highest brightness value of the downsampled brightness sample, wherein the maximum brightness value is obtained based on a first index and the second highest brightness value is obtained based on a second index. The minimum brightness average value is obtained based on the minimum brightness value and the second lowest brightness value of the downsampled brightness sample, wherein the second lowest brightness value is obtained based on a third index and the minimum brightness value is obtained based on a fourth index. The maximum average chromaticity value is obtained based on the first chromaticity value and the second chromaticity value of the chromaticity sample corresponding to the downsampled luminance sample, wherein the first chromaticity value is obtained based on the first index and the second chromaticity value is obtained based on the second index. The minimum average chromaticity value is obtained based on the third and fourth chromaticity values ​​of the chromaticity sample, wherein the third chromaticity value is obtained based on the third index, and the fourth chromaticity value is obtained based on the fourth index; Based on the maximum average luminance, the minimum average luminance, the maximum average chromaticity, and the minimum average chromaticity, obtain one or more elements of the equation used to obtain predicted chromaticity samples based on luminance samples; and The predicted chromaticity sample of the current chromaticity block is obtained by applying the downsampled luminance sample of the current luminance block and one or more elements to the equation.

2. The method according to claim 1, wherein, Obtaining one or more elements of the equation includes: The first difference value is obtained based on the difference between the average maximum brightness and the average minimum brightness. The second difference value is obtained based on the difference between the maximum average chromaticity value and the minimum average chromaticity value; Obtain the first element α related to the ratio of the second difference value to the first difference value; and The second element β is obtained based on the first element, the minimum average brightness, and the minimum average chromaticity. The process of obtaining the predicted chromaticity sample of the current chromaticity block includes obtaining the predicted chromaticity sample of the current chromaticity block based on the downsampled luminance sample of the current luminance block, the first element α, and the second element β.

3. The method according to claim 2 further includes downsampling the brightness sample of the current brightness block. in, Obtaining the predicted chromaticity sample of the current chromaticity block includes: Based on the following equation, the predicted chromaticity sample of the current chromaticity block is obtained: (predicted chromaticity sample of the current chromaticity block) = (((downsampled luminance sample of the current luminance block) * α) >> S) + β, where S is a constant.

4. A method for encoding a video signal by a device, the method comprising: The luminance samples of at least one luminance block adjacent to the current luminance block are downsampled, wherein the downsampled luminance samples are obtained by applying a filter to a plurality of samples included in at least three vertical sample lines of the left luminance block adjacent to the current luminance block or in at least two horizontal sample lines of the upper luminance block adjacent to the current luminance block. The maximum brightness average value is obtained based on the maximum brightness value and the second highest brightness value of the downsampled brightness sample, wherein the maximum brightness value is obtained based on a first index and the second highest brightness value is obtained based on a second index. The minimum brightness average value is obtained based on the minimum brightness value and the second lowest brightness value of the downsampled brightness sample, wherein the second lowest brightness value is obtained based on a third index and the minimum brightness value is obtained based on a fourth index. The maximum average chromaticity value is obtained based on the first chromaticity value and the second chromaticity value of the chromaticity sample corresponding to the downsampled luminance sample, wherein the first chromaticity value is obtained based on the first index and the second chromaticity value is obtained based on the second index. The minimum average chromaticity value is obtained based on the third and fourth chromaticity values ​​of the chromaticity sample, wherein the third chromaticity value is obtained based on the third index, and the fourth chromaticity value is obtained based on the fourth index; Based on the maximum average luminance, the minimum average luminance, the maximum average chromaticity, and the minimum average chromaticity, obtain one or more elements of the equation used to obtain predicted chromaticity samples based on luminance samples; and The predicted chromaticity sample of the current chromaticity block is obtained by applying the downsampled luminance sample of the current luminance block and one or more elements to the equation.

5. The method according to claim 4, wherein, Obtaining one or more elements of the equation includes: The first difference value is obtained based on the difference between the average maximum brightness and the average minimum brightness. The second difference value is obtained based on the difference between the maximum average chromaticity value and the minimum average chromaticity value; Obtain the first element α related to the ratio of the second difference value to the first difference value; and The second element β is obtained based on the first element, the minimum average brightness, and the minimum average chromaticity. The process of obtaining the predicted chromaticity sample of the current chromaticity block includes obtaining the predicted chromaticity sample of the current chromaticity block based on the downsampled luminance sample of the current luminance block, the first element α, and the second element β.

6. The method according to claim 5 further includes downsampling the brightness sample of the current brightness block. in, Obtaining the predicted chromaticity sample of the current chromaticity block includes: Based on the following equation, the predicted chromaticity sample of the current chromaticity block is obtained: (predicted chromaticity sample of the current chromaticity block) = (((downsampled luminance sample of the current luminance block) * α) >> S) + β, where S is a constant.

7. A method for storing a bitstream, wherein the bitstream is decoded by a decoding method. in, The decoding method includes: The luminance samples of at least one luminance block adjacent to the current luminance block are downsampled, wherein the downsampled luminance samples are obtained by applying a filter to a plurality of samples included in at least three vertical sample lines of the left luminance block adjacent to the current luminance block or in at least two horizontal sample lines of the upper luminance block adjacent to the current luminance block. The average maximum brightness is obtained based on the maximum and second-highest brightness values ​​of the downsampled brightness samples, wherein the maximum brightness value is obtained based on a first index and the second-highest brightness value is obtained based on a second index. The minimum brightness average value is obtained based on the minimum brightness value and the second lowest brightness value of the downsampled brightness sample, wherein the second lowest brightness value is obtained based on a third index and the minimum brightness value is obtained based on a fourth index. The maximum average chromaticity value is obtained based on the first chromaticity value and the second chromaticity value of the chromaticity sample corresponding to the downsampled luminance sample, wherein the first chromaticity value is obtained based on the first index and the second chromaticity value is obtained based on the second index. The minimum average chromaticity value is obtained based on the third and fourth chromaticity values ​​of the chromaticity sample, wherein the third chromaticity value is obtained based on the third index, and the fourth chromaticity value is obtained based on the fourth index; Based on the maximum average luminance, the minimum average luminance, the maximum average chromaticity, and the minimum average chromaticity, obtain one or more elements of the equation used to obtain predicted chromaticity samples based on luminance samples; and The predicted chromaticity sample of the current chromaticity block is obtained by applying the downsampled luminance sample of the current luminance block and one or more elements to the equation.

Images