Method and device for predicting chrominance signal on basis of multi-model that is based on correlation between components and uses adaptive filtering

Abstract

An image encoding / decoding method and device, and a recording medium, of the present disclosure, may comprise: a prediction unit determination step of determining a prediction unit for predicting a chrominance component; a prediction technology determination step of determining a prediction technology to be applied to the prediction unit; a prediction mode determination step of determining, on the basis of the prediction technology, a prediction mode for the prediction of the chrominance component; and a prediction step of predicting the chrominance component on the basis of the prediction mode.

Description

Method and device for predicting color difference signals based on a multi-model based on inter-component correlation using adaptive filtering

[0001] The present disclosure relates to a method, device, and recording medium for encoding / decoding a video signal.

[0002] The demand for high-resolution, high-quality images is growing across a wide range of applications. As image data becomes higher in resolution and quality, the relative amount of data increases compared to conventional image data. Therefore, transmitting image data using existing media such as wired or wireless broadband lines or storing it using existing storage media incurs increased transmission and storage costs. To address these issues arising from the increasing resolution and quality of image data, high-efficiency image compression technologies can be utilized.

[0003] The present disclosure aims to improve encoding / decoding efficiency in predicting a chrominance block using correlation between components.

[0004] The present disclosure seeks to improve the accuracy of component-by-component prediction.

[0005] The video encoding / decoding method, device, and recording medium of the present disclosure may include a prediction unit determination step of determining a prediction unit for performing prediction of a chrominance component; a prediction technique determination step of determining a prediction technique to be applied to the prediction unit; a prediction mode determination step of determining a prediction mode for predicting the chrominance component based on the prediction technique; and a prediction performance step of performing prediction of the chrominance component based on the prediction mode.

[0006] In the video encoding / decoding method, device, and recording medium of the present disclosure, the prediction unit may be a chrominance block.

[0007] In the video encoding / decoding method, device, and recording medium of the present disclosure, the prediction technology may be an intra-screen prediction technology, an inter-screen prediction technology, an IBC (Intra Block Copy) technology, a palette mode technology, or a technology that mixes intra-screen prediction and inter-screen prediction.

[0008] In the video encoding / decoding method, device, and recording medium of the present disclosure, in response to the prediction technique being determined to be a technique other than an in-screen prediction technique, the prediction mode may be determined based on a Skip flag.

[0009] In the video encoding / decoding method, device, and recording medium of the present disclosure, in response to the prediction technique being determined as a technique that mixes intra-screen prediction and inter-screen prediction, the prediction mode can be determined using information of some or all pixels of the template of the prediction unit.

[0010] In the video encoding / decoding method, device, and recording medium of the present disclosure, the shape of the prediction unit may be a shape determined through geometric division.

[0011] In the video encoding / decoding method, device, and recording medium of the present disclosure, in response to the prediction technique being determined as an intra-screen prediction technique, the prediction mode may be any one of CfL (Chroma from luma), CCLM (Cross component linear model), MHCCP (Multi-hypothesis cross component prediction), and CCCM (Convolutional cross component model).

[0012] In the video encoding / decoding method, device, and recording medium of the present disclosure, prediction of the chrominance component can be performed based on a luminance block at a position corresponding to the prediction unit and pre-restored surrounding luminance samples of the luminance block.

[0013] In the video encoding / decoding method, device, and recording medium of the present disclosure, prediction of the chrominance component is performed based on a linear model between the chrominance component and the luminance component, and the linear model may include a scale parameter and an offset parameter.

[0014] In the video encoding / decoding method, device, and recording medium of the present disclosure, the linear model can be derived based on a chrominance template of the prediction unit and a luminance template at a position corresponding to the chrominance template.

[0015] In the video encoding / decoding method, device, and recording medium of the present disclosure, the linear model can be derived based on representative values ​​of all or part of the samples of the chrominance template and representative values ​​of all or part of the samples of the luminance template.

[0016] The present disclosure proposes a device and method for modeling correlation between components using information of restored luminance blocks at corresponding positions in the process of predicting a chrominance block, and predicting a chrominance block using one or more linear models, thereby improving encoding / decoding efficiency in predicting a chrominance block using correlation between components.

[0017] The present disclosure enables effective prediction of color difference signals.

[0018] FIG. 1 is a block diagram illustrating an embodiment of the present disclosure, which determines prediction and inverse transformation units of a decoder, performs prediction and inverse transformation for the determined units, modes, and methods, and ultimately performs restoration.

[0019] Figure 2 is a diagram showing a template of a chrominance component for pre-restored samples around a chrominance block currently being encoded / decoded.

[0020] Figure 3 is a drawing showing a template for the restoration samples around the luminance block at the corresponding position based on the template of the current chrominance block to be encoded / decoded.

[0021] Figure 4 illustrates a type of filter for downsampling according to the first embodiment.

[0022] Figure 5 illustrates a type of filter for downsampling according to the second embodiment.

[0023] Figure 6 illustrates a type of filter for downsampling according to the third embodiment.

[0024] Figure 7 illustrates the available filter shapes and the index to the filter coefficients for each filter shape.

[0025] Figure 8 illustrates a linear model obtained based on a luminance template and a chrominance template.

[0026] Figure 9 illustrates a step of obtaining a chrominance prediction signal based on a luminance template and a chrominance template.

[0027] Figure 10 is a diagram showing the locations of samples used to derive a linear model.

[0028] Figure 11 illustrates different embodiments of a step of obtaining a chrominance prediction signal by searching for a corresponding position luminance block.

[0029] Figure 12 illustrates different embodiments of obtaining a chrominance prediction signal based on a chrominance template, a luminance template, and an initial chrominance prediction signal.

[0030] Figure 13 illustrates a first embodiment of deriving a linear model according to a range of luminance sample values.

[0031] Figure 14 illustrates a second embodiment of deriving a linear model according to a range of luminance sample values.

[0032] Figure 15 illustrates a first linear model in which the slope of the linear model is adjusted using the received slope adjustment value.

[0033] Figure 16 illustrates a second linear model in which the slope of the linear model is adjusted using the received slope adjustment value.

[0034] Figure 17 illustrates a linear model derived by dividing the luminance value range into two regions.

[0035] FIG. 18 is a block diagram illustrating an embodiment of the present disclosure, which determines prediction and transformation units of an encoder, performs prediction and transformation for the determined units, modes, and methods, and ultimately performs restoration.

[0036] The present disclosure is susceptible to various modifications and embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present invention. Throughout the description of each drawing, similar reference numerals have been used to designate similar components.

[0037] While terms such as "first" and "second" may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a "second component," and similarly, a second component may also be referred to as a "first component." The term "and / or" includes a combination of multiple related items described herein or any of multiple related items described herein.

[0038] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0039] The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0040] Hereinafter, with reference to the attached drawings, preferred embodiments of the present invention will be described in more detail. Hereinafter, identical components in the drawings will be designated by the same reference numerals, and redundant descriptions of identical components will be omitted.

[0041]

[0042] FIG. 1 is a block diagram illustrating an embodiment of the present disclosure, which determines prediction and inverse transformation units of a decoder, performs prediction and inverse transformation for the determined units, modes, and methods, and ultimately performs restoration.

[0043] The order of each process in the block diagram of Fig. 1 may be changed or omitted.

[0044] The prediction unit determination unit is a step for determining the size and shape of a block for performing prediction, and may include all methods based on direct or indirect information transmitted from the encoder. This may include not only a method for direct size and shape information of the current decoded block (=current block, current encoded block), but also a method for utilizing information that may influence the determination of size and shape information of the current decoded block, such as the number of divisions, depth, shape of division, direction of division, size information for the minimum division block, division information of surrounding pre-decoded neighboring blocks, and prediction mode.

[0045] The prediction technology decision unit may be a step that determines the prediction technology for each block. This may include intra-screen prediction technology, inter-screen prediction technology, intra-block copy (IBC) technology, palette mode technology, or a technology that combines intra-screen and inter-screen prediction methods.

[0046] The prediction mode decision unit is a step for deciding on a prediction mode according to the technology decided in the prediction technology decision unit, and is a step for deciding on a method for deciding on the direction of intra-screen prediction, the number of reference pictures in inter-screen prediction, affine mode, merge mode, etc. of pixel values ​​of actual reference blocks. In some embodiments, the prediction technology decision unit and the prediction mode decision unit may be integrated into one step and operated in the same form as the prediction technology and mode decision unit.

[0047] The prediction execution unit can generate a first prediction signal block or a plurality of first prediction signal blocks through the above-determined prediction technique and prediction mode.

[0048] In the inverse transform unit determination unit, the inverse transform unit in which inverse transform will be performed on the residual signal is determined, and the kernel used for inverse transform is determined through the inverse transform performing unit, so that inverse transform on the residual signal can be performed.

[0049] Afterwards, a restoration signal can be generated by adding the prediction signal generated in the prediction execution unit and the residual signal generated in the inverse transformation execution unit.

[0050] The above inverse transform unit may refer to a unit that determines whether to perform inverse transform on a residual signal and transmits information about the inverse transform. Thereafter, in the inverse transform performing unit, a kernel to be applied to the inverse transform unit is determined, and inverse transform may be performed through the corresponding kernel. At this time, at least one kernel may be determined for one inverse transform unit. Inverse transform may be performed for N times according to an embodiment, and the sizes of the inverse transform unit and the inverse transform kernel may not be the same. That is, transformation may be performed on only some signals among the residual signal or transformation signal input to the inverse transform performing unit. The N may be a natural number equal to or greater than 1, 2, or 3, and may be a value predefined in the encoder / decoder or a value determined by information signaled from a bitstream.

[0051] According to an embodiment, the process of FIG. 1 may be performed independently for a luminance block and a chrominance block, respectively, and when the color format of the input video is a YUV format (YUV420, YUV411, YUV422, YUV444, etc.), the process may be performed for the luminance block and then for the chrominance block. According to an embodiment, when the color format of the input video is RGB, color conversion to YUV may be performed, and then encoding may be performed.

[0052] The following describes in detail the contents of the invention of the present disclosure with reference to Fig. 1.

[0053]

[0054] Prediction unit decision unit

[0055] According to one embodiment, in the encoder / decoder, a prediction unit is determined by a prediction unit determination unit, and the prediction unit may be a current block, one of the sub-blocks into which the current block is divided, a set of pixels, or a value of a single pixel, depending on the embodiment. The prediction unit may be determined by size information and shape information for performing prediction on a chrominance component and a luminance component. The prediction unit may be determined dependently or independently of the chrominance component and the luminance component. Dependently means that the prediction units of the luminance component or the chrominance components are not determined independently for each component, but rather that when the prediction unit of one component is determined, the units of the other components or components are determined with a corresponding size and shape. In this case, one component may correspond to one or more of the luminance component and the chrominance component. That is, the case may include a case where the luminance component is determined through information of the chrominance components. In another embodiment, the prediction unit of the chrominance component may have a size and shape determined based on the prediction unit of the luminance component according to a color format or a converted color format of the input video.

[0056] When determined dependently, information about the prediction unit of another component corresponding to a component, i.e., the dependently determined component, may be omitted. When determined independently, this means that the prediction units for the luminance component and the chrominance component are determined separately, and when determined independently, information about the prediction unit for each component may be transmitted separately. Here, the information about the prediction unit may include information that determines the size and shape of the prediction unit.

[0057]

[0058] Prediction Technology Decision Division

[0059] In one embodiment, the prediction technique of each prediction unit may be determined by the prediction technique determination unit. The prediction technique may be one of inter-screen prediction, intra-screen prediction, IBC, palette, or a technique combining intra-screen and inter-screen prediction methods.

[0060] In one embodiment, if the current block is not an intra-picture prediction technology, a 1-bit flag may be signaled / parsed, and if the flag is Skip, the inter-picture prediction merge mode or the IBC prediction merge mode may be determined, and in this case, the inverse transformation process may be omitted, so that the prediction signal may be used as the restoration signal. Here, skip may mean a method of not transmitting motion information (e.g., motion vector, reference picture, reference picture list, etc.) or transmitting motion information through only one syntax information, and additionally, a case in which a differential signal of the current block is not transmitted.

[0061] In one embodiment, if the current block is not a skip (if the flag indicates not a skip), a 1-bit flag for the prediction technique of the current block can be signaled / parsed to determine one of inter-picture prediction, intra-picture prediction, IBC, palette, etc. as the prediction technique of the current block.

[0062] In one embodiment, if the current block is not a skip and the prediction technique is determined as inter-screen prediction or IBC, it is possible to determine whether to perform prediction on the current block in merge mode or in advanced motion vector prediction (AMVP) mode by signaling and parsing a 1-bit flag. Here, AMVP mode may mean a mode in which each element in motion information is transmitted separately, and in addition to the motion information, a motion vector difference value is parsed and motion compensation is performed using the corrected motion information in a subsequent prediction process.

[0063]

[0064] Prediction mode decision unit

[0065] Determination of the prediction mode can be performed according to the above-determined prediction technique.

[0066] If the prediction technology of the current prediction unit is inter-frame prediction, the prediction mode of the current prediction unit block may perform motion compensation using motion information or motion information about the prediction unit and determine a signal or signals predicted through the motion compensation. In addition, it may be a mode for generating a prediction signal of the current prediction unit block as a weighted sum of a plurality of prediction signals including prediction signals generated through one or more motion compensations. In this case, one or more of the prediction signals may be signals of a previously decoded area of ​​the same frame as the current block.

[0067] In one embodiment, when the prediction technique of the current prediction unit uses a technique that combines intra- and inter-picture prediction methods, the surrounding pre-reconstructed region of the current prediction unit block may be defined as a template, and an intra-picture prediction mode may be derived using information on some or all pixels of the template. In this case, the template may include both adjacent and non-adjacent regions with respect to the current prediction unit block, and the non-adjacent region may be an region within a certain distance of a pixel line from the current prediction unit block. When the non-adjacent region is used as a template, the distance may be transmitted from the encoder to the decoder. Alternatively, in one embodiment, information about the distance may be defined by an agreement between the encoder and the decoder, and the transmission of the distance information may be omitted. When defined by an agreement, the value may be fixed to a specific constant value, or may be variably determined by the horizontal and vertical pixel lengths of the prediction unit block, the block width, etc. Thereafter, the derived intra-picture prediction mode may be used to generate a final prediction signal of the current block.

[0068] In one embodiment, if the prediction technique of the current prediction unit block is inter-screen prediction, the current prediction unit may have a shape size determined through geometric partitioning. In this case, the prediction signal may be a geometric partitioning-based prediction mode that generates the prediction signal of the current prediction unit block through a weighted sum of the prediction signals for each geometric partitioning unit.

[0069] According to one embodiment, when the prediction technique of the current prediction unit block is intra-screen prediction, the prediction mode of the current prediction unit block may be a mode that generates a prediction signal of the current prediction unit block using at least one of a directional prediction mode, a Planar mode (Horizontal Planar or Vertical Planar or Regular Planar), a DC mode, a smooth mode, a recursive prediction mode or a prediction mode based on correlation between components (e.g., CfL (Chroma from luma), CCLM (Cross component linear model), MHCCP (Multi-hypothesis cross component prediction), CCCM (Convolutional cross component model), etc.).

[0070] According to one embodiment, when the prediction technique of the current prediction unit block is intra-screen prediction, the current prediction unit block may be divided into one or more sub-regions through geometric partitioning, and a prediction signal may be generated for each region using a prediction mode within the intra-screen prediction technique including a different directional prediction mode or a Planar mode or a DC mode, and the prediction signal of the current prediction unit block may be generated through a weighted sum of the respective prediction signals, which may be an intra-screen geometric partitioning-based prediction mode.

[0071] According to one embodiment, if the prediction technology of the current prediction unit block is intra-screen prediction, it may be a matrix-based intra-screen prediction mode that performs prediction by signaling / parsing the index of the matrix using a matrix predefined by an agreement between the encoder / decoder, or by signaling / parsing the matrix.

[0072] According to one embodiment, when the prediction technique of the current prediction unit block is an intra-screen prediction, the intra-screen template matching prediction mode may be configured to define a pre-restored area around the current prediction unit block as a template and perform template matching in the pre-restored surrounding area of ​​the current prediction unit block to generate a prediction signal.

[0073] According to one embodiment, when the prediction technique of the current prediction unit block is intra-picture prediction, the pre-reconstructed region surrounding the current prediction unit block may be defined as a template, and the pre-reconstructed region surrounding the current prediction unit block may be defined as a template, and an intra-picture prediction mode may be derived using the template. Thereafter, the derived mode may be used to generate a final prediction signal of the current prediction unit block. At this time, the template may also include a region that is not adjacent to the current prediction unit block. The non-adjacent region may be an region within a certain distance of a pixel line from the current prediction unit block. When the non-adjacent region is used as a template, the distance may be transmitted from the encoder to the decoder. According to one embodiment, information about the distance may be defined by a convention between the encoding / decoding periods, and the transmission of the information may be omitted. When defined by a convention, the value may be fixed to a specific constant value, or may be variably determined by the length of horizontal and vertical pixels of the prediction unit block, the width of the block, etc.

[0074] In one embodiment, when the current prediction unit block is a chrominance block and the prediction technique of the current prediction unit block is intra-screen prediction, prediction may be performed through DM (Direct mode). As an example, DM mode may be a method of performing prediction of the current chrominance block using the same prediction method as the prediction method of the luminance block at the corresponding position of the current chrominance block.

[0075] According to one embodiment, when the current prediction unit block is a chrominance block and the prediction technique of the current prediction unit block is intra-screen prediction, the relationship between the surrounding restored chrominance samples of the current chrominance component and the surrounding restored luminance samples of the luminance block at the corresponding position of the current chrominance prediction unit block can be modeled as a linear or / and nonlinear model to generate a prediction signal of the current chrominance prediction unit block.

[0076]

[0077] Prediction Performance Department

[0078] According to one embodiment, when the current prediction unit block is a chrominance block and the prediction mode is determined as a prediction mode based on correlation between components, the relationship between the restored surrounding luminance samples of the luminance block at the corresponding position of the current chrominance block and the restored surrounding chrominance samples of the current chrominance block can be modeled, and prediction can be performed using the value determined through the modeling.

[0079] Figure 2 is a diagram showing a template of a chrominance component for pre-restored samples around a chrominance block currently being encoded / decoded.

[0080] Figure 3 is a drawing showing a template for the restoration samples around the luminance block at the corresponding position based on the template of the current chrominance block to be encoded / decoded.

[0081] According to one embodiment, as shown in FIG. 2, the shape and size of the template of the chrominance component may be defined for the restored samples around the chrominance block to be currently encoded / decoded. In addition, as shown in FIG. 3, a template corresponding to the template of the chrominance block may also be defined for the restored samples around the luminance block at the corresponding position of the chrominance block to be currently encoded / decoded, and the relationship between the chrominance component template and the luminance component template may be modeled as a linear and / or nonlinear model through at least one of a regression method, LDL decomposition, etc. The pixel values ​​used when the modeling is performed may be some or all of the values ​​within the template, and according to an embodiment, the pixel values ​​may be the average value, the median value, or the value converted through downsampling / upsampling filtering of some or all of the pixel values ​​within the template. In addition, a, b, m, n, p, and q of FIG. 2 may be integers greater than or equal to 1.

[0082] The embodiments through FIGS. 2 and 3 illustrate the case where the template and the current sub / decoding target block are adjacent, but the template and the current sub / decoding target block may not be adjacent. Non-adjacent means that they may not be located in a continuous coordinate space in terms of coordinates within the image, and even in the non-adjacent embodiments, templates of chrominance and luminance components may be defined with templates of various shapes and sizes, as in the cases of FIGS. 2 and 3. In the non-adjacent case, the distance relationship between the current sub / decoding target block and the template may be added to the definition of the template, and the distance may be implicitly determined or may be a value transmitted from the encoder to the decoder.

[0083] In one embodiment, the size and shape of the template may be implicitly determined based on the size of the current block, the definition of available samples among the surrounding pre-reconstruction samples, the prediction technique and method of the surrounding pre-reconstruction samples, whether the reference pictures of the current sub / decoding target block and the pre-reconstruction samples are the same, the aspect ratio of the current sub / decoding target block, etc., and some or all of the information about the shape and size of the template may be transmitted from the encoder to the decoder and determined by the decoder through signaling / parsing. Here, some or all may include a method in which some are determined implicitly and some are determined by transmission.

[0084] As an example where both the top and left templates are available, the position and number of templates used may vary depending on the width and height of the current chrominance block as follows:

[0085] 1) If the width of the current color difference block is more than twice the height, only M position samples of the upper template can be used.

[0086] 2) If the height of the current color difference block is more than twice its width, only N position samples of the left template can be used.

[0087] 3) In cases other than the conditions of the height and width of the current color difference block, both the K position samples of the upper template and the L position samples of the left template can be used.

[0088] Here, M, N, L, and K are integers greater than or equal to 1, and M and N can be 8, and K and L can be M / 2 and N / 2 or 4.

[0089]

[0090] In FIG. 3, p and q may be values ​​determined according to the color format of the input image or video. In this case, depending on the embodiment, the size of the template area for each component may be different based on the color format of the input image or video.

[0091] At this time, modeling can be performed after downsampling is performed on the template of the luminance component. For example, the shape and coefficients of the downsampling filter can be implicitly determined in the decoder, or transmitted from the encoder to the decoder, and the decoder can determine them by parsing from the bitstream. At this time, depending on the embodiment, there may be one or more modes for the chrominance block prediction mode based on the correlation between components, and different downsampling filters can be used for each mode. For example, the shape and coefficients of the downsampling filter can be implicitly determined in the decoder, or transmitted from the encoder to the decoder, and the decoder can determine them by parsing from the bitstream. At this time, the shape and coefficients of the filter can be transmitted separately, or / and can be transmitted as a single piece of information.

[0092] According to one embodiment, the relationship between a chrominance template and a luminance template can be modeled without performing downsampling, regardless of the color format of the input video. In this case, a nonlinear model can be modeled.

[0093]

[0094] Figure 4 illustrates a type of filter for downsampling according to the first embodiment.

[0095] According to one embodiment, when the color format of the input video is YUV420, p and q in FIG. 3 may be 2. In this case, the filter for downsampling may have the type of FIG. 4, and information about it may be signaled / parsed. According to one embodiment, downsampling of the luminance component may be applied to some samples among the samples of the template of the luminance component. In one embodiment, when modeling is performed with only some samples in the template of the chrominance component, downsampling may be performed only for the corresponding samples in the luminance template corresponding to some samples in the template of the chrominance component. The signaling unit for the downsampling filter may be at least one of a sequence unit, a picture unit, a subpicture unit, a slice unit, a tile unit, and a block unit. Alternatively, it may be signaled that a specific filter is used in an upper unit, and changed to a specific slice block unit in a lower unit and used. For example, the shape of the filter and the transmission unit of the coefficients may be different from each other.

[0096] For example, the unit determining whether to use a filter and the unit determining the type of filter may be different. For example, while the use of a filter is determined at the sequence level unless there are special circumstances, the unit determining the type of filter may be determined at the block level. Here, special circumstances mean that the use of a filter determined at the sequence level is changed at the block level, and this can be determined by a related flag.

[0097]

[0098] Figure 5 illustrates a type of filter for downsampling according to the second embodiment.

[0099] In one embodiment, when the color format of the input video is YUV422, p and q in FIG. 3 may be 2 and 1 or 1 and 2. In this case, when p and q in FIG. 3 are 2 and 1, the filter for downsampling may have the type of FIG. 5 (a), or when p and q in FIG. 3 are 2 and 1, the filter may have the type of FIG. 5 (b), and information about this may be signaled / parsed.

[0100] In some embodiments, downsampling of the luminance component may be applied to some samples among the samples of the template of the luminance component. In some embodiments, when modeling is performed with only some samples in the template of the chrominance component, downsampling may be performed only on corresponding samples in the luminance template corresponding to some samples in the template of the chrominance component. The signaling unit for the downsampling filter may be at least one of a sequence unit, a picture unit, a subpicture unit, a slice unit, a tile unit, and a block unit. Alternatively, it may be signaled that a specific filter is used in an upper unit and changed to a specific slice block unit in a lower unit. For example, the shape of the filter and the transmission unit of the coefficients may be different from each other.

[0101] For example, the unit determining whether to use a filter and the unit determining the type of filter may be different. For example, while the use of a filter is determined at the sequence level unless there are special circumstances, the unit determining the type of filter may be determined at the block level. Here, special circumstances mean that the use of a filter determined at the sequence level is changed at the block level, and this can be determined by a related flag.

[0102]

[0103] According to one embodiment, when the color format of the input video is YUV444, the relationship between the chrominance template and the luminance template can be modeled without performing downsampling.

[0104] Downsampling in this specification may mean not only a method determined through coefficients illustrated in a drawing, but also all cases of a filter having a smaller number of output values ​​than the input pixel values ​​by taking a plurality of pixel values ​​as input, and the method may be defined as at least one of a weighted average value, an average value, a median value, and a partial value.

[0105] According to one embodiment, for each color format type of the input video, an applicable downsampling filter can be defined by a decoder / encoder agreement, and information about this can be signaled / parsed.

[0106]

[0107] Figure 6 illustrates a type of filter for downsampling according to the third embodiment.

[0108] According to one embodiment, when the color format of the input video is YUV420, the shape and coefficients of the downsampling filter can be signaled / parsed respectively, and the signaling units of the shape and coefficients can be different. For example, the filter shape of the example of FIG. 6 can be applied. At this time, the number of filter shapes can be smaller or larger than that of the example of FIG. 6 depending on the case. For example, the number of shapes of available filters can be determined differently depending on the resolution of the input video. At this time, the filter shapes of the examples of FIGS. 6 (a) to (d) can be transmitted in sequence units, and the coefficients (a to i) of the filters of the examples of FIGS. 6 (a) to (d) can be transmitted in picture units. At this time, the filter coefficients can be determined in integer form, and the total sum of the coefficients in one filter can be a multiple of 2 or an exponentiation of 2. In other words, the unit in which the shape of the filter is determined and the unit in which the coefficients in the filter are determined can be different. As another example, the filter shapes of the examples of Figs. 6 (a) to (d) can be transmitted in sequence units, and the coefficients (a to i) of the filters of the examples of Figs. 6 (a) to (d) can be transmitted in block units. As another example, the filter shapes of the examples of Figs. 6 (a) to (d) can be transmitted in slice units, and the coefficients (a to i) of the filters of the examples of Figs. 6 (a) to (d) can be transmitted in block units.

[0109] In one embodiment, when the shape and coefficients of a filter are transmitted separately, the coefficients of the filter may be transmitted separately, and for one filter shape, a set of available filter coefficients may be defined by an agreement between a decoder and a decoder and managed in the form of a table, and the transmission of the filter coefficients may be transmitted in the form of an index. In this case, the table may be defined differently depending on the resolution of the input video, etc. In one embodiment, only one shape of the filter may be used for each color format, and only the coefficients of the filter may be transmitted separately to enable the use of various filters.

[0110]

[0111] Figure 7 illustrates the available filter shapes and the index to the filter coefficients for each filter shape.

[0112] In one embodiment, as shown in FIG. 7, after defining available filter shapes and available filter coefficient sets for each filter shape by a decoder / decoder agreement, the available filters can be organized in a table format and information about the filters can be transmitted in an index format. In this case, depending on the embodiment, the table can be organized differently for each series of units, such as a picture, a GOP (Group of Pictures), etc.

[0113] According to one embodiment, the downsampling filter may be determined by transmitting it in the form of a single index by defining it in a table format as shown in FIG. 7, or the shape and coefficient of the filter may be transmitted separately to determine the downsampling filter, and the information may be transmitted in the form of a 1-bit flag.

[0114] In one embodiment, during the process of applying a downsampling filter, padding may be performed on the boundary for filtering purposes during the downsampling process of samples located at the block boundary or / and the picture boundary. Alternatively, the filter for the area exceeding the filtering boundary may be modified so as not to include the area outside the boundary.

[0115] In one embodiment, the same concept as the embodiment described above for the YUV420 color format may be applied to other color formats than YUV420.

[0116]

[0117] Figure 8 illustrates a linear model obtained based on a luminance template and a chrominance template.

[0118] According to one embodiment, by using all samples in the luminance template and the chrominance template downsampled according to the color format, the relationship between the two templates can be modeled by using a method such as a linear minimum mean square error (LMMSE) predictor, and by deriving the values ​​of the scaling parameter α and the offset parameter β, as in the example of Fig. 8, a linear function such as Equation (1) can be modeled. In this case, recT c (i,j) is the template of the current chroma block (chroma template) and recT' L (i,j) may be a template (luminance template) of a corresponding luminance block downsampled according to a color format.

[0119] recT c (i,j)=α×recT' L (i,j)+β Equation (1)

[0120]

[0121] Figure 9 illustrates a step of obtaining a chrominance prediction signal based on a luminance template and a chrominance template.

[0122] In one embodiment, as in the example of FIG. 9, a linear function may be modeled using only two or more partial samples of a luminance template and a chrominance template, and prediction may be performed using the same.

[0123] The process of performing the above prediction may be a 'color difference sample extraction' and a 'luminance sample extraction' process.

[0124] As an example for the case where the number of samples used to derive the upper template is P, 1) if max(W / P, 1) is 1, samples at all positions can be used. Conversely, 2) if max(W / P, 1) is not 1, sampling can be performed evenly in units of max(W / P, 1) from the upper template, so that P samples can be obtained. In this case, the starting position of the sampling can be the position where (max(W / P, 1))>>1. This can be equally applied to the left template.

[0125] At this time, some samples may be samples at locations determined by agreement between the encoder and decoder. For example, a set of multiple determined locations may be defined, and information about the locations may be signaled / parsed in the form of an index or other such information. At this time, the determined locations may differ depending on the shape and area of ​​the template in FIG. 2.

[0126]

[0127] Figure 10 is a diagram showing the locations of samples used to derive a linear model.

[0128] According to one embodiment, when the upper and left areas of the current decoding target block are defined as templates, as in FIG. 10, a linear model can be derived using four samples in the chrominance template and four samples in the luminance template at corresponding positions. This can correspond to the 'chrominance sample extraction' and 'luminance sample extraction' processes of FIG. 9. At this time, the sample positions of each template can be positions defined by the decoder / decoder agreement according to the size of the current block. According to an embodiment, when not all samples in the luminance template are used, but only a specific number of samples at a set position are used, downsampling can be performed only on the sample positions to be brought in for model derivation, rather than downsampling the entire luminance template in the example of FIG. 9.

[0129] In one embodiment, a combination of two or more sample positions may be defined in a table format by a decoder / buffer agreement per template shape. For example, for a chrominance region, when the upper left position of a current chrominance block of size WxH is (0, 0), the combinations of four reference sample positions may be {(W / 4, -1), (3W / 4, -1), (-1, H / 4), (-1, 3H / 4)}, {(W / 4+1, -1), (3W / 4+1, -1), (-1, H / 4+1), (-1, 3H / 4+1)}, …, and a plurality of combinations may be configured in a table format. At this time, samples of which positions to use may be transmitted in the form of an index, and the positions may be determined implicitly. For example, the values ​​of four position samples may be checked for each in a set order, and the positions of the samples to be used for linear model derivation may be determined implicitly based on the corresponding sample values. As an example, for each luminance value of the samples to be used for model derivation within the template (luminance template) of the corresponding luminance block, if the difference between the maximum and minimum values ​​is less than a certain threshold (t), the same process can be performed for the next position in the template (luminance template) of the corresponding luminance block. By repeating this, a linear model can be derived using the samples at the corresponding positions within each template for samples at positions greater than or equal to the certain threshold (t). At this time, the threshold may be a value defined by an agreement between the encoder and decoder.

[0130] In an embodiment, when four luminance samples are arranged in ascending order of magnitude and are referred to as Y1, Y2, Y3, and Y4, respectively, four chrominance samples at corresponding positions are defined as C1, C2, C3, and C4, respectively, and then the average values ​​of Y1 and Y2 (Y avg1 ) and the average value of C1, C2 (C avg1 ) as one coordinate, the average value of Y3, Y4 (Y avg2 ) and the average value of C3, C4 (C avg2) into a single coordinate, the slope and y-intercept values ​​of the linear equation connecting the two points can be derived using the scaling parameter α and offset parameter β of Fig. 8 and equation (1). This can be performed as in equations (2) and (3), depending on the embodiment.

[0131]

[0132]

[0133] As another example, a linear model can be derived using six samples in a chrominance template and six samples in a corresponding luminance template. As another example, a linear model can be derived using eight samples in a chrominance template and eight samples in a corresponding luminance template.

[0134] According to one embodiment, a prediction signal of a current chrominance block to be decoded can be generated using a model for prediction. For a restored luminance block of a corresponding position of a current chrominance block to be decoded, which has been downsampled according to a color format of an input video, a chrominance prediction signal (pred) of each position c (i,j)) is the restored luminance signal (rec') of the corresponding position, as in Equation (4). L (i,j)) is input into the linear model to get the output value (pred c (i,j)) can be used to perform prediction.

[0135] pred c (i,j) = α×rec' L (i,j)+β Equation (4)

[0136] In one embodiment, at least one of the two parameters (α, β) representing scale and offset parameters for representing the correlation between the chrominance component and the luminance component as a linear model can be obtained through signaling / parsing. For example, α can be used by parsing, and β can be derived and used by the decoder. In this case, α can be transmitted as follows. For example, α of the Cb component and the Cr component in the chrominance component can be transmitted with different values ​​and can be transmitted separately. In this case, the sign value and the magnitude value can be transmitted separately.

[0137]

[0138] Figure 11 illustrates different embodiments of a step of obtaining a chrominance prediction signal by searching for a corresponding position luminance block.

[0139] According to one embodiment, as in the example of FIG. 11, the scale parameter α may be signaled from the encoder, and the offset parameter β may be used as a chrominance prediction signal. In this case, the chrominance prediction signal may be a prediction signal generated by performing intra-screen prediction using surrounding reference samples of the chrominance block currently being decoded.

[0140] In the example of Fig. 11(a), the prediction mode used for ‘generating the initial luminance prediction signal’ and the prediction mode used for ‘generating the initial chrominance prediction signal’ may be the same mode.

[0141] In one embodiment, when a linear model is derived through the example of FIG. 11, the linear model can be obtained and prediction can be performed using equation (5). In this case, recT C (i,j) is the template of the current chrominance block to be decoded (chrominance template) and rec' L,AC(i,j) may be a differential signal of the initial luminance prediction signal generation value from the restored luminance signal of the corresponding position downsampled according to the color format (in the case of Fig. 11 (a)), or may be a differential signal of the average value from the restored luminance signal of the corresponding position downsampled according to the color format (in the case of Fig. 11 (b)).

[0142] In the example of Fig. 11, the average value may mean a representative value, which may be defined as at least one of a weighted average value, a mean value, a median value, and some values.

[0143] pred c (i,j) = α×rec' L,AC (i,j)+β Equation (5)

[0144] Figure 12 illustrates different embodiments of obtaining a chrominance prediction signal based on a chrominance template, a luminance template, and an initial chrominance prediction signal.

[0145] According to one embodiment, as in the example of FIG. 12, both the scale parameter α and the offset parameter β may be derived and calculated by the encoder and used. At this time, the initial chrominance prediction signal may not be used. At this time, the offset parameter β may be used as a chrominance prediction signal, which may be a prediction signal generated by performing intra-screen prediction using surrounding reference samples of the current decoding target chrominance block, and may be used as a different value according to each position of the current decoding target chrominance block. At this time, in the example of FIG. 12(a), the prediction mode used for 'generating the initial luminance prediction signal' and the prediction mode used for 'generating the initial chrominance prediction signal' may be the same mode. According to an embodiment, the 'average value extraction' process may be omitted, and if omitted, the average value difference process may be omitted in the description below.

[0146] In one embodiment, when a linear model is derived through the example of FIG. 12, a linear model can be obtained as in Equation (6), and prediction can be performed using Equation (7).

[0147] In the process of deriving a linear model, pairs of restored luminance samples and restored chrominance samples to be used for derivation can be determined in the 'chrominance sample extraction' and 'luminance sample extraction' processes of Fig. 12 (a) and (b). This may be the same process as the 'chrominance sample extraction' and 'luminance sample extraction' processes described above. At this time, all samples in the downsampled luminance template and all samples in the chrominance template according to the color format can be used, and two or more samples can be extracted from each template and used for model derivation. According to an embodiment, when not all samples in the luminance template are used, but only a specific number of samples at a set position are used, downsampling can be performed only on sample positions to be brought in for model derivation, rather than downsampling the entire luminance template in the example of Fig. 12.

[0148]

[0149] According to the embodiment, recT of equation (6) C,AC (i,j) may be a template whose mean value is different from the template of the current chrominance block (chrominance template), and recT' L,AC (i,j) may be a template in which the average value of the corresponding luminance template downsampled according to the color format is differential, and β may be an initial chrominance prediction signal.

[0150] At this time, in the example of Fig. 12, the average value may mean a representative value, which may be defined as at least one of a weighted average value, an average value, a median value, a partial value, etc. According to an embodiment, rec' of Equation (7) L,AC (i,j) may be a differential signal generating an initial luminance prediction signal from a restored luminance signal of a corresponding position downsampled according to a color format (in the case of Fig. 12 (a)), or may be a differential signal generating an average value from a restored luminance signal of a corresponding position downsampled according to a color format (in the case of Fig. 12 (b)), and β may be an initial chrominance prediction signal.

[0151] word C,AC (i,j) = α×recT' L,AC (i,j)+β (6)

[0152] before c (i,j) = α×rec' L,AC (i,j)+β (7)

[0153] According to one embodiment, a combination of two or more sample positions may be defined in a table format by a decoder / buffer agreement per template shape. For example, for a chrominance region, when the upper left position of the current decoding target chrominance block of the size WxH is (0, 0), the combination of four reference sample positions may be {(W / 4, -1), (3W / 4, -1), (-1, H / 4), (-1, 3H / 4)}, {(W / 4+1, -1), (3W / 4+1, -1), (-1, H / 4+1), (-1, 3H / 4+1)}, …, and a plurality of combinations may be configured in a table format. At this time, the samples of which positions to use may be transmitted in the form of an index, and the positions may be determined implicitly. For example, the values ​​of the four position samples may be checked for each in a set order, and the positions of the samples to be used for linear model derivation may be determined implicitly based on the corresponding sample values. As an example, for each luminance value of the samples to be used for model derivation within the template (luminance template) of the corresponding luminance block, if the difference between the maximum and minimum values ​​is less than a certain threshold (t), the same process can be repeated for the next order position of the template (luminance template) of the corresponding luminance block. By repeating this, a linear model can be derived using the samples at the corresponding positions within each template for the samples at positions greater than or equal to the certain threshold (t). At this time, the threshold may be a value defined by an agreement between the encoder and decoder. At this time, in the process of implicitly determining the position of the sample by repeatedly comparing it with the threshold as in the above example, if the condition is not satisfied by comparing it with the threshold, this process may be performed only up to a certain order, and if the condition is not satisfied by comparing it with the threshold, the prediction mode based on the correlation between the components may not be used.Alternatively, this process can be performed only up to a certain order, and if the condition is still not satisfied by comparison with the threshold, prediction can be performed using the sample location of the last order, or / and using the sample location of the order that is closest to the condition by comparison with the threshold among the previous results.

[0154]

[0155] Depending on the embodiment, when only some samples, not all samples, in the luminance and chrominance templates are used for model derivation, the method of model derivation may vary. For example, when the model is derived using four samples each in the luminance and chrominance templates, the mean values ​​of the corresponding positions of the luminance templates downsampled according to the color format are defined as Y1, Y2, Y3, and Y4 in ascending order of size, respectively, and the mean values ​​of the corresponding positions are defined as C1, C2, C3, and C4 for the four samples in the chrominance templates, and then the mean values ​​of Y1 and Y2 (Y avg1 ) and the average value of C1, C2 (C avg1 ) as one coordinate, the average value of Y3, Y4 (Y avg2 ) and the average value of C3, C4 (C avg2 ) into a single coordinate, the slope and y-intercept values ​​of the linear equation connecting the two points can be derived using the scaling parameter α and offset parameter β of Fig. 8 and equation (1).

[0156]

[0157] Figure 13 illustrates a first embodiment of deriving a linear model according to a range of luminance sample values.

[0158] Figure 14 illustrates a second embodiment of deriving a linear model according to a range of luminance sample values.

[0159] According to one embodiment, when the prediction mode of the current chrominance block is a prediction mode based on correlation between components, in the process of deriving a model for predicting the current chrominance block, a separate linear model can be derived and obtained according to the range of luminance sample values, as in the embodiments of FIGS. 13 and 14.

[0160] Looking at the examples of FIGS. 13 and 14, after obtaining one linear model, such as equation (2), equation (3), or equation (6) of the above example, the luminance sample value LT' AC1 , LT' AC2 For , different linear models can be obtained depending on the range of luminance sample values ​​and used for prediction. In an example, LT' AC1 , LT' AC2 may be a value determined as follows. In one embodiment, it may be applied differently depending on the number of samples used in the luminance and chrominance templates for model derivation. In this case, in the embodiments of FIGS. 13 and 14, L'(i,j) and C'(i,j) may be position-specific values ​​of the restored luminance component and position-specific values ​​of the restored chrominance component, respectively, and may have different meanings depending on the embodiment.

[0161] As an example, in the embodiment of Fig. 7, when N luminance samples in the luminance template and N chrominance samples in the chrominance template are used, and as an example, when N is 4, LT' AC1 , LT' AC2 Each of the four luminance samples can be the average of two values ​​with large brightness values ​​and the average of two values ​​with small brightness values.

[0162] As an example, in the embodiment of FIG. 7, when all luminance samples in the luminance template and all chrominance samples in the chrominance template are used, LT' AC1 , LT' AC2 Each of the samples in the luminance template can have the largest and smallest brightness values.

[0163] As an example, in the embodiment of FIGS. 11 and 12, when N luminance samples in the averaged luminance template and N chrominance samples in the averaged chrominance template are used, and as an example, when N is 4, LT' AC1 , LT' AC2 Each of the four luminance samples can be the average of two values ​​having a large brightness value and the average of two values ​​having a small brightness value. In this case, depending on the value of N, the average value can be defined as at least one of a weighted average, a mean, a median, and some value. For example, when N is 4, the average value can be used, and when N is greater than 8, the median value can be used.

[0164] As an example, in the embodiments of FIGS. 11 and 12, when all luminance samples in the averaged luminance template and all chrominance samples in the averaged chrominance template are used, LT' AC1 , LT' AC2 Each of the samples in the averaged luminance template may have the largest and smallest brightness values.

[0165] In some embodiments, LT' AC1 If the difference between the smallest available luminance value and the lowest available luminance value is less than a certain threshold, or / and LT' AC2 If the difference between the largest available luminance value and the maximum available luminance value is less than a certain threshold, the embodiments such as FIGS. 13 and 14 may not be applied to the corresponding area. Or / and LT' AC1 , LT' AC2 If the difference value is greater than a certain threshold value, embodiments such as FIGS. 13 and 14 may not be applied. The threshold values ​​described in the above specification may have different values, may be values ​​defined by the encoder / decoder agreement, or may be values ​​determined according to the size of the current block, etc.

[0166] According to the embodiment, in the example of FIG. 13, LT' AC1, LT' AC2 A linear model with an adjusted slope can be used using integer values ​​u and w as a reference. For example, when the embodiment of Fig. 13 is applied to the derived linear model, the values ​​u and w can be transmitted. In this case, u and w can be the same value, in which case only one value can be transmitted. The modified linear model can be expressed as in Equations (8), (9), and (10).

[0167]

[0168]

[0169]

[0170] In one embodiment, the decoder can implicitly adjust the slope without transmitting the u and w values ​​in the example of FIG. 13. As an example, in the linear model derived from the example above, the luminance sample value LT' AC1 , LT' AC2 About LT' AC1 In the range below / below {(LT' AC1 , α×LT' AC1 +β), (L min , C min )} A straight line passing through two points, LT' AC2 In the ideal / exceeding range {(LT' AC2 , α×LT' AC1 +β), (L max , C max )} You can use a straight line passing through two points. In this case, L min , C min are the smallest luminance value and chrominance value, L, that can be used for actual prediction in the example of Fig. 13, respectively. max , C max can be the largest luminance value and chrominance value that can be used for actual prediction in the example of Fig. 13, respectively.

[0171] According to the embodiment, in the example of FIG. 14, LT' AC1 , LT'AC2 A linear model whose slope is adjusted using integer values ​​V1 and V2 can be used as a reference. For example, for the derived linear model, when the embodiment of Fig. 14 is applied, the values ​​V1 and V2 can be transmitted and / or derived. In this case, V1 and V2 can be the same value or the average value of the color difference template. If V1 and V2 have different values, V1 = αLT' AC1 +β, V2=αLT' AC2 It can be +β. The modified linear model can be expressed as Equation (11).

[0172]

[0173] According to one embodiment, in the examples of FIGS. 11 and 12, both luminance and chrominance can use the prediction mode in DC mode. In this case, when using DC mode, the same downsampling method used in the downsampling process of the corresponding position luminance block can be applied to the luminance template, and then the value of the downsampled luminance template applied in DC mode can be used.

[0174]

[0175] In one embodiment, when performing prediction of a chrominance block based on a linear model derived between a chrominance component and a luminance component, as in the embodiments of FIGS. 9 and 12, the slope of the linear model may be adjusted by adjusting the derived scale parameter α through additional signaling / parsing. To this end, a 1-bit flag information indicating whether to adjust the slope may be signaled / parsed, and if the slope is adjusted, a constant value to be adjusted may be signaled / parsed. At this time, the adjustable slope value may be transmitted in the form of an index within a table defined by an agreement between a decoder and a processor.

[0176] Depending on the embodiment, only one of the slope adjustment of the linear model and the transmission of the additional offset parameter can be applied to a block.

[0177] Figure 15 illustrates a first linear model in which the slope of the linear model is adjusted using the received slope adjustment value.

[0178] According to an embodiment, when performing prediction of a chrominance block based on a linear model derived between a chrominance component and a luminance component, as in the embodiment of FIG. 9, when adjusting the slope of the linear model, the slope (α) of the linear model derived as in Equation (1) can be adjusted through Equations (12) and (13) using the slope adjustment value (u) transmitted as in FIG. 15 to obtain the linear model of Equation (14). At this time, Luma of Equation (13) m The value may be an average of two or more samples within a downsampled luminance template according to the color format, or a value transmitted from the encoder.

[0179] α' = α + u Equation (12)

[0180] β' = β - u + Luma m Equation (13)

[0181] recT C (i,j) = α' × recT' L (i,j) + β' Equation (14)

[0182] Figure 16 illustrates a second linear model in which the slope of the linear model is adjusted using the received slope adjustment value.

[0183] According to an embodiment, when performing prediction of a chrominance block based on a linear model derived between a chrominance component and a luminance component, as in the embodiment of FIG. 12, when adjusting the slope of the linear model, the slope (α) of the linear model can be adjusted using the slope adjustment value (u) transmitted as in FIG. 16 to obtain a linear model.

[0184] As an example, as shown in Fig. 16 (a), only the slope (α) of the linear model can be adjusted according to the slope change. At this time, the adjusted slope (α') of Fig. 16 (a) can be obtained using the slope adjustment value (u) transmitted as in Equation (12). Thereafter, the adjusted slope (α') can be applied to Equation (7) to perform prediction of the chrominance block.

[0185] As an example, as shown in Fig. 16 (b), the slope (α) and offset parameter (β) of the linear model can be adjusted according to the slope change. At this time, the adjusted slope (α') can be obtained through equation (12) and the adjusted offset parameter (β') can be obtained through equation (9) using the received slope adjustment value (u). In this case, Luma in equation (9) m The value may be an average of two or more samples in a downsampled luminance template according to a color format, an average of a corresponding position luminance block downsampled according to a color format, an implicitly determined value (for example, 0), or a value transmitted from an encoder. Then, the adjusted slope (α') and the adjusted offset parameter (β') may be applied to equation (7) to perform prediction of the chrominance block. In the above example, the average value may mean a representative value, which may be defined as at least one of a weighted average value, an average value, a median value, and a partial value.

[0186] According to one embodiment, when the prediction mode of the current chrominance block is a prediction mode based on correlation between components, multiple models can be derived and used to predict the current chrominance block without dividing the current block into sub-regions.

[0187]

[0188] Figure 17 illustrates a linear model derived by dividing the luminance value range into two regions.

[0189] In one embodiment, information indicating how multiple models are derived to predict the current chrominance block can be signaled / parsed in the form of a 1-bit flag. If this information is true, two or more models can be derived and used to predict the current chrominance block. As an example, as shown in FIG. 17, the luminance value range can be divided into two regions based on τ, thereby deriving two models. For example, in the case of obtaining a linear model as shown in FIGS. 9 and 12, in the parameter derivation process, in the case of FIG. 9, τ may be the average value of two or more samples in the luminance template downsampled according to the color format, or a value transmitted from the encoder, and in the case of FIG. 12, τ may be the average value of two or more samples in the luminance template downsampled according to the color format, or the average value of the corresponding position luminance block downsampled according to the color format, or an implicitly determined value (as an example, 0), or a value transmitted from the encoder, and in the case of FIG. 11, in the case of obtaining a linear model, a scaling parameter for each model may be transmitted. At this time, τ may be the average value of two or more samples in the luminance template downsampled according to the color format, or the average value of the corresponding position luminance block downsampled according to the color format, or an implicitly determined value (as an example, 0), or a value transmitted from the encoder. In the above example, the average value may mean a representative value, which may be defined as at least one of a weighted average value, a mean value, a median value, and some values.

[0190]

[0191] The above embodiment relates to a method for modeling the relationship between local sample values ​​of the chrominance component and the luminance component by using information about samples of the decoded surrounding area when the prediction for the luminance component is performed in the order in which the prediction for the chrominance component is performed during the process of performing encoding / decoding, and reflecting the modeled result in the prediction of the chrominance component. According to an embodiment, if the encoding / decoding order of the chrominance component and the luminance component is changed so that the chrominance component is encoded / decoded first and the luminance component is encoded / decoded later, the above embodiment can be modified and applied. The general modeling method that is modified and applied is the same, but it is possible for the downsampling process to be changed to upsampling and applied depending on the sampling ratio of the luminance component and the chrominance component. This relates to a general method in an area that can be inferred by a person having ordinary skill in the art.

[0192]

[0193] Transform coefficient entropy decoding unit

[0194] According to one embodiment, the transform coefficient entropy decoding unit can restore quantized second-order transform coefficients when a second-order transform is applied, and can perform restoration of quantized first-order transform coefficients when a second-order transform is not applied.

[0195]

[0196] Transformation coefficient inverse quantization part

[0197] According to one embodiment, the transform coefficient dequantization unit can perform dequantization by parsing information such as quantization method and quantization parameter information for the restored transform coefficients to obtain dequantized transform coefficients.

[0198]

[0199] Inverse transformation unit decision unit

[0200] According to one embodiment, the transform and inverse transform units in the encoder / decoder may be one transform unit (TU) or each sub-block that divides one TU into multiple units in the inverse transform unit determination unit.

[0201]

[0202] Inverse transform kernel decision unit and inverse transform execution unit

[0203] According to one embodiment, the inverse transform kernel determination unit can determine separable vertical and horizontal first-order inverse transform kernels and / or non-separable second-order inverse transform kernels, or determine a non-separable first-order inverse transform kernel.

[0204] According to one embodiment, the inverse transform performing unit can perform inverse transform on the inverse quantized transform coefficients using the inverse transform kernel determined by the inverse transform kernel determining unit.

[0205] In one embodiment, whether to perform a non-separable first-order inverse transform and whether to perform a non-separable second-order inverse transform can be determined through signaling / parsing, and can be determined implicitly based on the size of the current TU, etc.

[0206]

[0207] Restoration Department

[0208] According to one embodiment, a residual signal restored through an inverse transform performing unit can be combined with a final prediction signal generated through a prediction performing unit to generate a final restored signal.

[0209]

[0210] FIG. 18 is a block diagram of one embodiment of the present disclosure, which determines prediction and transformation units of an encoder and performs prediction and transformation for the determined units, modes, and methods to encode a residual signal.

[0211] The order of each process in the block diagram of Fig. 18 may be changed or omitted. Fig. 18 is a block diagram corresponding to the decryption block diagram of Fig. 1.

[0212] The prediction unit determination unit is a step for determining the size and shape of a block for performing prediction, and direct or indirect information about the determined prediction unit can be encoded. The information may include not only a method for direct size and shape information of the current coded block (=current block, current decoded block), but also information related to the size and shape of the current block, such as the number of divisions, depth, shape of division, direction of division, size information for the minimum division block, division information of surrounding pre-decoded neighboring blocks, and prediction mode.

[0213] The prediction technology decision unit may be a step that determines the prediction technology for each block. This may include intra-screen prediction technology, inter-screen prediction technology, intra-block copy (IBC) technology, palette mode technology, or a technology that combines intra-screen and inter-screen prediction methods.

[0214] The prediction mode decision unit is a step for deciding on a prediction mode according to the technology decided in the prediction technology decision unit, and is a step for deciding on a method for deciding on the direction of intra-screen prediction, the number of reference pictures in inter-screen prediction, affine mode, merge mode, etc. of pixel values ​​of actual reference blocks. In some embodiments, the prediction technology decision unit and the prediction mode decision unit may be integrated into one step and operated in the same form as the prediction technology and mode decision unit.

[0215] The prediction execution unit can generate a first prediction signal block or a plurality of first prediction signal blocks through the above-determined prediction technique and prediction mode.

[0216] If the prediction signal block generated by the prediction performing unit is a prediction signal block that exhibits optimal encoding / decoding efficiency, information related to the determined prediction mode, prediction technique, prediction unit, etc. can be transmitted to the encoder via a bitstream as prediction-related information.

[0217] The residual block acquisition unit can acquire a residual signal based on the generated prediction signal and the original signal. Here, the generated prediction signal may be a prediction signal that exhibits optimal encoding / decoding efficiency.

[0218] In the transformation unit determination unit, the transformation unit in which transformation will be performed on the residual signal is determined, and the kernel used for transformation can be determined through the transformation execution unit to perform transformation on the residual signal. The transformation unit corresponds to the aforementioned inverse transformation unit, and a detailed description thereof is omitted as described above.

[0219] The transform kernel determining unit can determine separable vertical and horizontal first-order transform kernels and / or non-separable second-order transform kernels, or can determine non-separable first-order transform kernels.

[0220] The transformation performing unit can perform transformation on the residual signal using the transformation kernel determined by the transformation kernel determining unit.

[0221] The transform coefficient quantization unit can perform quantization on the transformed coefficient to obtain a quantized transform coefficient. Here, the quantization method and quantization parameter information can be encoded and transmitted to the decoder.

[0222] The transform coefficient entropy encoding unit can encode quantized transform coefficients and transmit them to a decoder via a bitstream. If a secondary transform is applied, the quantized secondary transform coefficients can be encoded, and if no secondary transform is applied, the quantized primary transform coefficients can be encoded.

[0223]

[0224] While the exemplary methods of this disclosure are presented as a series of operations for clarity of description, this is not intended to limit the order in which the steps are performed, and individual steps may be performed simultaneously or in different orders, if desired. To implement a method according to this disclosure, additional steps may be included in addition to the steps illustrated, some steps may be excluded and the remaining steps included, or some steps may be excluded and additional steps included.

[0225] The various embodiments of the present disclosure are not intended to list all possible combinations but rather to illustrate representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combinations of two or more.

[0226] Additionally, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of hardware implementation, the embodiments may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general processors, controllers, microcontrollers, microprocessors, etc.

[0227] The scope of the present disclosure includes software or machine-executable instructions (e.g., operating systems, applications, firmware, programs, etc.) that cause operations according to the methods of various embodiments to be executed on a device or a computer, and a non-transitory computer-readable medium having such software or instructions stored thereon and executable on the device or computer.

[0228]

[0229]

[0230] The present disclosure may be industrially applicable in the fields of encoding / decoding methods, devices, and recording media for predicting a color difference block using correlation between components.

Claims

1. A prediction unit determination step for determining a prediction unit that performs prediction of a color difference component; A prediction technique determination step for determining a prediction technique to be applied to the above prediction unit; A prediction mode determination step for determining a prediction mode for predicting the color difference component based on the above prediction technology; and An image decoding method, characterized in that it includes a prediction performing step of performing prediction of the chrominance component based on the prediction mode.

2. In paragraph 1, An image decoding method, characterized in that the above prediction unit is a chrominance block.

3. In paragraph 2, A video decoding method, characterized in that the above prediction technology is an intra-screen prediction technology, an inter-screen prediction technology, an IBC (Intra Block Copy) technology, a palette mode technology, or a technology that is a mixture of intra-screen prediction and inter-screen prediction.

4. In paragraph 3, A video decoding method, characterized in that, in response to the above prediction technique being determined as a technique other than an on-screen prediction technique, the prediction mode is determined based on a Skip flag.

5. In paragraph 1, A video decoding method, characterized in that the prediction mode is determined using information of some or all pixels of the template of the prediction unit in response to the prediction technique being determined to be a mixed technique of intra-screen prediction and inter-screen prediction.

6. In paragraph 1, An image decoding method, characterized in that the shape of the above prediction unit is a shape determined through geometric division.

7. In paragraph 1, A method for decoding an image, characterized in that, in response to a case where the above prediction technique is determined as an in-screen prediction technique, the prediction mode is any one of CfL (Chroma from luma), CCLM (Cross component linear model), MHCCP (Multi-hypothesis cross component prediction), and CCCM (Convolutional cross component model).

8. In paragraph 1, An image decoding method, characterized in that the prediction of the above chrominance component is performed based on a luminance block at a position corresponding to the prediction unit and pre-restored surrounding luminance samples of the luminance block.

9. In paragraph 1, The prediction of the above chrominance component is performed based on a linear model between the chrominance component and the luminance component, An image decoding method, characterized in that the linear model includes a scale parameter and an offset parameter.

10. In paragraph 9, An image decoding method, characterized in that the linear model is derived based on a chrominance template of the prediction unit and a luminance template at a position corresponding to the chrominance template.

11. In paragraph 10, An image decoding method, characterized in that the linear model is derived based on representative values ​​of all or part of the samples of the chrominance template and representative values ​​of all or part of the samples of the luminance template.

12. A prediction unit determination step for determining a prediction unit that performs prediction of a color difference component; A prediction technique determination step for determining a prediction technique to be applied to the above prediction unit; A prediction mode determination step for determining a prediction mode for predicting the color difference component based on the above prediction technology; and An image encoding method, characterized in that it includes a prediction performing step of performing prediction of the chrominance component based on the prediction mode.

13. In a computer-readable recording medium storing a bitstream generated by an image encoding method, The above image encoding method comprises: a prediction unit determination step of determining a prediction unit that performs prediction of a chrominance component; A prediction technique determination step for determining a prediction technique to be applied to the above prediction unit; A prediction mode determination step for determining a prediction mode for predicting the color difference component based on the above prediction technology; and A computer-readable recording medium, characterized in that it includes a prediction performing step of performing prediction of the color difference component based on the prediction mode.

Images