Method and apparatus for processing video signal

Abstract

An image decoding method and apparatus according to the present disclosure may generate at least two prediction blocks for a current block, and generate a final prediction block of the current block by blending the at least two prediction blocks on the basis of a prescribed mask.

Description

Video signal processing method and device

[0001] The present invention relates to a video signal processing method and device, and more particularly, to a video encoding and decoding method and device.

[0002] The market demand for high-resolution video is growing, necessitating technologies capable of efficiently compressing high-resolution images. To address this market need, the ISO / IEC's Moving Picture Expert Group (MPEG) and the ITU-T's Video Coding Expert Group (VCEG) jointly formed the Joint Collaborative Team on Video Coding (JCT-VC). They completed development of the HEVC (High Efficiency Video Coding) video compression standard in January 2013 and have been actively conducting research and development on next-generation compression standards.

[0003] Video compression is broadly comprised of intra-frame prediction (or intra-prediction), inter-frame prediction (or inter-prediction), transform, quantization, entropy coding, and in-loop filtering. Meanwhile, as demand for high-resolution video grows, so does the demand for stereoscopic video content as a new video service. Discussions are underway on video compression technologies to effectively deliver high-resolution and ultra-high-resolution stereoscopic video content.

[0004] The purpose of the present invention is to provide a video signal processing method and device.

[0005] The video decoding method and device according to the present disclosure can generate at least two prediction blocks for a current block, and generate a final prediction block of the current block by weighting and adding the at least two prediction blocks based on a predetermined mask. Here, each of the at least two prediction blocks is a unidirectional or bidirectional prediction block, and the bidirectional prediction block can be generated based on a weighted sum of an L0 prediction block and an L1 prediction block.

[0006] In the image decoding method and device according to the present disclosure, the mask can be determined based on geometric segmentation information.

[0007] In the image decoding method and device according to the present disclosure, the geometric segmentation information can specify any one of a plurality of mask candidates pre-defined in the image decoding device.

[0008] In the image decoding method and device according to the present disclosure, the geometric segmentation information may include a flag indicating whether an inverted mask is used.

[0009] In the image decoding method and device according to the present disclosure, the mask can be generated based on a difference block between the at least two prediction blocks.

[0010] In the image decoding method and device according to the present disclosure, the mask may be generated based on a ratio block between the at least two prediction blocks. Here, the ratio block may be derived by dividing a sample value belonging to one of the at least two prediction blocks by a sample value at the same position belonging to the other one.

[0011] In the video decoding method and device according to the present disclosure, each of the at least two prediction blocks can be generated based on any one of an inter mode, an intra mode, or an intra block copy mode.

[0012] In the video decoding method and device according to the present disclosure, the weight for the weighted sum may be determined based on at least one of the ratio of surrounding blocks to which the intra mode is applied or the POC difference between the current picture to which the current block belongs and the reference picture.

[0013] The video encoding method and device according to the present disclosure can generate at least two prediction blocks for a current block, and generate a final prediction block of the current block by weighting and adding the at least two prediction blocks based on a predetermined mask. Here, each of the at least two prediction blocks is a unidirectional or bidirectional prediction block, and the bidirectional prediction block can be generated based on a weighted sum of an L0 prediction block and an L1 prediction block.

[0014] A computer-readable digital storage medium is provided having stored thereon video / image information that causes a method of decrypting a message according to the present disclosure to be performed.

[0015] A computer-readable digital storage medium storing video / image information generated by a mesh encoding method according to the present disclosure is provided.

[0016] A method and device for transmitting video / image information generated by a mesh encoding method according to the present disclosure are provided.

[0017] According to the present disclosure, the prediction performance of the final prediction block can be improved by using an adaptive mask on the texture of the current block.

[0018] According to the present disclosure, prediction performance can be improved by using parameters based on the correlation between the current template and the reference template.

[0019] Figure 1 is a block diagram showing an image encoding device according to the present invention.

[0020] Figure 2 is a block diagram showing an image decoding device according to the present invention.

[0021] FIG. 3 illustrates a prediction method performed by an image encoding / decoding device as an embodiment according to the present disclosure.

[0022] FIG. 4 illustrates a mask-based blending method for multiple prediction blocks as an embodiment according to the present disclosure.

[0023] Figures 5 to 7 illustrate a mask generation method according to the present disclosure.

[0024] FIG. 8 illustrates a parameter-based prediction method as an example according to the present disclosure.

[0025] FIG. 9 illustrates a parameter-based prediction method in color inter-prediction of intra prediction as an embodiment according to the present disclosure.

[0026] FIG. 10 illustrates a parameter-based prediction method in intra block copy (IBC) mode as an embodiment according to the present disclosure.

[0027] Figure 11 illustrates a parameter-based prediction method in inter prediction mode.

[0028] FIG. 12 is an example according to the present disclosure, illustrating a relationship between two different areas.

[0029] FIG. 13 is an example according to the present disclosure, illustrating multiple associations for two different areas.

[0030] FIG. 14 is an example according to the present disclosure, showing nonlinear correlations for two different areas.

[0031] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings attached to this specification so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the drawings, parts irrelevant to the description have been omitted to clearly explain the present invention, and similar parts have been designated with similar reference numerals throughout the specification.

[0032] Throughout this specification, when a part is said to be 'connected' to another part, this includes not only cases where they are directly connected, but also cases where they are electrically connected with another element in between.

[0033] Additionally, whenever a part throughout this specification is said to "include" a component, this does not mean that other components are excluded, but rather that other components may be included, unless specifically stated otherwise.

[0034] Additionally, while terms such as "first," "second," etc. may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another.

[0035] Additionally, in the embodiments of the devices and methods described herein, some components of the devices or some steps of the methods may be omitted. Furthermore, the order of some components of the devices or some steps of the methods may be changed. Furthermore, other components or other steps may be inserted into some components of the devices or some steps of the methods.

[0036] Additionally, some components or some steps of the first embodiment of the present invention may be added to the second embodiment of the present invention, or some components or some steps of the second embodiment may be replaced.

[0037] In addition, the components shown in the embodiments of the present invention are independently depicted to represent different characteristic functions, and this does not mean that each component is composed of separate hardware or a single software component. That is, each component is described by listing each component for convenience of explanation, and at least two components among each component may be combined to form a single component, or a single component may be divided into multiple components to perform a function. Such integrated and separate embodiments of each component are also included in the scope of the present invention as long as they do not deviate from the essence of the present invention.

[0038] First, the terms used in this application are briefly explained as follows.

[0039] The decoding device (Video Decoding Apparatus) described below may be a device included in a server terminal such as a civilian security camera, a civilian security system, a military security camera, a military security system, a personal computer (PC), a laptop computer, a portable multimedia player (PMP), a wireless communication terminal, a smart phone, a TV application server, a service server, etc., and may refer to various devices including user terminals such as various devices, communication devices such as a communication modem for communicating with a wired or wireless communication network, a memory for storing various programs and data for decoding a video or for inter or intra prediction for decoding, and a microprocessor for executing a program for calculation and control.

[0040] In addition, an image encoded into a bitstream by an image encoding device may be transmitted to an image decoding device through wired or wireless communication networks such as the Internet, a local area network, a wireless LAN, a WiBro network, a mobile communication network, etc. in real time or non-real time, or through various communication interfaces such as a cable or a universal serial bus (USB), and may be decoded, restored into an image, and played back. Alternatively, the bitstream generated by the image encoding device may be stored in a memory. The memory may include both volatile memory and non-volatile memory. In the present specification, the memory may be expressed as a recording medium that stores a bitstream.

[0041] Typically, a video may be composed of a series of pictures, and each picture may be divided into coding units such as blocks. In addition, those skilled in the art to which this embodiment pertains will understand that the term "picture" described below may be replaced with other terms having equivalent meanings, such as "image" or "frame." In addition, those skilled in the art to which this embodiment pertains will understand that the term "coding unit" may be replaced with other terms having equivalent meanings, such as "unit block" or "block."

[0042] Hereinafter, embodiments of the present invention will be described in more detail with reference to the attached drawings. In describing the present invention, duplicate descriptions of identical components will be omitted.

[0043] Figure 1 is a block diagram showing an image encoding device according to the present invention.

[0044] Referring to FIG. 1, a conventional image encoding device (100) may include a picture segmentation unit (110), a prediction unit (120, 125), a transformation unit (130), a quantization unit (135), a reordering unit (160), an entropy encoding unit (165), an inverse quantization unit (140), an inverse transformation unit (145), a filter unit (150), and a memory (155).

[0045] The picture segmentation unit (110) can segment the input picture into at least one processing unit. At this time, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). In the following embodiments of the present invention, the coding unit may be used to mean a unit that performs encoding or a unit that performs decoding.

[0046] A prediction unit may be divided into at least one square or rectangular shape of the same size within a single coding unit, or may be divided such that one prediction unit among the divided prediction units within a single coding unit has a different shape and / or size from another prediction unit. When a prediction unit that performs intra prediction based on a coding unit is generated and is not the minimum coding unit, intra prediction can be performed without being divided into a plurality of NxN prediction units.

[0047] The prediction unit (120, 125) may include an inter prediction unit (120) that performs inter prediction or inter-screen prediction, and an intra prediction unit (125) that performs intra prediction or intra-screen prediction. It may determine whether to use inter prediction or intra prediction for a prediction unit, and determine specific information (e.g., intra prediction mode, motion vector, reference picture, etc.) according to each prediction method. A residual value (residual block) between the generated prediction block and the original block may be input to the transformation unit (130). In addition, prediction mode information, motion vector information, etc. used for prediction may be encoded together with the residual value by an entropy encoding unit (165) and transmitted to an image decoding device.

[0048] The inter prediction unit (120) may predict a prediction unit based on information of at least one picture among the previous or subsequent pictures of the current picture, and in some cases, may predict a prediction unit based on information of a portion of an encoded region within the current picture. The inter prediction unit (120) may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0049] The reference picture interpolation unit can receive reference picture information from the memory (155) and generate pixel information less than an integer pixel from the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter with different filter coefficients can be used to generate pixel information less than an integer pixel in units of 1 / 4 pixels. In the case of a chrominance signal, a DCT-based 4-tap interpolation filter with different filter coefficients can be used to generate pixel information less than an integer pixel in units of 1 / 8 pixels.

[0050] The motion prediction unit can perform motion prediction based on a reference picture interpolated by the reference picture interpolation unit. Various methods such as FBMA (Full search-based Block Matching Algorithm), TSS (Three Step Search), and NTS (New Three-Step Search Algorithm) can be used to derive a motion vector. The motion vector can have a motion vector value in units of 1 / 2 or 1 / 4 pixels based on the interpolated pixel. The motion prediction unit can predict the current prediction unit by using different motion prediction methods. Various methods such as Skip Mode, Merge Mode, AMVP Mode, Intra Block Copy Mode, and Affine Mode can be used as motion prediction methods.

[0051] The intra prediction unit (125) can generate a prediction unit based on reference pixel information surrounding the current block, which is pixel information within the current picture. If the surrounding block of the current prediction unit is a block on which inter prediction has been performed and the reference pixel is a pixel on which inter prediction has been performed, the reference pixel included in the block on which inter prediction has been performed can be replaced and used with reference pixel information of the surrounding block on which intra prediction has been performed. That is, if the reference pixel is not available, the unavailable reference pixel information can be replaced and used with at least one reference pixel among the available reference pixels.

[0052] Additionally, a residual block containing residual value information, which is the difference between the prediction unit that performed the prediction based on the prediction unit generated in the prediction unit (120, 125) and the original block of the prediction unit, can be generated. The generated residual block can be input to the transformation unit (130).

[0053] In the transformation unit (130), the residual block including the residual value information of the prediction unit generated through the original block and the prediction unit (120, 125) can be transformed using a transformation method such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), or KLT. Whether to apply DCT, DST, or KLT to transform the residual block can be determined based on the intra prediction mode information of the prediction unit used to generate the residual block.

[0054] The quantization unit (135) can quantize values ​​converted to the frequency domain by the transformation unit (130). The quantization coefficients may vary depending on the block or the importance of the image. The values ​​produced by the quantization unit (135) can be provided to the dequantization unit (140) and the reordering unit (160).

[0055] The rearrangement unit (160) can perform rearrangement of coefficient values ​​for quantized residual values.

[0056] The rearrangement unit (160) can change a two-dimensional block-shaped coefficient into a one-dimensional vector form through a coefficient scanning method. For example, the rearrangement unit (160) can change the two-dimensional block-shaped coefficient into a one-dimensional vector form by scanning from the DC coefficient to the coefficient of the high-frequency region using a zig-zag scan method. Depending on the size of the transformation unit and the intra prediction mode, a vertical scan that scans the two-dimensional block-shaped coefficient in the column direction or a horizontal scan that scans the two-dimensional block-shaped coefficient in the row direction may be used instead of the zig-zag scan. That is, depending on the size of the transformation unit and the intra prediction mode, it is possible to determine which scan method among the zig-zag scan, the vertical scan, and the horizontal scan is to be used.

[0057] The entropy encoding unit (165) can generate a bitstream by performing entropy encoding based on the values ​​produced by the rearrangement unit (160). Entropy encoding can use various encoding methods such as, for example, Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC). In this regard, the entropy encoding unit (165) can encode residual value coefficient information of an encoding unit from the rearrangement unit (160) and the prediction units (120, 125). In addition, according to the present invention, it is possible to signal and transmit information indicating that motion information is derived and used on the decoder side and information on a technique used to derive motion information.

[0058] The inverse quantization unit (140) and the inverse transformation unit (145) inversely quantize the values ​​quantized in the quantization unit (135) and inversely transform the values ​​transformed in the transformation unit (130). The residual values ​​generated in the inverse quantization unit (140) and the inverse transformation unit (145) can be combined with the predicted prediction units predicted through the motion estimation unit, motion compensation unit, and intra prediction unit included in the prediction unit (120, 125) to generate a reconstructed block.

[0059] The filter unit (150) may include at least one of a deblocking filter, an offset correction unit, and an ALF (Adaptive Loop Filter). The deblocking filter may remove block distortion caused by boundaries between blocks in a restored picture. The offset correction unit may correct the offset of an image on which deblocking has been performed with respect to the original image on a pixel-by-pixel basis. In order to perform offset correction for a specific picture, a method may be used in which the pixels included in the image are divided into a certain number of regions, the regions to be offset are determined, and the offset is applied to the regions, or the offset is applied by considering edge information of each pixel. The ALF (Adaptive Loop Filtering) may be performed based on a value obtained by comparing the filtered restored image with the original image. After dividing the pixels included in the image into a predetermined group, one filter to be applied to the group is determined, and filtering may be performed differentially for each group.

[0060] The memory (155) can store a restored block or picture produced through the filter unit (150), and the stored restored block or picture can be provided to the prediction unit (120, 125) when performing inter prediction.

[0061] Figure 2 is a block diagram showing an image decoding device according to the present invention.

[0062] Referring to FIG. 2, the video image decoding device (200) may include an entropy decoding unit (210), a rearrangement unit (215), an inverse quantization unit (220), an inverse transformation unit (225), a prediction unit (230, 235), a filter unit (240), and a memory (245).

[0063] When a video bitstream is input to a video encoding device, the input bitstream can be decoded in the opposite procedure to that of the video encoding device.

[0064] The entropy decoding unit (210) can perform entropy decoding in a procedure opposite to that of the entropy encoding unit of the video image encoding device. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) can be applied in response to the method performed in the video image encoding device.

[0065] The entropy decoding unit (210) can decode information related to intra prediction and inter prediction performed in the image encoding device.

[0066] The reordering unit (215) can perform reordering based on the method by which the bitstream entropy-decoded by the entropy decoding unit (210) is reordered by the encoding unit. The coefficients expressed in the form of a one-dimensional vector can be reordered by restoring them back to coefficients in the form of a two-dimensional block.

[0067] The inverse quantization unit (220) can perform inverse quantization based on the quantization parameters provided by the image encoding device and the coefficient values ​​of the rearranged block.

[0068] The inverse transform unit (225) can perform inverse transform, i.e., inverse DCT, inverse DST, and inverse KLT, on the transforms, i.e., DCT, DST, and KLT, performed by the transform unit on the quantization result performed by the video image encoding device. The inverse transform can be performed based on the transmission unit determined by the video image encoding device. In the inverse transform unit (225) of the video image decoding device, a transform technique (e.g., DCT, DST, KLT) can be selectively performed according to a plurality of pieces of information, such as a prediction method, the size of the current block, and the prediction direction.

[0069] The prediction unit (230, 235) can generate a prediction block based on prediction block generation related information provided from the entropy decoding unit (210) and previously decoded block or picture information provided from the memory (245).

[0070] As described above, when performing intra prediction or intra-screen prediction in the same manner as the operation in a video encoding device, if the size of the prediction unit and the size of the transformation unit are the same, intra prediction for the prediction unit is performed based on the pixels on the left side of the prediction unit, the pixels on the upper left side, and the pixels on the upper side. However, when performing intra prediction, if the size of the prediction unit and the size of the transformation unit are different, intra prediction can be performed using a reference pixel based on the transformation unit. In addition, intra prediction using NxN division only for the minimum coding unit can be used.

[0071] The prediction unit (230, 235) may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit may receive various information such as prediction unit information input from the entropy decoding unit (210), prediction mode information of an intra prediction method, and motion prediction-related information of an inter prediction method, and may distinguish a prediction unit from a current encoding unit and determine whether the prediction unit performs inter prediction or intra prediction. On the other hand, if the video encoding device (100) does not transmit motion prediction-related information for the inter prediction, but instead transmits information indicating that motion information is to be derived and used on the decoder side and information on a technique used to derive motion information, the prediction unit determination unit determines whether the inter prediction unit (230) performs prediction based on the information transmitted from the video encoding device (100).

[0072] The inter prediction unit (230) can perform inter prediction on the current prediction unit based on information included in at least one picture among the previous picture or the subsequent picture of the current picture including the current prediction unit, using information required for inter prediction of the current prediction unit provided by the video image encoding device. In order to perform inter prediction, it can be determined based on the encoding unit whether the motion prediction method of the prediction unit included in the corresponding encoding unit is one of Skip Mode, Merge Mode, AMVP Mode, Intra Block Copy Mode, and Affine Mode.

[0073] The intra prediction unit (235) can generate a prediction block based on pixel information within the current picture. If the prediction unit is a prediction unit that has performed intra prediction, intra prediction can be performed based on intra prediction mode information of the prediction unit provided by the video image encoding device.

[0074] The intra prediction unit (235) may include an AIS (Adaptive Intra Smoothing) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is a unit that performs filtering on the reference pixels of the current block, and may determine whether to apply the filter according to the prediction mode of the current prediction unit and apply it. AIS filtering may be performed on the reference pixels of the current block using the prediction mode and AIS filter information of the prediction unit provided by the video image encoding device. If the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

[0075] The reference pixel interpolation unit can interpolate the reference pixel to generate a reference pixel of a pixel unit less than an integer value when the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on the pixel value interpolated from the reference pixel. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. The DC filter can generate a prediction block through filtering when the prediction mode of the current block is the DC mode.

[0076] The restored block or picture may be provided to a filter unit (240). The filter unit (240) may include a deblocking filter, an offset correction unit, and an ALF.

[0077] Information on whether a deblocking filter has been applied to a corresponding block or picture can be received from a video encoding device, and if a deblocking filter has been applied, information on whether a strong filter or a weak filter has been applied. The deblocking filter of the video decoding device can receive information related to the deblocking filter provided by the video encoding device, and the video decoding device can perform deblocking filtering on the corresponding block.

[0078] The offset correction unit can perform offset correction on the restored image based on the type of offset correction applied to the image during encoding and offset value information. ALF can be applied to the encoding unit based on information on whether ALF is applied and ALF coefficient information provided from the image encoding device. This ALF information can be provided by being included in a specific parameter set.

[0079] The memory (245) can store a restored picture or block so that it can be used as a reference picture or reference block, and can also provide the restored picture to an output unit.

[0080] FIG. 3 illustrates a prediction method performed by an image encoding / decoding device as an embodiment according to the present disclosure.

[0081] Referring to FIG. 3, at least two prediction blocks can be generated for the current block (S300).

[0082] At least two prediction blocks can be generated for a current block. For example, the current block can be divided into at least two partitions. For each of the at least two partitions, a prediction block can be generated from a reference picture of the corresponding partition. The division can be performed based on geometric division. The prediction based on geometric division according to the present disclosure can be any one of the inter modes pre-defined in a video encoding / decoding apparatus. Here, the prediction block can be a unidirectional prediction block or a bidirectional prediction block. The unidirectional prediction block can mean a prediction block in the L0 or L1 direction. The bidirectional prediction block can be generated through a weighted sum of the L0 and L1 prediction blocks. Through this, at least two prediction blocks can be generated for the current block.

[0083] Referring to FIG. 3, blending can be performed based on a predetermined mask for at least two prediction blocks to generate a final prediction block of the current block (S310).

[0084] The above mask, which has the same size as the current block, can define weights for each sample location within the prediction block. The mask can be defined for each of the at least two prediction blocks. The method for generating / determining the mask will be described in detail with reference to FIGS. 4 to 7.

[0085] FIG. 4 illustrates a mask-based blending method for multiple prediction blocks as an embodiment according to the present disclosure.

[0086] For at least two prediction blocks for the current block, blending can be performed based on a predetermined mask to generate a final prediction block of the current block.

[0087] The mask may be determined based on geometric segmentation information. The geometric segmentation information may be information related to one or more segmentation lines for dividing the current block into at least two partitions. For example, the geometric segmentation information may include at least one of the distance between the segmentation line and the center of the current block or the angle of a line segment perpendicular to the segmentation line. The geometric segmentation information may be signaled via a bitstream.

[0088] The mask weights can be calculated based on at least one of the aforementioned angles or distances. The angles and / or distances can be quantized and defined in the form of indices. For example, for functions such as cos() and sin(), predefined values ​​based on the indices can be used based on a table.

[0089] Alternatively, the geometric segmentation information may be an index. Based on the index, a mask predefined identically to the image encoding / decoding device may be used. The image encoding / decoding device may define multiple mask candidates, and any one of the multiple mask candidates may be selectively used based on the index.

[0090] The geometric segmentation information may include a flag indicating whether an inverted mask is used. For example, if the flag has a first value, this may indicate that an inverted mask is used. In this case, a mask derived by inverting the mask selected by the index (i.e., an inverted mask) may be used. On the other hand, if the flag has a second value, this may indicate that an inverted mask is not used. In this case, a mask selected by the index may be used. The flag may be encoded in the bitstream based on a determination of whether the current block uses an inverted mask, and the encoded flag may be signaled to an image decoding device via the bitstream.

[0091] Referring to Fig. 4, a curved edge may exist within a block, similar to the original block. In this case, a large amount of error may occur in the blending region of the final predicted block. Therefore, by using an adaptive mask based on the texture of the current block, the prediction performance of the final predicted block can be improved. To use an adaptive mask, an optimal mask is determined / generated in a video encoding device and information specifying this mask is transmitted to an video decoding device, so that the same mask can be used in the video encoding / decoding device.

[0092] Alternatively, the mask can be generated based on at least two prediction blocks.

[0093] For example, a mask may be generated based on a differential block between at least two prediction blocks. At this time, at least two weights for blending may be derived based on the differential sample value(s) within the differential block. Here, the at least two weights may be different values ​​or the same values. To this end, a table defining corresponding weights for each differential sample value may be used. That is, weights corresponding to differential sample values ​​within the differential block may be derived from the table. Here, the weights may include at least two weights applied to at least two prediction blocks. Alternatively, the at least two weights may be derived based on a mapping function according to the differential sample values.

[0094] Alternatively, the mask can be generated based on a ratio block between at least two prediction blocks.

[0095] A ratio block can be derived by dividing a prediction sample value in one of at least two prediction blocks by a prediction sample value at the same position in another prediction block. At least two weights for blending can be derived based on the sample value(s) in the ratio block.

[0096] Figure 5 illustrates a mask generation method according to the present disclosure.

[0097] Multiple prediction blocks can be used to generate a final prediction block for the current block. Each prediction block can be generated based on any one of inter mode, intra mode, or intra block copy (IBC) mode. For example, the multiple prediction blocks can be generated using the same prediction method (i.e., any one of inter mode, intra mode, or intra block copy mode). Alternatively, one of the multiple prediction blocks can be generated based on inter mode, and the other can be generated based on either intra mode or intra block copy mode. Alternatively, one of the multiple prediction blocks can be generated based on intra mode, and the other can be generated based on either inter mode or intra block copy mode.

[0098] A final prediction block can be generated by weighting multiple prediction blocks for the current block.

[0099] The weight for the above weighted sum may be determined based on at least one of the ratio of neighboring blocks to which the intra mode is applied among neighboring blocks adjacent to the current block or the difference between the POC (picture order count) of the current picture to which the current block belongs and the POC of the reference picture.

[0100] A final prediction block can be generated using multiple prediction block-specific masks.

[0101] The above mask can be derived based on information signaled in predetermined units as described above. Here, the predetermined unit can be a picture, a slice, a coding tree unit row (CTU row), a CTU, or a coding block.

[0102] Alternatively, the mask may be derived based on the differential block between the at least two prediction blocks described above. For example, if the differential sample value in the differential block is 0, the weight of the corresponding sample position may be derived as the average weight. On the other hand, a larger differential sample value in the differential block may be used as a larger weight. In this case, one of the at least two prediction blocks may be selected based on the sign of the differential sample value. A flag for the selection may be separately signaled. The differential sample value may be replaced with the absolute value of the differential sample value, and this may be equally applied to the embodiments described below.

[0103] Alternatively, a value mapped to the differential sample value may be derived based on a linear function, and a mask may be derived based on the derived value. The linear function according to the present disclosure may be defined as at least one of a weight multiplied by the differential sample value or an offset added / subtracted to / from the differential sample value. That is, deriving a linear function may mean deriving at least one of the weight or the offset. This may be applied to the embodiments described below in the same sense. At this time, when the differential sample value is greater than a predetermined threshold, the differential sample value may be clipped to a value based on the bit depth of the mask. On the other hand, when the differential sample value is less than or equal to the threshold, mapping may be performed based on a linear function different from the previous linear function. At this time, each linear function may be identically predefined in the image encoding / decoding device.

[0104] The image encoding device can transmit linear function information to the image decoding device in predetermined units. Here, the predetermined units are as described above.

[0105] Alternatively, a linear function may be derived based on the prediction block(s) for the neighboring block(s) of the current block, and a mask may be derived based on this. For example, if a neighboring block uses two prediction blocks, one or more linear functions may be obtained for the current block based on the reconstructed neighboring block and the two prediction blocks. Based on the obtained linear functions, a value mapped to the differential sample value for the current block may be derived, and a mask may be derived based on this.

[0106] Alternatively, a predetermined representative value may be derived from the prediction block of the current block, and a linear function may be derived based on the derived representative value. For example, the linear function may be derived based on the average value of any one of at least two prediction blocks for the current block. Based on the average value of any one of at least two prediction blocks for the current block, any one of a plurality of linear function candidates may be selected. The average value may be defined as the average of all or part of the prediction sample values ​​belonging to the prediction block.

[0107] Alternatively, a linear function can be derived based on representative values ​​of pre-restored sample values ​​around the current block. One of multiple linear function candidates can be selected based on representative values ​​of pre-restored sample values ​​around the current block.

[0108] Figure 6 illustrates a mask generation method according to the present disclosure.

[0109] At least two prediction blocks according to the present disclosure may be generated based on an intra mode. However, this is not limited to this, and as described above, the at least two prediction blocks may be generated based on different prediction methods.

[0110] Each prediction block may be generated based on a non-directional mode (e.g., planar mode, DC mode), a directional mode, IBC, or Intra-Template Matching Prediction (IntraTMP). Alternatively, the prediction samples of each prediction block may have position-based intra-prediction sample filtering (PDPC) applied.

[0111] A mask can be derived based on differential sample value(s) within a differential block for the current block, and the mask can be applied to at least two prediction blocks for the current block to generate a final prediction block.

[0112] At this time, a linear function (i.e., a weight and / or offset of the linear function) can be derived based on at least one of the maximum or minimum values ​​among the differential sample values ​​and the bit depth of the mask. The sign of the weight and / or offset of the linear function can be signaled in a predetermined unit.

[0113] The sign of the weight of the linear function can be predicted based on the previously restored reference sample value(s) for the current block, and the predicted sign can be used as is.

[0114] Alternatively, the sign of the gradient of a linear function can be derived based on the reconstructed sample value(s) surrounding the current block and the predicted block of the current block. For example, the sign of the gradient of a linear function can be derived based on the first derivative in the horizontal or vertical direction.

[0115] Alternatively, the sign of the gradient of the linear function can be derived based on the continuity between the pre-reconstructed sample value(s) surrounding the current block and the predicted block of the current block. Here, continuity can be determined based on the difference between the predicted sample located at the boundary of the current block and the pre-reconstructed surrounding samples. That is, depending on the sign of the gradient, more prediction candidates can be generated than with conventional intra prediction, and high encoding efficiency can be provided by adaptively using more appropriate prediction blocks.

[0116] Figure 7 illustrates a mask generation method according to the present disclosure.

[0117] According to FIG. 7, differential sample value(s) can be calculated for some regions within at least two prediction blocks for the current block, and a mask can be generated based on the differential sample value(s). Based on the generated mask, at least two prediction blocks can be blended to generate a final prediction block.

[0118] Referring to Fig. 7, differential sample values ​​can be calculated only for some areas (Difference areas) within the two prediction blocks (Predictor 0, Predictor 1). The above-mentioned areas (Difference areas) can be determined based on one or more straight lines. Hereinafter, the areas where differential sample values ​​are calculated will be referred to as "difference areas."

[0119] Here, information about a straight line can be signaled in a predetermined unit. The predetermined unit can be a picture, a slice, a coding tree unit (CTU) row, a CTU, or a coding block. The information about a straight line can include at least one of a weight (or slope) or an offset (or intercept) representing a linear function. Alternatively, the information about a straight line can include at least one of a slope or a distance from the center of the current block. Alternatively, the information about a straight line can be an index indicating a straight line parameter from a table defining multiple straight line parameters.

[0120] If the difference area is a straight-line-based area, the first area (1) as shown in Fig. 7 st area) and the second area (2 nd area) can be distinguished. Flags can be additionally signaled to derive masks for the two areas.

[0121] For example, if the value of the flag is 0, a mask may be derived that causes the first region to use the first prediction block (Predictor 0), and a mask may be derived that causes the second region to use the second prediction block (Predictor 1). Conversely, if the value of the flag is 1, a mask may be derived that causes the first region to use the second prediction block, and a mask may be derived that causes the second region to use the first prediction block.

[0122] Alternatively, the same prediction block can be used for both regions, with blending performed only on the difference region to generate the final prediction block. In this case, the decision on which prediction block to use for each region can be made based on a specific flag.

[0123] For example, if the value of the flag is 0, a mask may be derived that uses the first prediction block in the first region and the second region, and a mask based on the differential sample value may be derived in the difference region. Conversely, if the value of the flag is 1, a mask may be derived that uses the second prediction block in the first region and the second region, and a mask based on the differential sample value may be derived for the difference region.

[0124] In the mask for the difference region, the slope of the linear function may be determined based on at least one of the features of the first region and the difference region, or the features of the second region and the difference region. Here, the features of the first region and the difference region may refer to the continuity between the first region and the difference region. Similarly, the features of the second region and the difference region may refer to the continuity between the second region and the difference region. The meaning of continuity is as described above.

[0125] FIG. 8 illustrates a parameter-based prediction method as an example according to the present disclosure.

[0126] Referring to Fig. 8, the prediction mode of the current block can be determined (S800).

[0127] The prediction mode of the current block may be determined as one of a plurality of prediction modes predefined in the video encoding device and the video decoding device. The plurality of prediction modes may include at least one of an intra mode, an inter-color prediction mode, an inter mode, or an IBC mode.

[0128] Referring to Fig. 8, parameters can be derived based on the prediction mode of the current block (S810).

[0129] The parameters according to the present disclosure may refer to parameters of a mapping function, offset parameters, filter parameters, or parameters for illumination compensation. The method of deriving parameters according to the prediction mode will be described in detail with reference to FIGS. 9 to 11.

[0130] Referring to FIG. 8, a prediction block of the current block can be generated based on the reference block and parameters of the current block (S820).

[0131] A reference block according to the present disclosure may be a block of a different component type from the current block. For example, if the current block is a chrominance component block, the reference block may be a luminance component block corresponding to the current block. If the current block is a Cr component block, the reference block may be a Cb component block corresponding to the current block.

[0132] Alternatively, the reference block may belong to a previously restored region within the current picture to which the current block belongs. Alternatively, the reference block may belong to a different picture from the current block.

[0133] The parameters can be applied to samples of the reference block to derive prediction samples of the current block. The parameters can be applied to all samples belonging to the reference block, or only to some samples within the reference block. The reference block can be divided into at least two regions, and different parameters can be applied to each region.

[0134] FIG. 9 illustrates a parameter-based prediction method in color inter-prediction of intra prediction as an embodiment according to the present disclosure.

[0135] Color inter-prediction according to the present disclosure can predict at least one of a second block or a third block based on a pre-reconstructed or pre-predicted first block, as illustrated in FIG. 9. Alternatively, the third block can be predicted based on a pre-reconstructed or pre-predicted second block.

[0136] A mapping function having color correlation may be used to generate prediction blocks for the second block and / or the third block. The parameters of the mapping function used for the current block may be transmitted from the image encoding device to the image decoding device. Here, the parameters of the mapping function may include at least one of a weight multiplied by the sample value of the first block or one or more offsets added / subtracted from the samples of the first block. The same meaning may be interpreted in the embodiments described below.

[0137] Alternatively, any one of a plurality of mapping functions stored in the form of a pre-defined table or list in the image encoding device and the image decoding device may be selectively used.

[0138] Alternatively, the parameters of the mapping function may be derived based on the correlation between the pre-reconstructed sample value(s) around the first block and the pre-reconstructed sample value(s) around the second block, and prediction may be performed based on the derived parameters of the mapping function. In addition, the parameters of the mapping function may be derived based on the correlation between the pre-reconstructed sample value(s) around the first block and the pre-reconstructed sample value(s) around the third block, and prediction may be performed based on the derived parameters. The correlation will be examined in detail with reference to FIGS. 12 and 14.

[0139] FIG. 10 illustrates a parameter-based prediction method in intra block copy (IBC) mode as an embodiment according to the present disclosure.

[0140] The IBC mode can derive a prediction block of the current block based on a reference block at a position moved by a predetermined block vector (BV) from the current block, as illustrated in FIG. 10.

[0141] A reference block specified by a block vector may be derived as a prediction block of the current block, or a corrected reference block may be derived as a prediction block of the current block.

[0142] For example, IBC mode can be used when identical patterns exist within the reference block and the current block. In this case, illumination may vary depending on the location during image acquisition. In this case, using a mapping function, illumination offset, or filtering to compensate for the reference block can improve prediction performance.

[0143] Here, the parameters of the mapping function, the illumination offset, or the filter can be derived from the image encoding device and transmitted to the image decoding device.

[0144] Alternatively, an index specifying one or more parameters from a table defining a plurality of pre-defined parameters may be encoded in the image encoding device and transmitted to the image decoding device.

[0145] Alternatively, parameters can be derived based on the correlation between the pre-reconstructed sample(s) surrounding the current block and the pre-reconstructed sample(s) surrounding the reference block. This correlation will be further examined with reference to FIGS. 12 and 14.

[0146] Figure 11 illustrates a parameter-based prediction method in inter mode.

[0147] Inter mode, as illustrated in Fig. 11, is a mode for deriving a prediction block of the current block based on a reference block at a position shifted by a motion vector (MV) from a block at the same position as the current block within the reference picture. At this time, if the accuracy (or precision) of the MV is smaller than an integer unit, the prediction block can be derived by applying an interpolation filter to samples at integer positions within the reference block. Here, the interpolation filter can be determined based on the position according to the accuracy of the MV.

[0148] Although the reference block with the smallest error is selected in the video encoding device, errors due to illumination variations between pictures may still remain. Therefore, prediction performance can be improved by additionally performing illumination compensation on the prediction block derived through interpolation.

[0149] For the above lighting compensation, a mapping function, an offset, a filter, etc. may be used. The parameters of the mapping function, the offset, or the filter may be calculated in an image encoding device and transmitted to an image decoding device.

[0150] Alternatively, an index specifying one or more parameters from a table defining a plurality of pre-defined parameters may be encoded in the image encoding device and transmitted to the image decoding device.

[0151] Alternatively, parameters can be derived based on the correlation between the pre-reconstructed sample(s) surrounding the current block and the pre-reconstructed sample(s) surrounding the reference block. This correlation will be further examined with reference to FIGS. 12 and 14.

[0152] FIG. 12 is an example according to the present disclosure, illustrating a relationship between two different areas.

[0153] The correlation between the pre-restored sample(s) around the current block and the pre-restored sample(s) around the reference block can be expressed as a two-dimensional dot graph as shown in Fig. 12, which can be approximated by a linear straight line.

[0154] For example, the set of pre-reconstructed samples around the current block is called the current template, and the set of pre-reconstructed samples around the reference block is called the reference template. A dot graph can be derived using the sample values ​​belonging to each of the current template and the reference template.

[0155] In a specific block, the relationship between the current template and the reference template can be represented by a linear line as in Fig. 12. In Fig. 12, the first component may mean the reference template, and the second component may mean the current template.

[0156] In this case, a linear mapping function can be derived to express the association using the sample values ​​of the current and reference templates. Here, the linear mapping function can be expressed in terms of parameters such as a slope and an intercept, and the slope and intercept can be derived using the sample values ​​of the two templates.

[0157] At least one of the slope or intercept of the linear mapping function can be derived through ordinary regression analysis.

[0158] Alternatively, parameters can be derived based on the minimum value of the first component and the corresponding sample value of the second component, and the maximum value of the first component and the corresponding sample value of the second component.

[0159] Alternatively, the parameters may be derived based on the average of relatively small sample values ​​of the first component and the corresponding average of sample values ​​of the second component, and the average of relatively large sample values ​​of the first component and the corresponding average of sample values ​​of the second component.

[0160] FIG. 13 is an example according to the present disclosure, illustrating multiple associations for two different areas.

[0161] The correlation between the pre-restored sample(s) around the current block and the pre-restored sample(s) around the reference block can be expressed as a two-dimensional dot graph as shown in Fig. 13, and can be classified into multiple groups based on the characteristics of the samples.

[0162] For example, if the samples of the current template and the reference template are classified into two groups as illustrated in FIG. 13, a mapping function, offset, or filter that represents a separate association for each group may be used.

[0163] The K-means algorithm can be used to classify the above samples. Alternatively, the samples can be classified into two groups based on the average value of the first component. For example, samples smaller than the average value of the first component can be classified into the first group, and samples greater than or equal to the average value of the first component can be classified into the second group. Alternatively, the samples can be classified into two groups based on the average value of the second component. Alternatively, the samples can be classified into two groups based on the weighted average value of the first or second components.

[0164] Parameters can be derived for each group using a linear regression algorithm, and predictions can be made based on the parameters.

[0165] FIG. 14 is an example according to the present disclosure, showing nonlinear correlations for two different areas.

[0166] The correlation between the pre-restored sample(s) around the current block and the pre-restored sample(s) around the reference block can be expressed as a two-dimensional dot graph as shown in Fig. 14, which can be approximated by a curve.

[0167] The above curve can be expressed as a polynomial with the term "Cf(x)" added to the existing linear function. In this case, parameters can be derived through polynomial regression analysis, and predictions can be made based on these parameters. Here, C is a parameter derived through polynomial regression analysis, and f(x) can be a predefined function depending on x.

[0168] For example, f(x) could be a function that squares x.

[0169] Alternatively, f(x) could be a function that squares x and divides it by the maximum value of the current bit depth.

[0170] Alternatively, f(x) can mean a new value derived based on the value of x and its surrounding sample(s). For example, it can mean the first derivative in the horizontal direction. That is, since the curve changes depending on the first derivative in the horizontal direction, it can be approximated by various curves, and adaptive mapping is possible depending on the texture of the image. Alternatively, f(x) can mean the first derivative in the vertical direction. Alternatively, f(x) can mean the first derivative in the diagonal direction. Alternatively, f(x) can mean the sum of the first derivatives in the horizontal and vertical directions. Alternatively, f(x) can mean an activity such as the sum of the first derivatives in the horizontal, vertical, and diagonal directions. Here, the derivative can use a method such as the Sobel operation or differencing. Alternatively, f(x) can mean a directionality derived based on the derivatives in the horizontal and vertical directions. Alternatively, f(x) can mean the sum of directionality and activity derived based on the various differential values ​​mentioned above.

[0171] The various embodiments of the present disclosure are not intended to list all possible combinations but rather to illustrate representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combinations of two or more.

[0172] Additionally, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of hardware implementation, the embodiments may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general processors, controllers, microcontrollers, microprocessors, etc.

[0173] The scope of the present disclosure includes software or machine-executable instructions (e.g., operating systems, applications, firmware, programs, etc.) that cause operations according to the methods of various embodiments to be executed on a device or a computer, and a non-transitory computer-readable medium having such software or instructions stored thereon and executable on the device or computer.

Claims

1. A step of generating at least two prediction blocks for the current block; and A step of generating a final prediction block of the current block by weighting at least two prediction blocks based on a predetermined mask, Each of the above at least two prediction blocks is a unidirectional or bidirectional prediction block, A method for decoding an image, wherein the above bidirectional prediction block is generated based on a weighted sum of an L0 prediction block and an L1 prediction block.

2. In paragraph 1, An image decoding method, wherein the above mask is determined based on geometric segmentation information.

3. In paragraph 2, An image decoding method wherein the above geometric segmentation information specifies one of a plurality of mask candidates pre-defined in an image decoding device.

4. In paragraph 2, An image decoding method, wherein the geometric segmentation information includes a flag indicating whether an inverted mask is used.

5. In paragraph 1, A method for decoding an image, wherein the mask is generated based on a difference block between at least two prediction blocks.

6. In paragraph 1, The above mask is generated based on a ratio block between the at least two prediction blocks, A method for decoding an image, wherein the above ratio block is derived by dividing a sample value belonging to one of the at least two prediction blocks by a sample value at the same position belonging to the other one.

7. In paragraph 1, A method for decoding an image, wherein each of the at least two prediction blocks is generated based on one of an inter mode, an intra mode, or an intra block copy mode.

8. In paragraph 7, A method for decoding an image, wherein the weights for the above weighted sum are determined based on at least one of the ratio of surrounding blocks to which the intra mode is applied or the POC difference between the current picture to which the current block belongs and the reference picture.

9. A step of generating at least two prediction blocks for the current block; and A step of generating a final prediction block of the current block by weighting at least two prediction blocks based on a predetermined mask, Each of the above at least two prediction blocks is a unidirectional or bidirectional prediction block, A video encoding method, wherein the above bidirectional prediction block is generated based on a weighted sum of an L0 prediction block and an L1 prediction block.

10. A step of generating at least two prediction blocks for the current block; A step of generating a final prediction block of the current block by weighting and combining the at least two prediction blocks based on a predetermined mask; A step of deriving a residual block of the current block based on the final prediction block; A step of encoding the residual block to generate a bitstream; and Including a step of transmitting the above bitstream, Each of the above at least two prediction blocks is a unidirectional or bidirectional prediction block, A bitstream transmission method, wherein the above bidirectional prediction block is generated based on a weighted sum of an L0 prediction block and an L1 prediction block.

Images