Method for inter prediction of chroma components using dual prediction

Abstract

A method of inter prediction of chroma components using dual prediction is disclosed. In this embodiment, an image decoding device generates a dichroma prediction block for a current chroma block. An image decoding device derives a weight of a current chroma block by using a first luma prediction block, a second luma prediction block, and a final luma reconstruction block of a luma block corresponding to the current chroma block. The image decoding device applies the weight to the dichromaticity prediction block and generates a final chromaticity prediction block of the current chromaticity block.

Description

Technical Field

[0001] The present invention relates to a method of inter-predicting chrominance components by utilizing bi-prediction. Background Art

[0002] The statements in this section merely provide background information related to the present invention and may not necessarily constitute prior art.

[0003] Since video data has a large amount of data compared to audio data or still image data, video data requires a large amount of hardware resources (including memory) to store or transmit the video data without being compressed.

[0004] Accordingly, the encoder is generally used to compress and store or transmit video data. The decoder receives the compressed video data, decompresses the received compressed video data, and plays the decompressed video data. Video compression technologies include H.264 / Advanced Video Codec (AVC), High Efficiency Video Coding (HEVC), and Versatile Video Codec (VVC), which improves the coding efficiency of HEVC by about 30% or more.

[0005] However, as image size, resolution and frame rate gradually increase, the amount of data to be encoded will also increase. Accordingly, new compression technologies are needed that provide higher encoding and decoding efficiency and improved image enhancement effects than existing compression technologies.

[0006] In VVC technology, when performing bi-prediction, the advanced motion vector prediction (AMVP) mode of inter-frame prediction assigns weights to two predicted signals derived from two motion vectors at the coding unit (CU) level. This method is called Bi-prediction with CU-level Weight (BCW). In BCW, the decoder averages the two predicted signals based on the weights to generate the final predicted signal. Therefore, it is necessary to consider the effective setting of these weights for the chroma components in particular to improve video encoding and decoding efficiency and enhance video quality. Summary of the invention

[0007] Technical issues

[0008] The present invention is dedicated to providing a video coding method and apparatus, which calculates weights on the decoder side using previously reconstructed luminance prediction blocks or template matching in inter-frame prediction of chrominance components. The video coding method and apparatus apply the calculated weights to the chrominance dual prediction blocks to generate a final prediction block.

[0009] Technical Solution

[0010] At least one aspect of the present invention provides a method for predicting a current chroma block performed by a video decoding device. The method includes generating a chroma dual prediction block including a first chroma prediction block and a second chroma prediction block for the current chroma block. The method also includes deriving a weight of the current chroma block by using a first luminance prediction block, a second luminance prediction block, and a final luminance reconstruction block of a luminance block corresponding to the current chroma block. Here, the weight includes a first chroma weight and a second chroma weight. The method also includes generating a final chroma prediction block of the current chroma block by applying the weight to the chroma dual prediction block.

[0011] Another aspect of the present invention provides a method for predicting a current chroma block performed by a video encoding device. The method includes generating a chroma dual prediction block including a first chroma prediction block and a second chroma prediction block for the current chroma block. The method also includes deriving a weight of the current chroma block by using a first luminance prediction block, a second luminance prediction block, and a final luminance reconstruction block of a luminance block corresponding to the current chroma block. Here, the weight includes a first chroma weight and a second chroma weight. The method also includes generating a first final chroma prediction block of the current chroma block by applying the weight to the chroma dual prediction block.

[0012] Another aspect of the present invention provides a computer-readable recording medium that stores a bitstream generated by a video encoding method. The video encoding method includes generating a chroma dual prediction block including a first chroma prediction block and a second chroma prediction block for a current chroma block. The video encoding method also includes deriving a weight of the current chroma block by using a first luminance prediction block, a second luminance prediction block, and a final luminance reconstruction block of a luminance block corresponding to the current chroma block. Here, the weight includes a first chroma weight and a second chroma weight. The video encoding method also includes generating a final chroma prediction block of the current chroma block by applying the weight to the chroma dual prediction block.

[0013] Beneficial Effects

[0014] As described above, the present invention provides such a video coding method and apparatus, which calculates weights on the decoder side by using a previously reconstructed luminance prediction block or template matching in inter-frame prediction of chrominance components. The video coding method and apparatus apply the calculated weights to the chrominance dual prediction block to generate a final prediction block. Therefore, the video coding method and apparatus improve the video coding efficiency and enhance the video quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a block diagram of a video encoding device that can implement the technology of the present invention.

[0016] Figure 2A method of partitioning blocks using a quadtree plus binary tree ternary tree (QTBTTT) structure is shown.

[0017] Figure 3a and Figure 3b Multiple intra prediction modes are shown including a wide-angle intra prediction mode.

[0018] Figure 4 Shows the neighboring blocks of the current block.

[0019] Figure 5 is a block diagram of a video decoding device that can implement the technology of the present invention.

[0020] Figure 6 is a schematic diagram showing weighted chrominance prediction.

[0021] Figure 7 is a schematic diagram illustrating the use of a luma block in predicting a chroma block according to at least one embodiment of the present invention.

[0022] Figure 8 is a schematic diagram illustrating calculation of differences between a luma block and adjacent luma samples according to at least one embodiment of the present invention.

[0023] Fig. 9 is a schematic diagram illustrating template matching according to at least one embodiment of the present invention.

[0024] Fig.10 is a flowchart of a method for predicting a current chrominance block performed by a video encoding device according to at least one embodiment of the present invention.

[0025] Fig.11 is a flowchart of a method for predicting a current chrominance block performed by a video decoding device according to at least one embodiment of the present invention. DETAILED DESCRIPTION

[0026] Hereinafter, some embodiments of the present invention will be described in detail with reference to the attached illustrative drawings. In the following description, the same reference numerals represent the same elements, although the elements are shown in different drawings. In addition, in the following description of some embodiments, when it is considered that the detailed description of related known components and functions obscures the subject matter of the present invention, the detailed description of related known components and functions may be omitted for the sake of clarity and brevity.

[0027] Figure 1 is a block diagram of a video encoding device that can implement the technology of the present invention. Figure 1 A video encoding device and components of the device are described in detail.

[0028] The encoding device may include: an image divider 110, a predictor 120, a subtractor 130, a transformer 140, a quantizer 145, a rearrangement unit 150, an entropy encoder 155, an inverse quantizer 160, an inverse transformer 165, an adder 170, a loop filtering unit 180 and a memory 190.

[0029] Each component of the encoding device can be implemented as hardware or software, or as a combination of hardware and software. In addition, the function of each component can be implemented as software, and the microprocessor can also be implemented to execute the function of the software corresponding to each component.

[0030] A video consists of one or more sequences including multiple images. Each image is divided into multiple regions, and encoding is performed on each region. For example, an image is divided into one or more tiles or / and slices. Here, one or more tiles can be defined as a tile group. Each tile or / and slice is divided into one or more coding tree units (coding tree unit, CTU). In addition, each CTU is divided into one or more coding units (coding unit, CU) through a tree structure. The information applied to each coding unit (CU) is encoded as the syntax of the CU, and the information commonly applied to the CU included in a CTU is encoded as the syntax of the CTU. In addition, the information commonly applied to all blocks in a slice is encoded as the syntax of the slice header, and the information applied to all blocks constituting one or more images is encoded as a picture parameter set (Picture Parameter Set, PPS) or a picture header. In addition, the information commonly referenced by multiple images is encoded as a sequence parameter set (Sequence Parameter Set, SPS). In addition, the information commonly referenced by one or more SPSs is encoded as a video parameter set (Video Parameter Set, VPS). In addition, information commonly applied to one tile or tile group may also be encoded as syntax of the tile or tile group header.The syntax included in the SPS, PPS, slice header, tile or tile group header may be referred to as a high-level syntax.

[0031] The image divider 110 determines the size of a coding tree unit (CTU). Information on the size of the CTU (CTU size) is encoded as a syntax of an SPS or a PPS and transmitted to a video decoding device.

[0032] The image divider 110 divides each image constituting a video into a plurality of coding tree units (CTUs) having a predetermined size, and then recursively divides the CTUs by using a tree structure. A leaf node in the tree structure becomes a coding unit (CU), which is a basic unit of coding.

[0033] The tree structure can be a quadtree (quadtree, QT), in which a higher node (or parent node) is split into four lower nodes (or child nodes) of the same size. The tree structure can also be a binary tree (binarytree, BT), in which a higher node is split into two lower nodes. The tree structure can also be a ternary tree (ternarytree, TT), in which a higher node is split into three lower nodes at a ratio of 1:2:1. The tree structure can also be a structure in which two or more structures in the QT structure, BT structure and TT structure are mixed. For example, a quadtree plus binary tree (quadtree plus binarytree, QTBT) structure can be used, or a quadtree plus binary tree ternary tree (quadtree plus binarytreeternarytree, QTBTTT) structure can be used. Here, a binary tree ternary tree (BTTT) is added to the tree structure to be called a multiple-type tree (multiple-type tree, MTT).

[0034] Figure 2 is a schematic diagram for describing a method of partitioning a block by using a QTBTTT structure.

[0035] like Figure 2 As shown, the CTU can be first split into a QT structure. The quadtree splitting can be recursive until the size of the split block reaches the minimum block size (MinQTSize) of the leaf node allowed in QT. The first flag (QT_split_flag) indicating whether each node of the QT structure is split into four nodes of the lower layer is encoded by the entropy encoder 155 and notified to the video decoding device with a signal. When the leaf node of QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in BT, the leaf node can be further split into at least one of a BT structure or a TT structure. There may be multiple splitting directions in the BT structure and / or the TT structure. For example, there may be two directions, namely, a direction for horizontally splitting blocks of the corresponding node and a direction for vertically splitting blocks of the corresponding node. As Figure 2 As shown, when MTT splitting starts, the entropy encoder 155 encodes a second flag (mtt_split_flag) indicating whether the node is split, and a flag indicating the splitting direction (vertical or horizontal) and / or a flag indicating the splitting type (binary or trifurcated) when the node is split, and notifies the video decoding device of the same by signal.

[0036] Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node is split into four nodes of the lower layer, the CU split flag (split_cu_flag) indicating whether the node is split may also be encoded. When the value of the CU split flag (split_cu_flag) indicates that each node is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a CU, which is the basic unit of encoding. When the value of the CU split flag (split_cu_flag) indicates that each node is split, the video encoding device first starts encoding the first flag in the above scheme.

[0037] When QTBT is used as another example of a tree structure, there may be two types, namely, a type in which the block of the corresponding node is horizontally split into two blocks of the same size (i.e., symmetrical horizontal splitting) and a type in which the block of the corresponding node is vertically split into two blocks of the same size (i.e., symmetrical vertical splitting). The entropy encoder 155 encodes a split flag (split_flag) indicating whether each node of the BT structure is split into blocks of the lower layer and split type information indicating the split type, and transmits them to the video decoding device. On the other hand, there may be another type in which the block of the corresponding node is split into two blocks that are asymmetric to each other. The asymmetric form may include a form in which the block of the corresponding node is split into two rectangular blocks with a size ratio of 1:3, or may also include a form in which the block of the corresponding node is split in a diagonal direction.

[0038] The CU may have various sizes according to the QTBT or QTBTTT partitioned from the CTU. Hereinafter, the block corresponding to the CU to be encoded or decoded (i.e., the leaf node of the QTBTTT) is referred to as the "current block". When QTBTTT partitioning is adopted, the shape of the current block may also be a rectangular shape in addition to a square shape.

[0039] The predictor 120 predicts the current block to generate a predicted block. The predictor 120 includes an intra predictor 122 and an inter predictor 124.

[0040] Typically, each of the current blocks in an image may be predictively encoded. Typically, the prediction of the current block may be performed by utilizing an intra-prediction technique (which utilizes data from an image including the current block) or an inter-prediction technique (which utilizes data from an image encoded prior to the image including the current block). Inter-prediction includes both unidirectional prediction and bidirectional prediction.

[0041] The intra-frame predictor 122 predicts pixels in the current block by using pixels (reference pixels) located adjacent to the current block in the current image including the current block. There are multiple intra-frame prediction modes according to the prediction direction. For example, Figure 3aAs shown, the plurality of intra prediction modes may include two non-directional modes including a planar mode and a DC mode, and may include 65 directional modes. Neighboring pixels and algorithm equations to be used are defined differently according to each prediction mode.

[0042] In order to perform efficient directional prediction for a current block with a rectangular shape, it is possible to additionally use Figure 3b The directional modes are shown in the dashed arrows (#67 to #80, intra-frame prediction modes #-1 to #-14). The directional modes can be called "wide angle intra-frame prediction modes". Figure 3b In the figure, the arrows indicate the corresponding reference samples used for prediction, rather than representing the prediction direction. The prediction direction is opposite to the direction indicated by the arrow. When the current block has a rectangular shape, the wide-angle intra prediction mode is a mode in which prediction is performed in a direction opposite to the specific direction mode without additional bit transmission. In this case, in the wide-angle intra prediction mode, some wide-angle intra prediction modes available for the current block can be determined by the ratio of the width to the height of the current block having a rectangular shape. For example, when the current block has a rectangular shape with a height less than the width, a wide-angle intra prediction mode (intra prediction modes #67 to #80) with an angle less than 45 degrees is available. When the current block has a rectangular shape with a width greater than the height, a wide-angle intra prediction mode with an angle greater than -135 degrees is available.

[0043] The intra-frame predictor 122 may determine an intra-frame prediction to be used for encoding the current block. In some examples, the intra-frame predictor 122 may encode the current block by utilizing multiple intra-frame prediction modes, and may also select an appropriate intra-frame prediction mode to be used from a test mode. For example, the intra-frame predictor 122 may calculate a rate-distortion value by utilizing a rate-distortion analysis of multiple tested intra-frame prediction modes, and may also select an intra-frame prediction mode having the best rate-distortion characteristic in the test mode.

[0044] The intra-frame predictor 122 selects an intra-frame prediction mode among a plurality of intra-frame prediction modes and predicts the current block by using adjacent pixels (reference pixels) and an algorithm equation determined according to the selected intra-frame prediction mode. Information about the selected intra-frame prediction mode is encoded by the entropy encoder 155 and transmitted to the video decoding device.

[0045] The inter-frame predictor 124 generates a prediction block for the current block by using a motion compensation process. The inter-frame predictor 124 searches for a block that is most similar to the current block in a reference image that has been encoded and decoded earlier than the current image, and generates a prediction block for the current block by using the searched block. In addition, a motion vector (MV) is generated, which corresponds to the displacement between the current block in the current image and the prediction block in the reference image. Typically, motion estimation is performed on the luminance (luma) component, and the motion vector calculated based on the luminance component is used for both the luminance component and the chrominance component. The motion information including information of the reference image and information about the motion vector used to predict the current block is encoded by the entropy encoder 155 and transmitted to the video decoding device.

[0046] The inter-frame predictor 124 may also perform interpolation of a reference image or a reference block to increase the accuracy of the prediction. In other words, a subsample is interpolated between two consecutive integer samples by applying a filter coefficient to a plurality of consecutive integer samples including two integer samples. When performing a process of searching for a block most similar to the current block on the interpolated reference image, a decimal unit precision may be represented for the motion vector instead of an integer sample unit precision. For each target region to be encoded, for example, a unit such as a slice, a tile, a CTU, a CU, etc., the precision or resolution of the motion vector may be set differently. When such an adaptive motion vector resolution (AMVR) is applied, information about the motion vector resolution to be applied to each target region should be signaled for each target region. For example, when the target region is a CU, information about the motion vector resolution applied to each CU is signaled. The information about the motion vector resolution may be information representing the precision of the motion vector difference to be described below.

[0047] On the other hand, the inter-frame predictor 124 may perform inter-frame prediction by utilizing bidirectional prediction. In the case of bidirectional prediction, two reference images and two motion vectors representing the block positions most similar to the current block in each reference image are used. The inter-frame predictor 124 selects a first reference image and a second reference image from a reference image list 0 (RefPicList0) and a reference image list 1 (RefPicList1), respectively. The inter-frame predictor 124 also searches for a block most similar to the current block in the corresponding reference image to generate a first reference block and a second reference block. In addition, a prediction block of the current block is generated by averaging or weighted averaging the first reference block and the second reference block. In addition, motion information including information about two reference images used to predict the current block and including information about two motion vectors is transmitted to the entropy encoder 155. Here, the reference image list 0 may be composed of images that are before the current image in the display order in the pre-reconstructed image, and the reference image list 1 may be composed of images that are after the current image in the display order in the pre-reconstructed image. However, although not particularly limited thereto, the pre-reconstructed image that is after the current image in the display order may be additionally included in the reference image list 0. Conversely, a pre-reconstructed image preceding the current image may also be additionally included in the reference image list 1 .

[0048] In order to minimize the amount of bits consumed for encoding motion information, various methods may be used.

[0049] For example, when the reference image and motion vector of the current block are the same as those of the neighboring block, information capable of identifying the neighboring block is encoded to transmit the motion information of the current block to the video decoding device. This method is called merge mode.

[0050] In the merge mode, the inter predictor 124 selects a predetermined number of merge candidate blocks (hereinafter, referred to as 'merge candidates') from neighboring blocks of the current block.

[0051] As the neighboring blocks for deriving the merge candidate, all or some of the left block A0, the lower left block A1, the upper block B0, the upper right block B1 and the upper left block B2 adjacent to the current block in the current image may be used, such as Figure 4 As shown. In addition, in addition to the current image where the current block is located, a block located in a reference image (which may be the same as or different from the reference image used to predict the current block) may also be used as a merge candidate. For example, a co-located block of the current block in the reference image or a block adjacent to the co-located block may be additionally used as a merge candidate. If the number of merge candidates selected by the above method is less than the preset number, a zero vector is added to the merge candidate.

[0052] The inter-frame predictor 124 configures a merge list including a predetermined number of merge candidates by using adjacent blocks. A merge candidate to be used as motion information of the current block is selected from among the merge candidates included in the merge list, and merge index information for identifying the selected candidate is generated. The generated merge index information is encoded by the entropy encoder 155 and transmitted to the video decoding device.

[0053] Merge skip mode is a special case of merge mode. After quantization, when all transform coefficients for entropy coding are close to zero, only the neighbor block selection information is transmitted without the residual signal. By using merge skip mode, relatively high coding efficiency can be achieved for images with slight motion, still images, screen content images, etc.

[0054] Hereinafter, merge mode and merge skip mode are collectively referred to as merge / skip mode.

[0055] Another method for encoding motion information is the advanced motion vector prediction (AMVP) mode.

[0056] In the AMVP mode, the inter-frame predictor 124 derives a motion vector prediction candidate for a motion vector of a current block by using neighboring blocks of the current block. As neighboring blocks for deriving motion vector prediction candidates, the neighboring blocks may be used. Figure 4 All or some of the left block A0, lower left block A1, upper block B0, upper right block B1 and upper left block B2 adjacent to the current block in the current image shown. In addition, in addition to the current image where the current block is located, blocks located in a reference image (which may be the same as or different from the reference image used to predict the current block) may also be used as adjacent blocks for deriving motion vector prediction candidates. For example, a co-located block of the current block in the reference image or a block adjacent to the co-located block may be used. If the number of motion vector candidates selected by the above method is less than the preset number, a zero vector is added to the motion vector candidates.

[0057] The inter-frame predictor 124 derives a motion vector prediction candidate by using the motion vectors of the neighboring blocks, and determines a motion vector prediction of the motion vector of the current block by using the motion vector prediction candidate. In addition, a motion vector difference is calculated by subtracting the motion vector prediction from the motion vector of the current block.

[0058] The motion vector prediction can be obtained by applying a predefined function (e.g., median and mean calculations, etc.) to the motion vector prediction candidate. In this case, the video decoding device also knows the predefined function. In addition, since the neighboring blocks used to derive the motion vector prediction candidate are blocks that have completed encoding and decoding, the video decoding device may also already know the motion vectors of the neighboring blocks. Therefore, the video encoding device does not need to encode the information for identifying the motion vector prediction candidate. Accordingly, in this case, information about the motion vector difference and information about the reference image used to predict the current block are encoded.

[0059] On the other hand, the motion vector prediction can also be determined by selecting any one of the motion vector prediction candidates. In this case, information for identifying the selected motion vector prediction candidate is additionally encoded together with information about the motion vector difference used to predict the current block and information about the reference image.

[0060] The subtractor 130 generates a residual block by subtracting a prediction block generated by the intra predictor 122 or the inter predictor 124 from the current block.

[0061] The transformer 140 transforms the residual signal in the residual block having the pixel value of the spatial domain into the transform coefficient of the frequency domain. The transformer 140 may transform the residual signal in the residual block by using the entire size of the residual block as the transform unit, or may also divide the residual block into a plurality of sub-blocks, and may perform the transform by using the sub-block as the transform unit. Alternatively, the residual block is divided into two sub-blocks, namely, a transform region and a non-transform region, so as to transform the residual signal by using only the transform region sub-block as the transform unit. Here, the transform region sub-block may be one of two rectangular blocks having a size ratio of 1:1 based on the horizontal axis (or the vertical axis). In this case, the entropy encoder 155 encodes a flag (cu_sbt_flag) indicating that only the sub-block is transformed, and direction (vertical / horizontal) information (cu_sbt_horizontal_flag) and / or position information (cu_sbt_pos_flag), and notifies the video decoding device with a signal. In addition, the size of the transform region subblock may have a size ratio of 1:3 based on the horizontal axis (or vertical axis). In this case, a flag (cu_sbt_quad_flag) for dividing the corresponding partition is additionally encoded by the entropy encoder 155 and signaled to the video decoding apparatus.

[0062] On the other hand, the transformer 140 may perform transformation of the residual block separately in the horizontal direction and the vertical direction. For the transformation, various types of transformation functions or transformation matrices may be used. For example, paired transformation functions for horizontal transformation and vertical transformation may be defined as a multiple transform set (MTS). The transformer 140 may select a transformation function pair with the highest transformation efficiency in the MTS, and may transform the residual block in each of the horizontal and vertical directions. The information (mts_idx) about the transformation function pair in the MTS is encoded by the entropy encoder 155 and notified to the video decoding device with a signal.

[0063] The quantizer 145 quantizes the transform coefficients output from the transformer 140 using the quantization parameters and outputs the quantized transform coefficients to the entropy encoder 155. The quantizer 145 may also quantize the relevant residual block immediately without transforming any block or frame. The quantizer 145 may also apply different quantization coefficients (scaling values) according to the position of the transform coefficients in the transform block. The quantization matrix applied to the quantized transform coefficients arranged in two dimensions may be encoded and signaled to the video decoding device.

[0064] The rearrangement unit 150 may perform rearrangement of coefficient values ​​on the quantized residual value.

[0065] The rearrangement unit 150 can change the 2D coefficient array into a 1D coefficient sequence by using coefficient scanning. For example, the rearrangement unit 150 can use a zig-zag scan or a diagonal scan to scan the DC coefficient to the coefficient of the high frequency region to output a 1D coefficient sequence. Depending on the size of the transform unit and the intra-frame prediction mode, the zig-zag scan can also be replaced by a vertical scan that scans the 2D coefficient array in the column direction and a horizontal scan that scans the 2D block type coefficient in the row direction. In other words, depending on the size of the transform unit and the intra-frame prediction mode, the scanning method to be used can be determined among zig-zag scanning, diagonal scanning, vertical scanning, and horizontal scanning.

[0066] The entropy encoder 155 encodes the sequence of 1D quantized transform coefficients output from the rearrangement unit 150 by using various encoding schemes including Context-based Adaptive Binary Arithmetic Code (CABAC), Exponential Golomb, etc. to generate a bitstream.

[0067] In addition, the entropy encoder 155 encodes information related to block segmentation (e.g., CTU size, CTU segmentation flag, QT segmentation flag, MTT segmentation type, and MTT segmentation direction, etc.) so that the video decoding device can segment the block equivalent to the video encoding device. In addition, the entropy encoder 155 encodes information about the prediction type indicating whether the current block is encoded by intra-frame prediction or inter-frame prediction. The entropy encoder 155 encodes intra-frame prediction information (i.e., information about intra-frame prediction mode) or inter-frame prediction information (merge index in the case of merge mode, and information about reference image index and motion vector difference in the case of AMVP mode) according to the prediction type. In addition, the entropy encoder 155 encodes information related to quantization (i.e., information about quantization parameters and information about quantization matrices).

[0068] The inverse quantizer 160 inversely quantizes the quantized transform coefficient output from the quantizer 145 to generate a transform coefficient. The inverse transformer 165 transforms the transform coefficient output from the inverse quantizer 160 from the frequency domain to the spatial domain to reconstruct a residual block.

[0069] The adder 170 adds the reconstructed residual block and the prediction block generated by the predictor 120 to reconstruct the current block. When intra-prediction is performed on the next block, the pixels in the reconstructed current block are used as reference pixels.

[0070] The loop filtering unit 180 performs filtering on the reconstructed pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. that occur due to block-based prediction and transform / quantization. The loop filtering unit 180 as an in-loop filter may include all or some of a deblocking filter 182, a sample adaptive offset (SAO) filter 184, and an adaptive loop filter (ALF) 186.

[0071] The deblocking filter 182 filters the boundaries between the reconstructed blocks to remove blocking artifacts that occur due to block unit encoding / decoding, and the SAO filter 184 and the ALF 186 perform additional filtering on the deblocking filtered video. The SAO filter 184 and the ALF 186 are filters for compensating for the difference between the reconstructed pixels and the original pixels that occur due to lossy coding. The SAO filter 184 applies an offset in units of CTU to enhance subjective image quality and encoding efficiency. On the other hand, the ALF 186 performs block unit filtering and applies different filters to compensate for distortion by dividing the boundaries of the corresponding blocks and the degree of the amount of change. Information about the filter coefficients to be used for the ALF can be encoded and notified to the video decoding device with a signal.

[0072] The reconstructed blocks filtered by the deblocking filter 182, the SAO filter 184, and the ALF 186 are stored in the memory 190. When all blocks in one image are reconstructed, the reconstructed image can be used as a reference image for inter-frame prediction of blocks within an image to be subsequently encoded.

[0073] Figure 5 is a functional block diagram of a video decoding device that can implement the technology of the present invention. Figure 5 , describes a video decoding device and components of the device.

[0074] The video decoding apparatus may include an entropy decoder 510 , a rearrangement unit 515 , an inverse quantizer 520 , an inverse transformer 530 , a predictor 540 , an adder 550 , a loop filtering unit 560 , and a memory 570 .

[0075] Similar to Figure 1 Each component of the video encoding device and the video decoding device can be implemented as hardware or software, or as a combination of hardware and software. In addition, the function of each component can be implemented as software, and the microprocessor can also be implemented to execute the function of the software corresponding to each component.

[0076] The entropy decoder 510 extracts information related to block partitioning by decoding a bitstream generated by a video encoding apparatus to determine a current block to be decoded, and extracts prediction information required to reconstruct the current block and information about a residual signal.

[0077] The entropy decoder 510 determines the size of the CTU by extracting information about the CTU size from a sequence parameter set (SPS) or a picture parameter set (PPS), and divides the image into CTUs of the determined size. In addition, the CTU is determined as the highest level (i.e., the root node) of the tree structure, and the partition information of the CTU can be extracted to partition the CTU by using the tree structure.

[0078] For example, when CTU is segmented by using the QTBTTT structure, the first flag (QT_split_flag) related to the segmentation of QT is first extracted to segment each node into four nodes of the lower layer. In addition, the second flag (mtt_split_flag) related to the segmentation of MTT, the segmentation direction (vertical / horizontal) and / or the segmentation type (binary / tripartite) are extracted with respect to the node corresponding to the leaf node of QT to segment the corresponding leaf node into the MTT structure. As a result, each node below the leaf node of QT is recursively segmented into a BT or TT structure.

[0079] As another example, when a CTU is split by using a QTBTTT structure, a CU split flag (split_cu_flag) indicating whether the CU is split is extracted. When the corresponding block is split, a first flag (QT_split_flag) may also be extracted. During the splitting process, for each node, 0 or more recursive MTT splits may occur after 0 or more recursive QT splits. For example, for a CTU, MTT splits may occur immediately, or, conversely, multiple QT splits may occur only.

[0080] As another example, when CTU is segmented by using the QTBT structure, the first flag (QT_split_flag) related to the segmentation of the QT is extracted to segment each node into four nodes of the lower layer. In addition, the segmentation flag (split_flag) indicating whether the node corresponding to the leaf node of the QT is further segmented into BT and the segmentation direction information are extracted.

[0081] On the other hand, when the entropy decoder 510 determines the current block to be decoded by using the partitioning of the tree structure, the entropy decoder 510 extracts information about the prediction type indicating whether the current block is intra-predicted or inter-predicted. When the prediction type information indicates intra-prediction, the entropy decoder 510 extracts syntax elements of intra-prediction information (intra-prediction mode) for the current block. When the prediction type information indicates inter-prediction, the entropy decoder 510 extracts information representing syntax elements of inter-prediction information, that is, a motion vector and a reference image to which the motion vector refers.

[0082] In addition, the entropy decoder 510 extracts quantization-related information and extracts information on a quantized transform coefficient of the current block as information on a residual signal.

[0083] The rearrangement unit 515 may change the sequence of 1D quantized transform coefficients entropy-decoded by the entropy decoder 510 into a 2D coefficient array (ie, block) again in the reverse order of the coefficient scanning order performed by the video encoding device.

[0084] The inverse quantizer 520 inversely quantizes the quantized transform coefficients, and inversely quantizes the quantized transform coefficients by using quantization parameters. The inverse quantizer 520 may also apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in 2D. The inverse quantizer 520 may perform inverse quantization by applying a matrix of quantization coefficients (scaling values) from a video encoding device to a 2D array of quantized transform coefficients.

[0085] The inverse transformer 530 reconstructs a residual signal by inversely transforming the inversely quantized transform coefficient from the frequency domain to the spatial domain to generate a residual block of the current block.

[0086] In addition, when the inverse transformer 530 inversely transforms a partial region (sub-block) of the transform block, the inverse transformer 530 extracts a flag (cu_sbt_flag) that only the sub-block of the transform block is transformed, direction (vertical / horizontal) information (cu_sbt_horizontal_flag) of the sub-block, and / or position information (cu_sbt_pos_flag) of the sub-block. The inverse transformer 530 also inversely transforms the transform coefficient of the corresponding sub-block from the frequency domain to the spatial domain to reconstruct the residual signal, and fills the region that is not inversely transformed with a value of "0" as the residual signal to generate a final residual block of the current block.

[0087] In addition, when MTS is applied, the inverse transformer 530 determines the transform index or transform matrix to be applied in each of the horizontal and vertical directions by using the MTS information (mts_idx) signaled from the video encoding device. The inverse transformer 530 also performs inverse transform on the transform coefficients in the transform block in the horizontal and vertical directions by using the determined transform function.

[0088] The predictor 540 may include an intra predictor 542 and an inter predictor 544. The intra predictor 542 is activated when the prediction type of the current block is intra prediction, and the inter predictor 544 is activated when the prediction type of the current block is inter prediction.

[0089] The intra predictor 542 determines an intra prediction mode of a current block among a plurality of intra prediction modes according to syntax elements of the intra prediction mode extracted from the entropy decoder 510. The intra predictor 542 also predicts the current block by using neighboring reference pixels of the current block according to the intra prediction mode.

[0090] The inter predictor 544 determines a motion vector of a current block and a reference image to which the motion vector refers by using a syntax element of the inter prediction mode extracted from the entropy decoder 510 .

[0091] The adder 550 reconstructs the current block by adding the residual block output from the inverse transformer 530 and the prediction block output from the inter predictor 544 or the intra predictor 542. When intra-predicting a subsequent block to be decoded, pixels within the reconstructed current block are used as reference pixels.

[0092] The loop filtering unit 560 as an in-loop filter may include a deblocking filter 562, an SAO filter 564, and an ALF 566. The deblocking filter 562 performs deblocking filtering on the boundary between the reconstructed blocks in order to remove block artifacts that occur due to block unit decoding. The SAO filter 564 and the ALF 566 perform additional filtering on the reconstructed blocks after deblocking filtering in order to compensate for the difference between the reconstructed pixels and the original pixels that occurs due to lossy encoding. The filter coefficients of the ALF are determined by using information about the filter coefficients decoded from the bitstream.

[0093] The reconstructed blocks filtered by the deblocking filter 562, the SAO filter 564, and the ALF 566 are stored in the memory 570. When all blocks in one image are reconstructed, the reconstructed image can be used as a reference image for inter-frame prediction of blocks within an image to be subsequently encoded.

[0094] In some embodiments, the present invention relates to encoding and decoding video images as described above. More specifically, the present invention provides such a video encoding and decoding method and apparatus, which calculates weights at the decoder side by using previously reconstructed luma prediction blocks or template matching in inter-frame prediction of chroma components. The video encoding and decoding method and apparatus apply the calculated weights to the chroma dual prediction block to generate a final prediction block.

[0095] The following embodiments may be performed by the inter-frame predictor 124 in the video encoding apparatus. The following embodiments may also be performed by the inter-frame predictor 544 in the video decoding apparatus.

[0096] The video encoding device can generate signaling information associated with the present embodiment from the perspective of optimizing rate distortion when inter-predicting the chrominance component. The video encoding device can encode the signaling information using the entropy encoder 155 and send the encoded signaling information to the video decoding device. The video decoding device can use the entropy decoder 510 to decode the signaling information associated with the inter-frame prediction of the chrominance component from the bitstream.

[0097] In the following description, the term 'target block' may be used interchangeably with a current block or a coding unit (CU), or may refer to some regions of a coding unit.

[0098] In addition, a flag value of true indicates a case where the flag is set to 1. In addition, a flag value of false indicates a case where the flag is set to 0.

[0099] Hereinafter, the embodiments of the present invention are described centering around a video decoding apparatus, but the embodiments can be similarly applied to a video encoding apparatus.

[0100] I. Intra-frame prediction technology

[0101] The following describes the intra-frame prediction techniques referred to by the implementation scheme. Several techniques for improving encoding and decoding efficiency by utilizing intra-frame prediction are introduced.

[0102] In the Versatile Video Codec (VVC) technology, the intra prediction mode of a luma block has 65 subdivided directional modes (ie, 2 to 66) in addition to the non-directional modes (ie, planar and DC), such as Figure 3a The 65 directional modes, planar and DC modes are collectively referred to as 67 IPMs.

[0103] In VVC technology, Cross-Component Linear Model prediction (CCLM) and Multi-Model Linear Model prediction (MMLM) are techniques for predicting chrominance samples by determining the correlation between luminance and chrominance components.

[0104] When the CCLM mode is applied to the intra-frame prediction of the current chroma block, the video decoding device determines the area in the luminance image corresponding to the current chroma block (hereinafter, referred to as the "corresponding luminance area"). The left reference pixels and the upper reference pixels of the corresponding luminance area and the left reference pixels and the upper reference pixels of the target chroma block can be used for the prediction of the linear model. Hereinafter, the left reference pixels and the upper reference pixels are collectively referred to as reference pixels, adjacent pixels or neighboring pixels. In addition, the reference pixels in the chroma channel are represented as chroma reference pixels, and the reference pixels in the luminance channel are represented as luminance reference pixels.

[0105] Under CCLM prediction, in order to generate a prediction block (which is a predictor of a target chrominance block), a linear model is derived between reference pixels of a corresponding luminance region and reference pixels of a chrominance block, and the linear model is applied to reconstructed pixels of the corresponding luminance region. For example, a linear model may be derived using four pairs of pixels obtained by combining pixels in adjacent pixel lines of a current chrominance block and pixels in a corresponding luminance region. The video decoding apparatus may derive a linear model represented by α, β as shown in Equation 1 for four pairs of pixels.

[0106] [Equation 1]

[0107]

[0108] Here, for the corresponding brightness pixels of the four pairs of pixels, X a and X b Represents the average of the two minimum values ​​and the average of the two maximum values. In addition, for chrominance pixels, Y a and Y b Represent the average of the two minimum values ​​and the average of the two maximum values ​​respectively. Then, the video decoding device can use a linear model to obtain the pixel value rec' of the corresponding brightness area. L [x][y] generates the prediction factor Pred of the current chrominance block C [x][y], as shown in Equation 2.

[0109] [Equation 2]

[0110] Pred C [x][y]=α·rec′ L [x][y]+β

[0111] As described above, according to the position of the neighboring pixels used in the linear model derivation process, the CCLM mode is divided into three modes: CCLM_LT, CCLM_L and CCLM_T. The CCLM_LT mode uses two pixels in each direction of the neighboring pixels on the left and upper sides of the current chromaticity block. The CCLM_L mode uses four pixels in the neighboring pixels on the left side of the current chromaticity block. Finally, the CCLM_T mode uses four pixels in the neighboring pixels on the upper side of the current chromaticity block.

[0112] In VVC technology, weighted chroma prediction (WCP) uses the difference vector between the luma sample and the adjacent luma sample to calculate the weight of the weighted sum of the adjacent chroma samples. Then, by weighted summing the adjacent chroma samples using the calculated weights, a prediction factor for the chroma sample is generated, such as Figure 6 shown.

[0113] First, W is defined as the width of the chroma block, and H is defined as the height of the chroma block. If necessary, the reconstructed luma block and the adjacent luma samples can be downsampled according to the chroma format. For x=0, ..., W-1, y=0, ..., H-1, the video decoding device calculates Delta L [k], which is the luma sample rec' in the reconstructed luma block L [x][y] and the adjacent brightness vector Ref of length K L The absolute value of the difference between [k], as shown in Equation 3.

[0114] [Equation 3]

[0115] Delta L [k] = |rec′ L [x][y]-Ref L [k]|

[0116] Here, k is an index indicating an element of an adjacent brightness vector.

[0117] The video decoding apparatus then calculates a weight vector Weight[k] by using a normalization process as shown in Equation 4.

[0118] [Equation 4]

[0119]

[0120] By using the adjacent sample vector Ref of the chrominance block C [k] and weight vector Weight[k], the video decoding device can calculate the prediction factor Pred of the chrominance sample for x=0, ..., W-1, y=0, ..., H-1 C [x][y], as shown in Equation 5.

[0121] [Equation 5]

[0122]

[0123] II. Bi-prediction with CU-level Weight (BCW)

[0124] In VVC technology, when performing bi-prediction, the advanced motion vector prediction (AMVP) mode of inter-frame prediction assigns weights to two predicted signals derived from two motion vectors at the coding unit (CU) level. This method is called bi-prediction with CU-level weight (BCW). In BCW, the video decoding device generates the final predicted signal by averaging the two predicted signals based on the weights in Equation 6.

[0125] [Equation 6]

[0126] P bi-pred =((8-w)×P 0 +w×P 1 +4)>>3

[0127] Here, P 0 and P 1 Represents two bi-prediction signals.

[0128] In low-latency encoding, the weight w can be one of five weights {-2, 3, 4, 5, 10}. On the other hand, if the encoding is not low-latency, the weight w can be one of three weights {3, 4, 5}. These CU-level weights can be determined by the weight index transmitted per CU in AMVP mode. In merge mode, the CU-level weights can be determined based on the merge candidate index.

[0129] As described above, in conventional bidirectional inter prediction, preset weights are used for luma components and chroma components. The following describes a method for calculating weights of chroma components at the decoder side by using previously reconstructed luma components in addition to chroma components when performing bi-prediction on chroma components.

[0130] III. Embodiments according to the present invention

[0131] Similar to the luma signal, the bi-prediction of the chroma signal can also be set with weights at the CU level. As shown in Equation 7, BCW can be implemented to generate the final predicted signal by weighted summation of the two predicted signals derived from the two motion vectors.

[0132] [Equation 7]

[0133] P bi-pred =w0×P 0 +w1×P 1

[0134] Hereinafter, w0 and w1 are referred to as a first chrominance weight and a second chrominance weight.

[0135] In this embodiment, instead of using the preset weights for the chroma components as described above, the video decoding device adaptively calculates the weights of the chroma components. In this case, similar to the traditional BCW using preset weights, the adaptive weight derivation method can be controlled based on the on / off flag of the CU level (hereinafter referred to as the "chroma weight derivation flag"). For example, if the chroma weight derivation flag is true, the video decoding device derives the weights of the chroma prediction block according to an embodiment of the present invention. On the other hand, if the chroma weight derivation flag is false, the video decoding device derives the weights of the chroma prediction block according to a conventional method. Here, the conventional method can utilize the preset weights as described above.

[0136] <Implementation 1> Chroma BCW using weighted chroma prediction

[0137] In this embodiment, the video decoding device performs bi-prediction (BCW) with CU-level weights on the chroma component by using weighted chroma prediction (WCP). Here, WCP uses the difference vector between the luma sample and the adjacent luma sample to calculate the weight. The calculated weight is then used for weighted prediction of the relevant chroma sample.

[0138] In this embodiment, in order to perform BCW of the chrominance components, WCP may be applied as in Embodiment 1-1 or Embodiment 1-2.

[0139] <Implementation 1-1> Using the difference between the luminance double prediction block and the final luminance reconstruction block

[0140] When BCW is performed on the luma component, a bi-prediction block pred0 is generated. L and pred1 L And the final brightness reconstruction block pred L rec Here, pred0 L and pred1 L Respectively represent the forward prediction block and the backward prediction block of the luminance block. In addition, pred L rec is equal to the output of adder 550. The video decoding device can use pred0 L 、pred1 L and pred L rec To determine the weights of the two predicted signals in the bi-prediction of the chrominance component.

[0141] Figure 7 is a schematic diagram illustrating the use of a luma block in predicting a chroma block according to at least one embodiment of the present invention.

[0142] Video decoding device calculates Delta p0 and Delta p1 , which is the absolute value of the difference between the luma bi-predicted block and the final luma reconstructed block, as shown in Equation 8.

[0143] [Equation 8]

[0144] Delta p0 [x][y]=|pred0 L [x][y]-pred L rec [x][y]|

[0145] Delta p1 [x][y]=|pred1 L [x][y]-pred L rec [x][y]|

[0146] In one example, if the chrominance block has a size of W×H and the corresponding luminance block has a size larger than the chrominance block, the video decoding apparatus may p0 and Delta p1 Down-sampling is performed to a size of W×H. Alternatively, the luma block may be first down-sampled to a size of W×H, and then the process of Equation 8 may be applied. Here, W represents the width of the chroma block, and H represents the height of the chroma block. In the following, Delta p0 and Delta p1 They are called the first differential block and the second differential block respectively.

[0147] It can be achieved by using the dual prediction block pred0 of the chrominance component C and pred1 C To generate the final chrominance prediction block Pred C Here, pred0 C and pred1 C Represent the forward prediction block and backward prediction block of the chrominance block respectively. At this time, Delta p0 and Delta p1 The weights w0 and w1 of Equation 7 are calculated for the chrominance components. In this case, each chrominance sample Pred C [x][y] determines w0 and w1 differently.

[0148] First, the video decoding apparatus calculates sum[x][y] as shown in Equation 9.

[0149] [Equation 9]

[0150] sum[x][y]=Delta p0[x][y]+Delta p1 [x][y]

[0151] Then, by adding Delta p0 [x][y] and Delta p1 [x][y] is divided by sum[x][y] to generate normalized w0[x][y] and w1[x][y] so that the sum of w0[x][y] and w1[x][y] is 1. Hereinafter, w0[x][y] and w1[x][y] are referred to as a first weighted block and a second weighted block. The video decoding apparatus performs dual prediction of the chrominance components as shown in Equation 10 by using weights normalized for x=0, ..., W-1, y=0, ..., H-1, respectively.

[0152] [Equation 10]

[0153] Pred C [x][y]=w0[x][y]×pred0 C [x][y]+w1[x][y]×pred1 C [x][y]

[0154] Here, Pred C [x][y] represents the final predicted samples of the chrominance components.

[0155] In one example, in Equation 10, a weight for each chrominance sample based on the luminance block is used. As another example, the final predicted sample of the chrominance component can be generated by using one weight for each prediction block. For example, the video decoding device calculates Delta p0 The average value μ p0 and Delta p1 The average value μ p1 , then calculate w0 = μ p0 / (μ p0 +μ p1 ),w1=μ p1 / (μ p0 +μ p1 ). In the following, μ p0 and μ p1 They are referred to as the first mean and the second mean, respectively. The video decoding apparatus performs bi-prediction of the chroma component as shown in Equation 11 by using weights normalized for x=0, ..., W-1, y=0, ..., H-1.

[0156] [Equation 11]

[0157] Pred C [x][y]=w0×pred0 C[x][y]+w1×pred1 C [x][y]

[0158] <Implementation 1-2> Using the difference between the luminance block and the adjacent luminance samples

[0159] As described above, when BCW is performed on the luminance component, a bi-prediction block pred0 is generated. L and pred1 L And the final brightness reconstruction block pred L rec The pixel vector around each block is defined as Ref 0 Ref 1 and Ref L The video decoding device calculates Delta p0 、Delta p1 and Delta L , which is the difference vector between each luminance block and the adjacent pixels, as shown in Equation 12. In the following, Delta p0 、Delta p1 and Delta L They are respectively called the first difference vector, the second difference vector and the brightness difference vector.

[0160] [Equation 12]

[0161] Delta p0 [k]=|pred0 L [x][y]-Ref 0 [k]|

[0162] Delta L [k]=|pred L rec [x][y]-Ref L [k]|

[0163] Delta p1 [k]=|pred1 L [x][y]-Ref 1 [k]|

[0164] The video decoding device then uses Delta p0 、Delta p1 and Delta L To calculate the weights for the chrominance components. First, define Diff 0 =Delta L -Delta p0 and Diff 1 =Delta L -Delta p1. After that, Diff 0 and Diff 1 is called the first double difference vector and the second double difference vector. If the number of pixels around the prediction block is K, then Diff 0 and Diff 1 are vectors of size 1×K respectively. Figure 8 The example in schematically shows how to calculate Diff based on the difference between the luma block and the adjacent luma samples 0 and Diff 1 .

[0165] For example, a video decoding device may use Diff 0 and Diff 1 The weights for the chrominance components are calculated as follows. The video decoding device converts the Diff 0 and Diff 1 Cascade to generate a Diff of size 1×2K p , as shown in Equation 13. In the following, Diff p It is called the concatenated double difference vector.

[0166] [Equation 13]

[0167] Diff p [k] = concat(Diff 1 [k],Diff 0 [k])

[0168] The video decoding device then normalizes Diff p Each element of is used to calculate the weight vector Weight[k], as shown in Equation 14.

[0169] [Equation 14]

[0170]

[0171] Here, concat represents an operation of concatenating two vectors.

[0172] For the normalized weight vector Weight, Weight0 may be defined as k=0 to K-1 elements in Weight[k], and Weight1 may be defined as k=K to 2K-1 elements in Weight[k]. The video decoding apparatus may then calculate the weights w0 and w1 of Equation 7 for the chroma components by calculating the sum of the vector elements of Weight0 and Weight1.

[0173] The video decoding device can perform dual prediction of the chrominance component by using the calculated weights. For x=0, ..., W-1, y=0, ..., H-1, the final predicted sample Pred of the chrominance component can be calculated as shown in Formula 15 C [x][y].

[0174] [Equation 15]

[0175] Pred C [x][y]=w0×pred 0C [x][y]+w1×pred 1c [x][y]

[0176] Here, pred0 C and pred1 C Represent the forward prediction block and backward prediction block of the chroma block respectively.

[0177] <Implementation 2> Chroma BCW using template matching

[0178] In this embodiment, the video decoding device calculates the weights w0 and w1 of Equation 7 by using template matching. For example, the video decoding device can calculate the template matching cost by using the neighboring blocks of the predicted block in each predicted direction, such as Fig. 9 As shown in the example of , the calculated template matching cost is then normalized to calculate the weight.

[0179] For example, to predict an N×N chroma block, the video decoding device may define (N+2)×2 samples at the upper side of the block and 2×N samples at the left side of the block as a template. If the template matching cost is defined as the sum of squared differences (SSD), the video decoding device may calculate the template matching cost as shown in Equation 16.

[0180] [Equation 16]

[0181]

[0182] Here, T curr and T ref Represents the template area of ​​the current block and the reference block. In addition to SSD, based on |T curr (i,j)-T ref The sum of absolute differences (SAD) of (i,j)| can be defined as the template matching cost.

[0183] The (N+2)×2 samples at the upper side of the block and the 2×N samples at the left side are defined (but not necessarily) as a template, and a previously reconstructed area adjacent to the current block may also be defined as a template. For example, the 2N samples at the upper side of the block and the 2N samples at the left side may be defined as a template.

[0184] In the following, the template matching cost of the predicted blocks of pred0 and pred1 is defined as TC pred0 and TC pred1 In the following, TC pred0 and TC pred1 They are referred to as the first template matching cost and the second template matching cost, respectively. After normalizing the two matching costs, the video decoding apparatus may calculate weights w0 and w1 as shown in Equation 17 to set a larger weight for a block with a smaller matching cost.

[0185] [Equation 17]

[0186]

[0187] The video decoding apparatus may then use the normalized w0 as the weight of pred0, and use the normalized w1 as the weight of pred1.

[0188] In the aforementioned examples, the neighboring samples of the chrominance block are used to calculate the template matching cost, which does not limit the present invention. For example, instead of using the neighboring samples of the chrominance block, the neighboring samples of the luminance block may be used. In this case, instead of using the cost function shown in Equation 10, the template matching cost may be calculated by using a cost function including a first-order linear function. The first-order linear function may be used to convert the luminance sample value to the chrominance sample value, as shown in the CCLM example described above.

[0189] Reference now Fig.10 and Fig.11 , a method for deriving a weight of a current chrominance block used in bidirectional inter-frame prediction and using the derived weight to generate a prediction block of the current chrominance block is described.

[0190] Fig.10 is a flowchart of a method for predicting a current chrominance block performed by a video encoding device according to at least one embodiment of the present invention.

[0191] The video encoding apparatus generates a luma bi-prediction block and a final luma reconstruction block for a luma block (S1000).

[0192] Here, the luma bi-prediction block includes a first luma prediction block and a second luma prediction block. For example, the first luma prediction block may be a forward prediction luma block, and the second luma prediction block may be a backward prediction luma block.

[0193] The video encoding apparatus generates a chroma bi-prediction block for a current chroma block corresponding to a luminance block (S1002).

[0194] Here, the chroma dual prediction block includes a first chroma prediction block and a second chroma prediction block. For example, the first chroma prediction block may be a forward prediction chroma block, and the second chroma prediction block may be a backward prediction chroma block.

[0195] The video encoding apparatus derives the weight of the current chrominance block by using the final luma reconstruction block and the luma bi-prediction block (S1004). Here, the weight of the current chrominance block includes a first chrominance weight w0 and a second chrominance weight w1.

[0196] In one example, to derive the weights, the video encoding apparatus may utilize the difference between the luma bi-prediction block and the final luma reconstructed block.

[0197] The video encoding device can calculate the final luminance reconstruction block pred L rec With the first luma prediction block pred0 L The absolute value of the difference between the two to generate the first difference block Delta p0 The video encoding device can also calculate the final brightness reconstruction block and the second brightness prediction block pred1 L The absolute value of the difference between the two to generate the second difference block Delta p1 .

[0198] At this time, if the luminance block is larger than the current chrominance block, the video encoding device may generate the first difference block and the second difference block after downsampling the luminance bi-prediction block and the final luminance reconstruction block. Alternatively, the video encoding device may first generate the first difference block and the second difference block, and then downsample the first difference block and the second difference block.

[0199] The video encoding apparatus normalizes the first differential block and the second differential block based on the sum of the first differential block and the second differential block to generate a first weighted block w0[x][y] and a second weighted block w1[x][y].

[0200] As described above, per-pixel weights may be used, but one weight per prediction block may also be used. The video encoding apparatus calculates a first mean as the average of the elements of the first difference block and a second mean as the average of the elements of the second difference block. The video encoding apparatus may then normalize the first mean and the second mean to generate a first chrominance weight w0 and a second chrominance weight w1.

[0201] As another example, the video encoding apparatus may utilize the difference between a luma block and adjacent luma samples.

[0202] The video encoding device can calculate the pixel in the first brightness prediction block and the adjacent pixel Ref of the first brightness prediction block. 0 The absolute value of the difference between the two is used to generate the first difference vector Delta p0 The video encoding device calculates the pixel in the final brightness reconstruction block and the adjacent pixel Ref of the final brightness reconstruction block. L The absolute value of the difference between the two is used to generate the brightness difference vector Delta L The video encoding device also calculates the pixel in the second luminance prediction block and the adjacent pixel Ref of the second luminance prediction block. 1 The absolute value of the difference between the two is used to generate the second difference vector Delta p1 .

[0203] The video encoding device generates a first double difference vector Diff by calculating the absolute value of the difference between the first difference vector and the brightness difference vector 0 The video encoding device also generates a second double difference vector Diff by calculating the absolute value of the difference between the second difference vector and the brightness difference vector 1 .

[0204] The video encoding device concatenates the first double difference vector and the second double difference vector to generate a concatenated double difference vector Diff p The video encoding apparatus may normalize each element of the concatenated double difference vector, and may use the normalized elements to generate the first chroma weight w0 and the second chroma weight w1.

[0205] As another example, in order to derive the weight, the video encoding apparatus may calculate the first template matching cost TC by using the neighboring blocks of the predicted block in each predicted direction. pred0 and the second template matching cost TC pred0 After normalizing the calculated template matching cost, the video encoding apparatus may calculate the first chrominance weight w0 and the second chrominance weight w1 according to Equation 17 to set a larger weight to a block having a smaller matching cost.

[0206] The video encoding apparatus applies the weight to the chroma bi-prediction block to generate a first final chroma prediction block of the current chroma block (S1006).

[0207] When using per-pixel weights, the video encoding apparatus may apply the first weighted block and the second weighted block to the first chroma prediction block pred0 on a per-pixel basis. C and the second chroma prediction block pred0 C , to generate the first final chroma prediction block, as shown in Equation 10.

[0208] In the case of using the weight of each prediction block, the video encoding apparatus may apply the first chroma weight and the second chroma weight to the first chroma prediction block and the second chroma prediction block to generate a first final chroma prediction block, as shown in Equation 11.

[0209] The video encoding apparatus applies the preset weight to the chroma bi-prediction block to generate a second final chroma prediction block of the current chroma block (S1008). For example, the video encoding apparatus may generate the second final chroma prediction block of the current chroma block by using the preset weight according to BCW.

[0210] The video encoding apparatus selects an optimal chroma prediction block between the first final chroma prediction block and the second final chroma prediction block (S1010). For example, the video encoding apparatus may select the optimal chroma prediction block from the perspective of optimizing distortion from the original chroma block.

[0211] As described above, the flag indicating whether the chroma weight is to be derived is called a chroma weight derivation flag.

[0212] The video encoding apparatus determines a chroma weight derivation flag based on whether the best chroma prediction block is a first final chroma prediction block or a second final chroma prediction block ( S1012 ).

[0213] The video encoding apparatus encodes a chroma weight derivation flag (S1014).

[0214] Fig.11 is a flowchart of a method for predicting a current chrominance block performed by a video decoding device according to at least one embodiment of the present invention.

[0215] The video decoding apparatus generates a luma bi-prediction block and a final luma reconstruction block for a luma block ( S1100 ).

[0216] Here, the luma bi-prediction block includes a first luma prediction block and a second luma prediction block. For example, the first luma prediction block may be a forward prediction luma block, and the second luma prediction block may be a backward prediction luma block.

[0217] The video decoding apparatus generates a chroma bi-prediction block for a current chroma block corresponding to a luminance block (S1102).

[0218] Here, the chroma dual prediction block includes a first chroma prediction block and a second chroma prediction block. For example, the first chroma prediction block may be a forward prediction chroma block, and the second chroma prediction block may be a backward prediction chroma block.

[0219] The video decoding apparatus decodes the chroma weight derivation flag ( S1104 ).

[0220] The video decoding apparatus checks the chroma weight derivation flag ( S1106 ).

[0221] If the chroma weight derivation flag is true (Yes in S1106 ), the video decoding apparatus performs the following steps ( S1108 and S1110 ).

[0222] The video decoding apparatus derives the weight of the current chrominance block by using the final luminance reconstruction block and the luminance bi-prediction block (S1108). Here, the weight of the current chrominance block includes a first chrominance weight and a second chrominance weight.

[0223] Since the video decoding apparatus may adopt the same method as the video encoding apparatus, the method for deriving weights performed by the video decoding apparatus will not be further described.

[0224] The video decoding apparatus applies the weight to the chroma bi-prediction block to generate a final chroma prediction block of the current chroma block (S1110).

[0225] On the other hand, if the chroma weight derivation flag is false (S1106 is no), the video decoding device applies the preset weight to the chroma dual prediction block to generate a final chroma prediction block (S1120). For example, the video decoding device can generate a final chroma prediction block of the current chroma block by using the preset weight according to the BCW.

[0226] Although the steps in the various flowcharts are described as being performed sequentially, these steps merely illustrate the technical ideas of some embodiments of the present invention. Therefore, a person of ordinary skill in the art to which the present invention belongs can perform the steps by changing the order described in the various figures or by performing two or more steps in parallel. Therefore, the steps in the various flowcharts are not limited to the order shown in the order in which they occur.

[0227] It should be understood that the above description presents illustrative embodiments that can be implemented in various other ways. The functions described in some embodiments can be implemented by hardware, software, firmware and / or their combination. It should also be understood that the functional components described in the present invention are marked as "... units" to highlight the possibility of their independent implementation.

[0228] On the other hand, the various methods or functions described in some embodiments can be implemented as instructions stored in a non-volatile recording medium, which can be read and executed by one or more processors. Non-volatile recording media can include various types of recording devices that store data in a form that is readable by a computer system. For example, non-volatile recording media can include storage media, such as erasable programmable read-only memory (EPROM), flash drives, optical disk drives, magnetic hard drives, and solid-state drives (SSDs), etc.

[0229] Although exemplary embodiments of the present invention are described for illustrative purposes, it will be appreciated by those skilled in the art that various modifications, additions and substitutions may be made without departing from the spirit and scope of the present invention. Therefore, embodiments of the present invention are described for simplicity and clarity. The scope of the technical ideas of the embodiments of the present invention are not limited by the examples. Accordingly, it will be appreciated by those skilled in the art that the scope of the present invention should not be limited by the embodiments clearly described above, but by the claims and their equivalents.

[0230] Reference numerals:

[0231] 124: Inter-frame predictor

[0232] 155: Entropy Encoder

[0233] 510: Entropy Decoder

[0234] 544: Inter-frame predictor.

[0235] CROSS-REFERENCE TO RELATED APPLICATIONS

[0236] This application claims priority to and the benefit of Korean Patent Application No. 10-2022-0058181, filed on May 12, 2022, and Korean Patent Application No. 10-2023-0048426, filed on April 12, 2023, the entire contents of each of which are incorporated herein by reference.

Claims

1. A method for predicting a current chrominance block performed by a video decoding device, the method include: Generate a chroma dual prediction block including a first chroma prediction block and a second chroma prediction block for a current chroma block; deriving a weight of the current chrominance block by using a first luminance prediction block, a second luminance prediction block, and a final luminance reconstruction block of a luminance block corresponding to the current chrominance block, the weight comprising a first chrominance weight and a second chrominance weight; A final chroma prediction block for the current chroma block is generated by applying the weights to the chroma bi-prediction block.

2. The method according to claim 1, further comprising: include: decoding a flag indicating whether to derive weights for the current chroma block; as well as Check the signs; Wherein, when the value of the flag is true, the method further comprises: Derived weights.

3. The method according to claim 1, further comprising: include: A first luminance prediction block, a second luminance prediction block and a final luminance reconstruction block are generated for the luminance block.

4. The method according to claim 1, in, The derived weights include: generating a first differential block by calculating an absolute value of a difference between a final luma reconstruction block and a first luma prediction block; The second differential block is generated by calculating the absolute value of the difference between the final luma reconstruction block and the second luma prediction block.

5. The method according to claim 4, in, The derived weights include: The first weighted block and the second weighted block are generated by normalizing the first difference block and the second difference block based on the sum of the first difference block and the second difference block.

6. The method according to claim 5, in, Generating the final chroma prediction block includes: The first weighting block and the second weighting block are applied to the first chroma prediction block and the second chroma prediction block on a pixel-by-pixel basis.

7. The method according to claim 4, further comprising: include: When the luma block is larger than the current chroma block, the first luma prediction block, the second luma prediction block, and the final luma reconstruction block are downsampled before generating the first differential block and generating the second differential block.

8. The method according to claim 4, in, The derived weights include: The first chroma weight and the second chroma weight are generated by calculating a first mean value, which is an average value of elements of the first difference block, and a second mean value, which is an average value of elements of the second difference block, and by normalizing the first mean value and the second mean value.

9. The method according to claim 1, in, The derived weights include: generating a first difference vector by calculating an absolute value of a difference between a pixel in the first luma prediction block and an adjacent pixel of the first luma prediction block; generating a luma difference vector by calculating an absolute value of a difference between a pixel in the final luma reconstruction block and an adjacent pixel of the final luma reconstruction block; and A second difference vector is generated by calculating an absolute value of a difference between a pixel in the second luma prediction block and an adjacent pixel of the second luma prediction block.

10. The method according to claim 9, in, The derived weights further include: generating a first double difference vector by calculating an absolute value of a difference between the first difference vector and the brightness difference vector; and The second double difference vector is generated by calculating the absolute value of the difference between the second difference vector and the brightness difference vector.

11. The method according to claim 10, in, The derived weights further include: generating a concatenated double difference vector by concatenating the first double difference vector and the second double difference vector; The first chroma weight and the second chroma weight are generated by normalizing respective elements of the concatenated double difference vectors into normalized elements and by using the normalized elements.

12. A method for predicting a current chrominance block performed by a video encoding device, the method include: Generate a chroma dual prediction block including a first chroma prediction block and a second chroma prediction block for a current chroma block; deriving a weight of the current chrominance block by using a first luminance prediction block, a second luminance prediction block, and a final luminance reconstruction block of a luminance block corresponding to the current chrominance block, the weight comprising a first chrominance weight and a second chrominance weight; as well as A first final chroma prediction block of the current chroma block is generated by applying the weights to the chroma bi-prediction blocks.

13. The method according to claim 12, further comprising: include: generating a second final chroma prediction block of the current chroma block by applying a preset weight to the chroma bi-prediction block; as well as An optimal chroma prediction block is selected between the first final chroma prediction block and the second final chroma prediction block.

14. The method according to claim 13, further comprising: include: determining a flag indicating whether to derive a weight based on whether the best chroma prediction block is a first final chroma prediction block or a second final chroma prediction block; Encode the flag.

15. The method according to claim 12, further comprising: include: A first luminance prediction block, a second luminance prediction block and a final luminance reconstruction block are generated for the luminance block.

16. A computer-readable recording medium storing a bit stream generated by a video encoding method, the video encoding method include: Generate a chroma dual prediction block including a first chroma prediction block and a second chroma prediction block for a current chroma block; deriving a weight of the current chrominance block by using a first luminance prediction block, a second luminance prediction block, and a final luminance reconstruction block of a luminance block corresponding to the current chrominance block, the weight comprising a first chrominance weight and a second chrominance weight; as well as A final chroma prediction block for the current chroma block is generated by applying the weights to the chroma bi-prediction block.

Images