Method and apparatus for encoding / decoding image signal

Abstract

This disclosure provides a method and apparatus for encoding / decoding image signals. The method for decoding image signals according to the present invention may include the following steps: determining whether a brightness change exists between a current image including a current block and a reference image of the current image; if a brightness change is determined to exist between the current image and the reference image, determining candidate weight prediction parameters for the current block; determining weight prediction parameters for the current block based on index information for specifying any one of the candidate weight prediction parameters; and performing prediction for the current block based on the weight prediction parameters.

Description

[0001] This application is a divisional application of the invention patent application filed on April 28, 2017, with application number 201780041348.7 and entitled "Method and apparatus for encoding / decoding image signals". Technical Field

[0002] This invention relates to a video signal encoding / decoding method and device. Background Technology

[0003] In recent years, the demand for multimedia data (such as moving images) on the Internet has been increasing rapidly. However, the rate of bandwidth development of channels has struggled to keep pace with the rapidly increasing volume of multimedia data. As a result, the Video Coding Experts Group (VCEG) of the International Organization for Standardization (ITU-T) and the Moving Picture Experts Group (MPEG) of ISO / IEC released High Efficiency Video Coding (HEVC) version 1 in February 2014.

[0004] HEVC defines techniques such as intra-frame prediction, inter-frame prediction, transform, quantization, entropy coding, and loop filtering. Among them, inter-frame prediction is performed using previously reconstructed frames and motion information (such as motion vectors, reference frame indexes, prediction directions (inter-frame prediction indicators), etc.).

[0005] Higher correlation between images leads to higher prediction efficiency. However, if the correlation between images is reduced due to changes in brightness (such as fade-in or fade-out), inter-frame prediction results may be inaccurate. To address this problem, this invention proposes weighted prediction. Here, weighted prediction refers to estimating weights based on the degree of brightness change between images and applying the estimated weights to inter-frame prediction. Summary of the Invention

[0006] Technical issues

[0007] The main objective of this invention is to improve the efficiency of inter-frame prediction by using weights to perform inter-frame prediction when encoding / decoding images.

[0008] The main objective of this invention is to provide an apparatus and method that can effectively perform inter-frame prediction by selectively using weights when encoding / decoding an image, even when there are multiple light sources in the image or only brightness variations in local areas.

[0009] Technical solution

[0010] According to a method and apparatus for decoding a video signal, the present invention determines whether there is a brightness change between a current image including a current block and a reference image of the current image; if there is a brightness change between the current image and the reference image, determines a candidate weight prediction parameter for the current block; determines a weight prediction parameter for the current block based on index information for specifying one of the candidate weight prediction parameters; and performs prediction of the current block based on the weight prediction parameter.

[0011] In the method and apparatus for decoding video signals according to the present invention, the candidate weight prediction parameters may include a first prediction parameter for the reference image.

[0012] In the method and apparatus for decoding video signals according to the present invention, when the current image includes at least one region from which a second weight prediction parameter can be derived, the weight prediction parameter candidate further includes at least one second weight prediction parameter.

[0013] In the method and apparatus for decoding video signals according to the present invention, the first weight prediction parameter may be derived based on the predicted value of the first weight prediction parameter and the residual value of the first weight prediction parameter.

[0014] In the method and apparatus for decoding video signals according to the present invention, the predicted value for the first weight prediction parameter can be determined based on the accuracy of the current block.

[0015] In the method and apparatus for decoding video signals according to the present invention, the maximum number of candidates for weight prediction parameters can be adaptively determined according to the size of the current block.

[0016] In the method and apparatus for decoding video signals according to the present invention, the candidate weight prediction parameters may include initial weight prediction parameters having predetermined weight values.

[0017] According to a method and apparatus for encoding a video signal, the present invention determines whether there is a brightness change between a current image including a current block and a reference image of the current image; if there is a brightness change between the current image and the reference image, determines candidate weight prediction parameters for the current block; determines a weight prediction parameter for the current block from the candidate weight prediction parameters, and encodes index information for specifying the determined weight prediction parameter; and performs prediction of the current block based on the weight prediction parameter.

[0018] In the method and apparatus for encoding video signals according to the present invention, the candidate weight prediction parameters may include a first weight prediction parameter for the reference image.

[0019] In the method and apparatus for encoding video signals according to the present invention, when the current image includes at least one region from which a second weight prediction parameter can be derived, the weight prediction parameter candidate further includes at least one second weight prediction parameter.

[0020] The method and apparatus for encoding video signals according to the present invention encode a residual value indicating the difference between the first weight prediction parameter and the predicted value used for the first weight prediction parameter.

[0021] In the method and apparatus for encoding video signals according to the present invention, the predicted value for the first weight prediction parameter can be determined based on the accuracy of the current block.

[0022] In the method and apparatus for encoding video signals according to the present invention, the maximum number of candidates for weight prediction parameters can be adaptively determined according to the size of the current block.

[0023] In the method and apparatus for encoding video signals according to the present invention, the candidate weight prediction parameters may include initial weight prediction parameters having predetermined weight values.

[0024] Beneficial effects

[0025] In this invention, inter-frame prediction efficiency is improved by using weights to perform inter-frame prediction when encoding / decoding images.

[0026] Even when there are multiple light sources in an image or only brightness variations exist in local areas, the present invention can effectively perform inter-frame prediction by selectively using weights when encoding / decoding the image. Attached Figure Description

[0027] Figure 1 This is a block diagram illustrating an image encoding device according to an embodiment of the present invention.

[0028] Figure 2 This is a block diagram of an image decoding device according to an embodiment of the present invention.

[0029] Figure 3 This is a block diagram schematically illustrating a motion estimation method according to an embodiment of the present invention.

[0030] Figure 4 This is a diagram illustrating the position of adjacent blocks for obtaining motion information to be applied to the current block to be encoded, according to an embodiment of the present invention.

[0031] Figure 5 It is a diagram showing the brightness variation pattern between the current image and the reference image, including the current block.

[0032] Figure 6 This is a flowchart illustrating a method for estimating weight prediction parameters in an image coding device.

[0033] Figure 7 This is a diagram illustrating an example of using weight parameters to perform rate-distortion optimization (RDO) on prediction blocks.

[0034] Figure 8 This is a flowchart illustrating a method for encoding information related to the weight prediction parameters used for the current block.

[0035] Figure 9 This is a diagram illustrating an example of decoding weight prediction parameters in a decoding device.

[0036] Figure 10 This is a flowchart illustrating a method for using weighted prediction parameters in an encoding device.

[0037] Figure 11 This is a diagram showing the weight prediction parameters of the adjacent blocks that are adjacent to the current block.

[0038] Figure 12 This is a diagram used to illustrate an example of estimating the weight prediction parameters for the current block using only a few pixels.

[0039] Figure 13 An example of generating combined weighted prediction parameters is shown.

[0040] Figure 14 An example of decoding weight prediction parameters in a decoding device is shown. Detailed Implementation

[0041] This invention can be changed and modified in various ways and can be shown with reference to different exemplary embodiments, some of which will be described and illustrated in the accompanying drawings. However, these embodiments are not intended to limit the invention, but should be interpreted as including all modifications, equivalents, and substitutions that fall within the spirit and scope of the invention. The same reference numerals in the drawings consistently indicate the same elements.

[0042] Although the terms "first," "second," etc., can be used to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the teachings of the invention, a first element may be referred to as a second element, and a second element may also be referred to as a first element. The term "and / or" includes any and all combinations of the plurality of related listed items.

[0043] It should be understood that when a component is referred to as "connected to" or "combined to" another component, the component may be directly connected to or combined with the other component or there may be an intermediate component. Conversely, when a component is referred to as "directly connected to" or "directly combined to" another component, there is no intermediate component.

[0044] The terminology used herein is for the purpose of describing particular specific embodiments only and is not intended to limit the invention. As used herein, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” are intended to include the plural forms as well. It should also be understood that, when used in the specification, the terms “comprising” and / or “having” specify the presence of the stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof.

[0045] In the following description, exemplary embodiments of the invention will be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings consistently indicate the same elements, and redundant descriptions of the same elements will be omitted herein.

[0046] Figure 1 This is a block diagram illustrating an image encoding device according to an embodiment of the present invention.

[0047] Reference Figure 1 The image encoding device 100 includes an image segmentation unit 110, prediction units 120 and 125, a transformation unit 130, a quantization unit 135, a reordering unit 160, an entropy encoding unit 165, an inverse quantization unit 140 and an inverse transformation unit 145, a filter unit 150, and a memory 155.

[0048] Figure 1 Each of the elements shown is illustrated individually to represent a different feature or function in the encoding device, and does not imply that each component consists of a separate hardware or single software unit. That is, the elements are arranged individually for ease of description, wherein at least two elements may be combined into a single element, or a single element may be divided into multiple elements to perform a function. It should be noted that embodiments that integrate some elements into a combined element and / or divide elements into multiple individual elements are included within the scope of this invention without departing from the spirit of the invention.

[0049] Some components are not essential to the essential function of this invention and may be optional components used only to improve performance. This invention can be implemented by including only the components essential to embodiments of the invention, excluding those used only to improve performance. Structures including only the essential components, excluding the optional components used only to improve performance, are within the scope of this invention.

[0050] Image segmentation unit 110 can divide the input image into at least one block. Here, a block can refer to a coding unit (CU), a prediction unit (PU), or a transform unit (TU). The segmentation can be performed based on at least one of a quadtree or a binary tree. A quadtree is a method of dividing an upper block into sub-blocks whose width and height are both half that of the upper block. A binary tree is a method of dividing an upper block into sub-blocks whose width or height is half that of the upper block. Blocks can have non-square shapes as well as square shapes based on the segmentation using the quadtree or binary tree described above.

[0051] In embodiments of the present invention, CU can be used to refer not only to the unit of encoding but also to the unit of decoding.

[0052] Prediction units 120 and 125 may include an inter-frame prediction unit 120 for performing inter-frame prediction and an intra-frame prediction unit 125 for performing intra-frame prediction. Prediction units 120 and 125 may determine which of the two prediction methods—inter-frame prediction and intra-frame prediction—to perform on the PU, and may determine specific information about the determined prediction method (e.g., intra-frame prediction mode, motion vectors, and reference frames). Here, the processing unit performing the prediction may differ from the processing unit determining the prediction method and its specific information. For example, the prediction method and prediction mode may be determined for each PU, while the prediction may be performed for each TU.

[0053] The residual value (residual block) between the generated prediction block and the original block can be input to the transform unit 130. Furthermore, prediction mode information, motion vector information, etc., used for prediction can be encoded together with the residual value by the entropy coding unit 165 and sent to the decoding device. When using a specific coding mode, the original block can be encoded and sent to the decoding device without generating a prediction block through prediction units 120 and 125.

[0054] The inter-frame prediction unit 120 can predict the PU based on information from at least one of the previous image and the subsequent image of the current image. In these cases, the inter-frame prediction unit 120 can predict the PU based on information from a portion of the coded region in the current frame. The inter-frame prediction unit 120 may include a reference image interpolation unit, a motion information generation unit, and a motion compensation unit.

[0055] Reference image information can be provided to the reference image interpolation unit by the memory 155, and the reference image interpolation unit can generate pixel information less than or equal to integer pixels in the reference image. In the case of luminance pixels, an 8-tap DCT-based interpolation filter with variable filter coefficients can be used to generate pixel information less than or equal to integer pixels in 1 / 4 pixel units. In the case of chroma pixels, a 4-tap DCT-based interpolation filter with variable filter coefficients can be used to generate pixel information less than or equal to integer pixels in 1 / 8 pixel units.

[0056] The motion information generation unit can generate motion information based on a reference image interpolated by the reference image interpolation unit. Here, motion information refers to motion vectors, reference image indices, prediction directions, etc. Various methods such as FBMA (Block Matching Algorithm Based on Full Search), TSS (Three-Step Search), and NTS (New Three-Step Search Algorithm) can be used as methods for estimating motion vectors. Furthermore, the motion vector can have motion vector values ​​based on 1 / 2 pixel units or 1 / 4 pixel units of the interpolated pixels. In inter-frame prediction, the current prediction unit can be predicted by generating motion information in different ways. Various methods such as merging methods using motion vectors from neighboring blocks and motion estimation methods (e.g., AMVP (Adaptive Motion Vector Prediction)) can be used as methods for generating motion information.

[0057] For example, Figure 3 This is a block diagram schematically illustrating a motion estimation method according to an embodiment of the present invention. When a reference block that is the same as or similar to the prediction block in a reference image that has been encoded and decoded is determined, motion estimation will determine the motion vector, reference image index, and inter-frame prediction direction of the current block based on the determination.

[0058] When using the AMVP method, the encoding device can generate a predicted motion vector (MVP: motion vector prediction) by predicting the motion vector estimated in the current block, and can encode the difference between the motion vector and the generated predicted motion vector (MVD: motion vector difference).

[0059] The method using motion vectors from adjacent blocks involves applying motion information from neighboring blocks adjacent to the current block to the current block. In this case, neighboring blocks can include spatially adjacent blocks and temporally adjacent blocks that exist at the same location as the current block and are included in the reference image. The encoding device can achieve this by... Figure 4 The motion information of the current block's neighboring blocks (spatial neighboring blocks: A to E, temporal neighboring blocks: Col) shown in the image is applied to the current block to determine its motion information. Here, Col represents a block that has the same or similar position as the current block and exists in the reference image.

[0060] Intra-prediction unit 125 can generate prediction units based on reference pixel information surrounding the current block, where the reference pixel information is pixel information in the current image. When the neighboring block of the current prediction unit is the block to which inter-frame prediction is performed and the reference pixel is a pixel reconstructed through inter-frame prediction, the reference pixel included in the block to which inter-frame prediction is performed can be replaced by reference pixel information from the block to which intra-frame prediction is performed. That is, when a reference pixel is unavailable, the unavailable reference pixel information can be replaced by at least one of the available reference pixels.

[0061] In intra-frame prediction, the prediction mode can have a directional prediction mode that uses reference pixel information based on the prediction direction, and a non-directional mode that does not use directional information when performing prediction. The mode used to predict luminance information can be different from the mode used to predict chrominance information. Intra-frame prediction mode information or predicted luminance signal information used to predict luminance information can be used to predict chrominance information.

[0062] In intra-frame prediction methods, prediction blocks can be generated by applying an adaptive intra-frame smoothing (AIS) filter to a reference pixel based on the prediction mode. Different types of AIS filters can be applied to the reference pixel. In intra-frame prediction methods, the intra-frame prediction mode of the current PU can be predicted using the intra-frame prediction modes of neighboring PUs. During the process of predicting the prediction mode of the current PU using mode information predicted from neighboring PUs, when the current PU and neighboring PUs have the same intra-frame prediction mode, information indicating that the current PU and neighboring PUs have the same prediction mode can be sent using predetermined flag information. When the current PU and neighboring PUs have different prediction modes, information about the prediction mode of the current block can be encoded using entropy coding.

[0063] Furthermore, residual blocks including residual information can be generated, the residual information being the difference between the original block of the prediction unit and the prediction unit as predicted by the prediction unit generated by the prediction unit (120, 125). The generated residual blocks can be input to the transformation unit 130.

[0064] Transform unit 130 can use transformation methods such as DCT, DST, and Karhunen Loeve Transform (KLT) to transform the residual block, which includes the residual data. In this case, the transformation method can be determined based on the intra-prediction mode of the prediction unit used to generate the residual block. For example, DCT can be used for the horizontal direction, and DST can be used for the vertical direction, depending on the intra-prediction mode.

[0065] The quantization unit 135 can quantize the values ​​transformed to the frequency domain by the transformation unit 130. The quantization coefficients can be changed according to the importance of the block or image. The values ​​calculated by the quantization unit 135 can be provided to the dequantization unit 140 and the reordering unit 160.

[0066] Transform unit 130 and / or quantization unit 135 may be selectively included in image coding device 100. That is, image coding device 100 may perform at least one of transformation or quantization on the residual data of the residual block, or may encode the residual block by skipping both transformation and quantization. Even if no transformation or quantization is performed in image coding device 100, or neither transformation nor quantization is performed, the block input to input entropy coding unit 165 is generally referred to as a transform block.

[0067] The reordering unit 160 can reorder the coefficient values ​​relative to the quantized residual values.

[0068] The reordering unit 160 can transform the coefficients of a two-dimensional (2D) block into a one-dimensional (1D) vector using a coefficient scanning method. For example, the reordering unit 160 can use a predetermined scan type to scan DC coefficients into coefficients in the high-frequency region and transform them into a one-dimensional vector form.

[0069] Entropy coding unit 165 can perform entropy coding based on the value obtained by reordering unit 160. Various coding methods such as exponential Golomb coding, context-adaptive variable-length coding (CAVLC), or context-adaptive binary arithmetic coding (CABAC) can be used for entropy coding.

[0070] Entropy coding unit 165 can encode various information (such as residual coefficient information, block type information, prediction mode information, partitioning unit information, prediction unit information, transform unit information, motion vector information, reference image information, block interpolation information, filtering information, etc.) from reordering unit 160 and prediction units 120 and 125. In entropy coding unit 165, the coefficients of the transform block can be encoded based on flags indicating whether the value of the coefficient in units of sub-blocks within the transform block is zero, flags indicating whether the absolute value of the coefficient is greater than 1, and flags indicating whether the absolute value of the coefficient is greater than 2. Entropy coding unit 165 only encodes the sign of non-zero coefficients. Coefficients with absolute values ​​greater than 2 can be encoded by subtracting 2 from the absolute value.

[0071] The entropy encoding unit 165 can entropy encode the coefficient values ​​input from the reordering unit 160 to the encoding unit.

[0072] The dequantization unit 140 dequantizes the value quantized by the quantization unit 135, and the inverse transform unit 145 inverse transforms the value transformed by the transform unit 130. The reconstructed block can be generated by adding the residual value to the predicted PU. The residual value can be generated by the dequantization unit 140 and the inverse transform unit 145. The predicted PU can be predicted by the motion vector prediction unit, motion compensation unit, and intra-frame prediction unit of the prediction units 120 and 125.

[0073] The filter unit 150 may include at least one of a deblocking filter, an offset unit, and an adaptive loop filter (ALF).

[0074] Deblocking filters remove block distortion caused by boundaries between blocks in a reconstructed image. Whether to apply a deblocking filter to the current block can be determined based on the pixels included in several rows or columns of the block. When a deblocking filter is applied to a block, a strong or weak filter can be applied according to the desired deblocking filtering intensity. When horizontal and vertical filtering are performed while applying a deblocking filter, they can be performed in parallel.

[0075] The offset unit can apply an offset relative to the original image to the deblocked filtered image on a pixel-by-pixel basis. After dividing the image's pixels into a predetermined number of regions, the regions to which the offset will be applied can be determined. The offset can be applied to the determined regions by taking into account the edge information of each pixel or by the method of applying the offset to the determined regions.

[0076] ALF can perform filtering based on a comparison between the reconstructed image and the original image. Pixels included in the image can be divided into predetermined groups, the filter to be applied to each group can be determined, and differential filtering can be performed for each group. Information regarding whether to apply ALF can be transmitted by each coding unit (CU), and the shape and filter coefficients of the ALF to be applied to each block can be changed. Furthermore, regardless of the characteristics of the block, an ALF with the same form (fixed form) can be applied to the block.

[0077] The memory 155 can store reconstructed blocks or images output from the filter unit 150, and when performing inter-frame prediction, the stored reconstructed blocks or images can be provided to the prediction units 120 and 125.

[0078] Figure 2 This is a block diagram of an image decoding device according to an embodiment of the present invention.

[0079] Reference Figure 2The image decoding device 200 may include an entropy decoding unit 210, a reordering unit 215, an inverse quantization unit 220, an inverse transform unit 225, prediction units 230 and 235, a filter unit 240, and a memory 245.

[0080] When an image bitstream is input from an image encoding device, the input bitstream can be decoded in the reverse manner of the process in the image encoding device.

[0081] The entropy decoding unit 210 can perform entropy decoding in the reverse order of the entropy encoding process performed in the entropy encoding unit of the image encoding device. For example, various encoding methods such as Exponential Columbus, Context Adaptive Variable-Length Coding (CAVLC), and Context Adaptive Binary Arithmetic Coding (CABAC) can be applied according to the method performed by the image encoding device. In the entropy decoding unit 210, the coefficients of the transform block can be decoded based on flags indicating whether the value of the coefficient in units of sub-blocks within the transform block is zero, flags indicating whether the absolute value of the coefficient is greater than 1, and flags indicating whether the absolute value of the coefficient is greater than 2. Then, the entropy decoding unit 210 decodes the sign of the coefficients associated with non-zero coefficients. Coefficients with an absolute value greater than 2 can be decoded by subtracting 2.

[0082] The entropy decoding unit 210 can decode information associated with intra-frame prediction and inter-frame prediction performed by the encoding device.

[0083] The reordering unit 215 can reorder the bitstream entropy decoded by the entropy decoding unit 210. The reordering unit 215 can reconstruct and reorder the coefficients of a 1D vector into coefficients of a 2D block. The reordering unit 215 can be provided with information about the coefficient scan performed by the encoding device, and can perform reordering using a reverse scan coefficient method based on the scan order performed by the encoding device.

[0084] The dequantization unit 220 can perform dequantization based on the quantization parameters and the coefficients of the block reordering.

[0085] The inverse transform unit 225 can perform an inverse transform on the inverse-quantized transform coefficients based on a predetermined transform type. In this case, the transform type can be determined based on at least one of the prediction mode (inter-frame prediction / intra-frame prediction), block size / shape, and intra-frame prediction mode.

[0086] Prediction units 230 and 235 can generate prediction blocks based on information used to generate prediction blocks and information about previously decoded blocks or images. The information used to generate prediction blocks can be provided from entropy decoding unit 210. Information about previously decoded blocks or images can be provided from memory 245.

[0087] Prediction units 230 and 235 may include a PU determination unit, an inter-frame prediction unit, and an intra-frame prediction unit. The PU determination unit may receive various information from the entropy decoding unit 210 (such as PU information, information related to the intra-frame prediction mode of the intra-frame prediction method, information related to motion prediction of the inter-frame prediction method, etc.) and may determine the PU for the current CU. The PU determination unit may determine whether to perform inter-frame prediction or intra-frame prediction on the PU. The inter-frame prediction unit 230 may perform inter-frame prediction on the current PU based on information about at least one of the previous and subsequent images of the current image including the current PU. The inter-frame prediction unit 230 may use information required for inter-frame prediction of the current PU provided from the encoding device. Inter-frame prediction may be performed based on information about pre-reconstructed local regions in the current image including the current PU.

[0088] To perform inter-frame prediction, the method for generating motion information for the PUs included in the CU can be determined on a CU basis: either the merging mode method or the motion estimation method.

[0089] Intra-prediction unit 235 can generate prediction blocks based on pixel information in the current image. When the PU is the PU to which intra-prediction is performed, intra-prediction can be performed based on intra-prediction mode information about the PU provided from the encoding device. Intra-prediction unit 235 may include an AIS (Adaptive Intra-Smoothing) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter performs filtering on the reference pixels of the current block. The AIS filter can determine whether to apply the filter based on the prediction mode used for the current PU. AIS filtering can be performed on the reference pixels of the current block using the prediction mode used for the PU and the information about the AIS filter provided from the encoding device. When the prediction mode used for the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

[0090] When the prediction mode for the PU indicates a prediction mode that performs intra-frame prediction based on pixel values ​​obtained by interpolating reference pixels, the reference pixel interpolation unit can generate reference pixels in units of fractional pixels smaller than integer pixels (i.e., full pixels) by interpolating the reference pixels. When the prediction mode for the current PU indicates a prediction mode that generates a prediction block without interpolating reference pixels, the reference pixels may not be interpolated. When the prediction mode for the current block is DC mode, the DC filter can generate the prediction block by filtering.

[0091] The reconstructed blocks or image can be provided to the filtering unit 240. The filtering unit 240 includes a deblocking filter, an offset unit, and an ALF.

[0092] The image encoding device can provide information about whether a deblocking filter is applied to a corresponding block or image, and about which of the strong and weak filters is applied when the deblocking filter is used. The image encoding device can provide information about the deblocking filter to the image decoding device's deblocking filter, and the image decoding device's deblocking filter can perform deblocking filtering on the corresponding block.

[0093] The offset unit can apply an offset to the reconstructed picture based on information about the type and value of the offset applied to the picture during the encoding process.

[0094] ALF can be applied to the CU based on information such as whether an ALF is applied and ALF coefficient information provided from the encoding device. ALF information can be included and set in a specific set of parameters.

[0095] The memory 245 can store reconstructed images or blocks for use as reference images or reference blocks, and can provide reconstructed images to the output unit.

[0096] Figure 5 It is a diagram showing the brightness variation pattern between the current image and the reference image, including the current block.

[0097] In the case of inter-frame prediction for the current block, the greater the brightness change between the current image and the reference image, the greater the brightness change between the selected prediction block in the current block and the reference image. Therefore, it can be expected that the energy of the residual signal of the current block will increase as the error caused by the inter-frame prediction of the current block increases. Furthermore, as the energy of the residual signal increases, it can be expected that the error due to quantization will also increase. As a result, compared to the case where there is no brightness change, the error for the residual block will increase when there is a brightness change between the current image and the reference image.

[0098] Accordingly, this invention proposes a method for generating weighted prediction parameters by estimating brightness variations between images and using these parameters to perform inter-frame prediction. By using weighted prediction parameters in inter-frame prediction, the energy of residual blocks can be prevented from increasing drastically, and prediction efficiency can be improved.

[0099] The weight prediction parameters can be generated by considering brightness changes (such as fade-in or fade-out) between the current image and the reference image. However, when determining the weight prediction parameters by considering only the above factors, it is difficult to handle situations where the brightness of the current image and the reference image is changed by multiple light sources or where the brightness of only a local area in the current image changes. Accordingly, in this invention, a method for estimating the weight prediction parameters is also proposed, taking into account situations where the brightness is changed by multiple light sources or where the brightness of only a local area changes.

[0100] In the following sections, inter-frame prediction using weighted prediction parameters will be described in detail with reference to the accompanying drawings.

[0101] Figure 6 This is a flowchart illustrating a method for estimating weight prediction parameters in an image coding device.

[0102] The weight prediction parameters can be generated based on the brightness variation between the current image and the reference image. For ease of explanation, in the following embodiments, the weight prediction parameter generated based on the brightness variation between the current image and the reference image is referred to as the "first weight prediction parameter," and the weight prediction parameter generated based on the brightness variation between a portion of the current image and a portion of the reference image is referred to as the "second weight prediction parameter." The terms "first" and "second" for weight prediction parameters are used for ease of description only and do not imply that the first and second weight prediction parameters should have different properties.

[0103] Before encoding the image, the encoding device can use the current image and the reference image to estimate a first weight prediction parameter for the reference image (S601). For example, the encoding device can assign the first weight prediction parameter to each of a plurality of reference images included in the list of reference images.

[0104] The first weighted prediction parameter is a value generated based on the brightness change between the reference image and the current image.

[0105] When the first weighted prediction parameter is used in the inter-frame prediction of the current block, the prediction block can be generated based on a reference block indicated by a motion vector and the first weighted parameter for a reference image that includes the reference block.

[0106] The first weighted prediction parameter may include at least one of a multiplication parameter multiplied by the predicted pixel or an addition parameter added to the predicted pixel. In this case, the multiplication parameter and the addition parameter can be derived based on regression analysis. For example, equation (1) below shows an example of a regression analysis model.

[0107] [Equation (1)]

[0108] e 2 =∑[Y-(wX+o)] 2

[0109] In equation (1), Y represents the data of the current image, X represents the data of the reference image, w represents the slope of the regression line, o represents the intercept value of the regression line, and e represents the regression line prediction error. In this case, Y is the pixel value of the current image, and the entirety or a portion of the current image can be a range. X can be the pixel value of the reference image, and the entirety or a portion of the reference image can be a range.

[0110] The first weighted prediction parameter can be obtained by partially dividing equation (1) into w and o, respectively. For example, when equation (1) is partially divided into w and o, w and o that minimize the square of the error (e) can be set as multiplication parameter and addition parameter, respectively.

[0111] The first weight prediction parameter value calculated based on Equation (1) can have a real value. The first weight prediction parameter can be set to a real value calculated based on Equation (1) or an integer value obtained by integerizing a real value calculated based on Equation (1). In one example, the first weight prediction parameter can be derived as an integer value obtained by multiplying a real value calculated based on Equation (1) by 2N. The variable N used to integerize the first weight prediction parameter can be encoded block-by-block, region-by-region, or in the upper header.

[0112] When the first weight parameter is determined, the encoding device can calculate the cost based on whether the first weight parameter is applied (S602). For example, the encoding device can calculate the sum of absolute differences (SAD) between the current image and a reference image to which the first weight parameter is applied as a whole, as well as the SAD between the reference image without the first weight parameter and the current image. If the SAD of the reference image with the first weight parameter applied is smaller than the SAD of the reference image without the first weight parameter applied, it can be determined that there is a brightness change between the current image and the reference image. Conversely, if the SAD of the reference image with the first weight parameter applied is larger than the SAD of the reference image without the first weight parameter applied, it can be determined that there is no brightness change (S603).

[0113] In addition to the SAD described above, the sum of squared differences (SSD) or the sum of absolute transformation differences (SATD) can also be used to perform cost calculations.

[0114] If it is determined that there is no brightness change between the current image and the reference image (S603), then it can be determined that the weighted prediction parameters will not be used in the inter-frame prediction for the current image. Accordingly, inter-frame prediction can be used for the current block included in the current image without using the weighted parameters.

[0115] On the other hand, if it is determined that there is a brightness change between the current image and the reference image (S603), a process can be performed to derive a second weighting parameter for each predetermined region in the current image. For example, the second weighting parameter for the predetermined region in the current image can be obtained by comparing the pixel values ​​of a block included in the predetermined region of the current image with the pixel values ​​of a block included in the region of the reference image. Here, the region of the reference image corresponds to the same location as the predetermined region in the current image.

[0116] To calculate the second weighted prediction parameters for a predetermined region in the current image, the current image and the reference image can be divided into multiple regions (S604). The current image and the reference image can be divided into multiple blocks of the same size. The current image and the reference image can be divided in the same way, such that the number of blocks in the current image and the position of the blocks can be set to be the same.

[0117] Then, the encoding device can determine whether the current image includes at least one region in the current image where a local brightness change occurs, based on the pixel values ​​of each block in the current image and the reference image (S605).

[0118] In step S605, the at least one region may consist of a set of blocks having the same or approximately the same pixel value ratio. For example, if the average pixel value of the first block divided in the current image is 100 and the average pixel value of the first block divided in the reference image is 80, then the brightness difference between the first block in the current image and the first block in the reference image is 1.25 times. The encoding device may group blocks that have a brightness difference of 1.25 times relative to the reference block, or group blocks that have a brightness difference of approximately 1.25 times relative to the first block. In this way, by grouping blocks with the same or approximately the same brightness difference, regions with local brightness variations can be identified.

[0119] In other words, the encoding device can compare the pixel values ​​of blocks included in the current image with the pixel values ​​included in the reference image, and can group blocks with the same or similar pixel value ratios. Grouped blocks can be treated as a single region.

[0120] The encoding device can generate second weight prediction parameters for at least one region included in the current image (S606). If multiple regions exist in the current image, multiple second weight prediction parameters can be generated. Each region can be generated as a set of blocks with approximate pixel value ratios.

[0121] Equation (1) can be used to obtain a second weight prediction parameter for each region. In this case, Y can be data of a predetermined region in the current image where the second weight prediction parameter will be estimated, and X can be data of a region in a reference image. Here, the region in the reference image can be a region at the same location as the predetermined region in the current image. For example, Y can be a pixel value within the predetermined region, and all or some of the region can be a range, and X can be a pixel value within the same location region, and all or some of the region can be a range. The value of the second weight prediction parameter calculated based on Equation (1) can have a real value. The second weight prediction parameter can be set to a real value calculated based on Equation (1) or set to a value obtained by integerizing the real value calculated based on Equation (1). In one example, the second weight prediction parameter can be derived as an integer value by multiplying the real value calculated based on Equation (1) by 2N. The variable N used for integerizing the second weight prediction parameter can be encoded block by block, region by region, or in the upper header. It is also possible to have the same value of N as used for integerizing the first weight prediction parameter and to use a different N.

[0122] Figure 7 This is a diagram illustrating an example of using weight parameters to perform rate-distortion optimization (RDO) on prediction blocks.

[0123] like Figure 6 As shown, when there is a brightness change between the current image and the reference image, the encoding device can obtain a first weight prediction parameter, and when there is a region in the current image where there is a local brightness change, the encoding device can obtain a second weight prediction parameter (S701). That is, depending on whether there is a brightness change or a local brightness change between the current image and the reference image, the encoding device can obtain candidate weight prediction parameters (such as the first weight prediction parameter or the second weight prediction parameter) that can be applied to the current block.

[0124] If there is a brightness variation between the current block and the reference image, the candidate weight prediction parameters for the current block may include a first weight prediction parameter. Furthermore, when there are regions in the current image where local brightness variations exist, the candidate weight prediction parameters for the current block may also include a second weight prediction parameter. If multiple second weight prediction parameters exist for the current image, the candidate weight prediction parameters may include multiple second weight prediction parameters.

[0125] The encoding device can apply the set of weighted parameters as candidates for weighted prediction parameters to the current block and calculate the corresponding cost (S702). Then, the encoding device can determine the optimal weighted prediction parameters for the current block based on the calculation results (S703).

[0126] Determining the weight prediction parameters for the current block corresponds to selecting one of several weight prediction parameter candidates that best represents the inter-frame prediction performance of the current block. For example, if a first weight prediction parameter is derived for a reference image and a second weight prediction parameter is derived for a predetermined region in the current image, the best weight prediction parameter for the current block can be selected from the first and second weight prediction parameters.

[0127] For this purpose, the encoding device can select the optimal weight prediction parameter by comparing the inter-frame prediction results of each of the candidate weight prediction parameters. For example, the encoding device can determine the optimal weight prediction parameter for the current block based on the inter-frame prediction performance results of a first weight prediction parameter and a plurality of second weight parameters used for the current block.

[0128] The decision to use weighted prediction parameters for inter-frame prediction can also be made by comparing the results of inter-frame prediction performed with candidates who did not use weighted prediction parameters.

[0129] In the example above, the weight prediction parameters are described as including a first weight prediction parameter, and in some cases, the weight prediction parameters may also include a second weight prediction parameter. Conversely, in the example above, the first weight prediction parameter may be used as a weight prediction parameter candidate only when the second weight prediction parameter is unavailable or when the number of second weight prediction parameters is equal to or less than a predetermined number.

[0130] The number of weight prediction parameter candidates can be fixed or variable. When the number of weight prediction parameter candidates is variable, the encoding device can encode information indicating the number of available weight prediction parameter candidates for the current block via a bit stream (e.g., an upper header). In this case, the number of weight prediction parameter candidates can be variably set according to the size or depth of the current block. Accordingly, the encoding device can encode information about the number of weight prediction parameter candidates that can be used based on the block size or depth information via a bit stream (e.g., an upper header).

[0131] As another example, the encoding and decoding devices can limit the use of the same number of weight prediction parameter candidates by predetermined conditions. For example, the number of weight prediction parameter candidates can be adaptively determined based on the size, shape, or intra-prediction mode of the current block. Suppose the number of available weight prediction parameters is 5. When the current block size is 8x8 or smaller, five weight prediction parameter candidates can be used. When the current block size is 16x16, four weight prediction parameter candidates can be used. When the current block size is 32x32, three weight prediction parameter candidates can be used. When the current block size is 64x64, two weight prediction parameter candidates can be used.

[0132] If a second weight prediction parameter is not obtained for the current image, the first weight prediction parameter can be used to encode the current block. Furthermore, if there is no brightness variation between the current image and the reference image, the current block can be encoded without using the weight prediction parameter, or the initial weight prediction parameter can be used to encode the current block.

[0133] The initial weight prediction parameters refer to setting the multiplication and addition parameters to their initial values. Here, the initial values ​​of the multiplication and addition parameters can be 1 and 0, respectively. As another example, the initial value of the multiplication parameter can be set to 1 << N, and the initial value of the addition parameter can be set to zero.

[0134] In the example above, the initial weight prediction parameters can be shown as being used when there is no brightness change between the current image and the reference image. Even when there is a brightness change between the current image and the reference image, the initial weight prediction parameters can still be used to encode the current block. In this case, to select the initial weight prediction parameters, they can be added as candidates for weight prediction parameters. By adding the initial weight prediction parameters as candidates, a new index can be assigned to the initial weight prediction parameters. In this case, the index assigned to the initial weight prediction parameters can be encoded via the bitstream or can have a predetermined value.

[0135] Furthermore, by performing RDO using a list of weight prediction parameter candidates, including a first weight prediction parameter, a second weight prediction parameter, and an initial weight prediction parameter, weight prediction parameters suitable for the current block can be specified among the first weight prediction parameter, the second weight prediction parameter, and the initial prediction parameter.

[0136] Next, we will describe a method for encoding information related to the weight prediction parameters.

[0137] Figure 8 This is a flowchart illustrating a method for encoding information related to the weight prediction parameters used for the current block.

[0138] Reference Figure 8The encoding device can encode information about whether there is a brightness change in each reference image used for the current image (S801). Whether to use weighted prediction parameters depends on whether there is a brightness change, so the information can be used to indicate whether there is a weighted prediction parameter in the bitstream. At this time, the information about whether there is a brightness change can be a 1-bit flag, but is not limited to this. Furthermore, the information about whether there is a brightness change can be encoded in units of prediction blocks, or it can be encoded in a header higher than the prediction blocks. In one example, the information can be encoded in units of sequences, frames, stripes, or tiles. Based on the information to be decoded from the bitstream, the decoding device can determine whether there is a brightness change between the current image and the reference images.

[0139] When there is no brightness variation between the current image and the reference image, the encoding device does not perform encoding of information related to the weight prediction parameters used for the current block.

[0140] On the other hand, if there is a brightness change between the current image and the reference image (S802), the encoding device can encode the first weight prediction parameter and information about whether the current image includes a region with at least one second weight prediction parameter (S803, S804). When the current image includes a region assigned a second weight prediction parameter, information about the second weight prediction parameter is further encoded. Therefore, this information can be used to indicate whether there are other weight prediction parameters besides the first weight prediction parameter. Here, the information about whether the current image includes at least one region used to derive the second weight prediction parameter can be a 1-bit flag, but is not limited to this. Furthermore, information about the existence of a brightness change can be encoded in a header higher than the prediction block. In one example, the information can be encoded in units of sequence, frame, strip, or parallel block.

[0141] If it is determined that the current image includes at least one region to which at least one second weight prediction parameter has been assigned (S805), the encoding device may encode the second weight prediction parameter for each of the M regions to be included in the current image (S806).

[0142] Information about the first and second weight prediction parameters can be encoded at the prediction block level, or it can be encoded in a header higher than the prediction block level. For example, information about the first and second weight prediction parameters can be encoded at the sequence, frame, stripe unit, or parallel block level.

[0143] The first and second weight prediction parameters can also be encoded in different layers. For example, the first weight prediction parameter can be encoded based on the video, and the second weight prediction parameter can be encoded based on the strip.

[0144] The first and second weight prediction parameters may include multiplication parameters and addition parameters. In this case, the multiplication parameter can be divided into a predicted value determined based on the precision N of the multiplication parameter and the difference between the multiplication parameter and the predicted value. For example, 1 << N can be set as the predicted value of the multiplication parameter, and the difference between the multiplication parameter and the predicted value (1 << N) can be set as the difference. The encoding device can encode information about the multiplication parameter by encoding the difference. The precision N of the current block can be encoded in units of blocks, stripes, or frames, and can be transmitted to the decoding device via a bitstream.

[0145] Alternatively, the precision N of the current block can be determined in the same manner in both the encoding and decoding devices. For example, the precision N of the current block can be adaptively determined based on the size, type, etc. of the current block.

[0146] If it is determined that the current image does not include a region assigned at least one second weight prediction parameter, then the encoding for the second weight prediction parameter can be omitted.

[0147] Information “M” regarding the number of regions included in the current block (i.e., the number of second weight prediction parameters or the number of regions from which the second weight prediction parameters can be derived) can be encoded block-by-block or in the upper header. Optionally, the encoding and decoding devices can determine the number of regions included in the current block in the same manner based on predetermined conditions.

[0148] If it is determined that the current image includes at least one region for deriving the second weight prediction parameter, then the index information indicating the weight prediction parameter of the current block among multiple weight prediction parameter candidates can be encoded (S807). At this time, the index information can be encoded on a per-prediction-block basis.

[0149] As another example, the encoding device can encode information about regions that identify derived weight parameters and weight prediction parameters. For instance, the encoding device can encode the location or size of the regions that derive second weight prediction parameters and second weight parameters. The encoding device can also encode the index assigned to said regions.

[0150] Assuming the index of the weight prediction parameter candidates starts from index 0, the index information will indicate any one from 0 to the number of weight prediction parameter candidates - 1. Here, as described in the example above, the number of weight prediction parameter candidates can have a fixed value or a variable value.

[0151] Multiple weight prediction parameter candidates may include at least one of a first weight prediction parameter, a second weight prediction parameter, or an initial weight prediction parameter.

[0152] The following section will describe in detail an example of decoding information about the weight prediction parameters in a decoding device.

[0153] Figure 9 This is a diagram illustrating an example of decoding weight prediction parameters in a decoding device. (See also...) Figure 9 The decoding device can decode information from the bitstream indicating whether the brightness of each reference image has changed compared to the current image (S901).

[0154] Based on the information, the decoding device can determine whether a first weight prediction parameter exists for the current block (S902). For example, if it is determined based on the information that there is no brightness change between the current image and the reference image, the decoding device may not decode the information related to the weight prediction parameter.

[0155] On the other hand, if it is determined that there is a brightness change between the current image and the reference image (S902), the decoding device can decode the first weight prediction parameter and information indicating whether the current image includes at least one region from which the second weight prediction parameter can be derived (S903, S904).

[0156] If the current image includes at least one region from which a second weight prediction parameter can be derived, the decoding device can decode the information related to the second weight prediction parameter from the bitstream (S905). Based on the information, the decoding device can determine whether a second weight prediction parameter exists for the current block.

[0157] If there are multiple (M) regions in the current image from which the second weight prediction parameters can be derived, the decoding device can decode the information about the M second weight prediction parameters (S906).

[0158] At this point, information about the first and second weight prediction parameters can be decoded on a block-by-block basis, or in a header higher than the prediction block. For example, information about the first and second weight prediction parameters can be decoded on a sequence, frame, stripe unit, or parallel block basis.

[0159] The first and second weight prediction parameters can be decoded in different layers. For example, the first weight prediction parameter can be decoded based on the video, while the second weight prediction parameter can be decoded based on the strip.

[0160] The first and second weight prediction parameters may include multiplication parameters and addition parameters. In this case, based on the precision N of the multiplication parameters, the decoding device can decode information indicating the difference between the predicted values ​​of the multiplication parameters and the multiplication parameters. If the precision used for the multiplication parameters is N, then (1 << N) can be set as the predicted value of the multiplication parameters.

[0161] Subsequently, the decoding device can decode the index information of the weight prediction parameters for the current block from among a plurality of weight prediction parameter candidates (S907). When the weight prediction parameters for the current block are specified through the index information, inter-frame prediction for the current block can be performed based on the specified weight prediction parameters.

[0162] Specifically, the decoding device can obtain a first predicted pixel by performing inter-frame prediction on the current block, and obtain a second predicted pixel by applying weighted prediction parameters to the obtained first predicted pixel. In one example, the second predicted pixel can be obtained by multiplying the first predicted pixel by a multiplication parameter and adding an addition parameter.

[0163] As another example, the decoding device can decode weight prediction parameters and information identifying the regions from which the weight prediction parameters are derived. For example, the decoding device can decode second weight prediction parameters and the location or size of the regions to which the second weight prediction parameters are assigned. The decoding device can decode the index assigned to said region. In this case, the decoding device can adaptively select weight prediction parameters based on whether the current block is included in a region to which the second weight prediction parameters are assigned. For example, when the current block is included in a region not to which the second weight prediction parameters are assigned, the decoding device can use the first weight prediction parameters to select weight prediction parameters for the current block. On the other hand, when the current block is included in a region to which the second weight prediction parameters are assigned, the decoding device can use the second weight prediction parameters assigned to the region including the current block to perform inter-frame prediction on the current block.

[0164] In the above embodiments, whether the first weight prediction parameter can be used as a weight prediction parameter candidate is determined based on information indicating whether there is a brightness change between the current image and the reference image. It has been described that whether the second weight prediction parameter can be used as a weight prediction parameter candidate is determined based on whether the current block includes a region from which the weight prediction parameter can be derived. That is, whether to add the second weight prediction parameter to the weight prediction parameter candidate is determined based on whether the current image includes a region from which the weight prediction parameter can be derived.

[0165] In contrast to the example above, when there is a brightness variation between the current image and the reference image, the encoding device determines the number of available weight prediction parameter candidates as the number of available weight prediction parameter candidates (rather than information indicating whether the current image includes a second weight prediction parameter), or the number of available weight prediction parameter candidates can be encoded.

[0166] In this scenario, the decoding device can decode at least one or more weight prediction parameter candidates based on the received quantity information. For example, when the number of available weight prediction parameters is N, the weight prediction parameter candidates can be configured using one first weight prediction parameter and (N-1) second weight prediction parameters.

[0167] In the above embodiments, the weight prediction parameters include a first weight prediction parameter and a second weight prediction parameter. The first weight prediction parameter is generated based on the brightness difference between the current image and the reference image, and the second weight prediction parameter is generated based on the brightness difference between a portion of the current image and a portion of the reference image. However, the first and second weight prediction parameters described above are merely illustrative examples of generating weight prediction parameters, and the present invention is not limited thereto.

[0168] Next, we will describe in detail the method of performing inter-frame prediction using a set of weighted prediction parameters.

[0169] Figure 6 and Figure 10 This is a flowchart illustrating a method for using weighted prediction parameters in an encoding device.

[0170] The encoding device may generate a set of weight prediction parameters for the current block based on the weight prediction parameters of blocks encoded prior to the current block to be encoded (S6S1001). Here, the set of weight prediction parameters may include the weight prediction parameters of blocks encoded prior to the current block that used inter-frame prediction.

[0171] The encoding device can generate a set of weight prediction parameters based on the weight prediction parameters used in neighboring blocks adjacent to the current block in blocks encoded before the current block. Here, neighboring blocks adjacent to the current block can include spatially adjacent blocks and temporally adjacent blocks. For example, spatially adjacent blocks can include the upper-left block, the upper block, the upper-right block, the left block, and the lower-left block adjacent to the current block, and temporally adjacent blocks can include collocated blocks existing in the same position as the current block. Collocated blocks exist in the reference image.

[0172] The encoding device can construct a set of weight prediction parameters based on the weight prediction parameters of the reconstructed block used to encode the block preceding the current block and the weight prediction parameters of the prediction block corresponding to the reconstructed block.

[0173] Weight prediction parameters can be set differently depending on the inter-frame prediction direction. For example, forward weight prediction parameters can be used when performing forward prediction on the block to be encoded, and backward weight prediction parameters can be used when performing backward prediction on the block to be encoded. When encoding blocks by performing bidirectional prediction, both forward and backward weight prediction parameters can be used. In this case, the forward direction indicates a past reference image of the current image (i.e., a reference image with a POC smaller than the current image's POC), and the opposite direction indicates a future reference image of the current image (i.e., a reference image with a POC larger than the current image's POC).

[0174] Based on the inter-frame prediction direction of the current block, the coding device can construct the set of weight prediction parameters for the current block by using at least one of the forward weight prediction parameters and backward weight prediction parameters used for adjacent blocks.

[0175] Figure 7 and Figure 11 This is a diagram showing the weight prediction parameters of the adjacent blocks that are adjacent to the current block.

[0176] exist Figure 7 and Figure 11 In the example shown, A to E represent the spatially adjacent blocks of the current block, and Col represents the temporally adjacent block of the current block. For information on the positions of A to E and Col, please refer to [reference needed]. Figure 4 The example shown.

[0177] Reference Figure 7 and Figure 11 The adjacent blocks have at least one of the forward weight prediction parameters (i.e., the weight parameters assigned to list 0) and the backward weight prediction parameters (i.e., the weight parameters assigned to list 1).

[0178] Based on the prediction direction of the current block, the encoding device can determine the weight prediction parameters of neighboring blocks to be included in the weight prediction parameter set. For example, if the current block is encoded using forward prediction, the weight prediction parameter set can be constructed using the forward weight prediction parameters of neighboring blocks; and if the current block is encoded using backward prediction, the weight prediction parameter set can be constructed using the backward weight prediction parameters of neighboring blocks. If the current block is encoded using bidirectional prediction, the weight prediction parameter set can be constructed using both the forward and backward weight prediction parameters of neighboring blocks.

[0179] For example, in Figure 7 and Figure 11In the example shown, if the current block is encoded in the forward prediction, the forward weight prediction parameters (65, 1), (64, 2), (66, -2), (61, 7), (59, -2) of the adjacent blocks can be included in the set of weight prediction parameters.

[0180] Here, the weight prediction parameters may include at least one of the multiplication parameter multiplied by the predicted sample or the addition parameter added to the predicted sample. Figure 7 and Figure 11 In this context, the weighted prediction parameters are represented in the form (w, o). Here, w represents the multiplication parameter and o represents the addition parameter.

[0181] The encoding device can select the best weight prediction parameter for the current block from the weight prediction parameters included in the weight prediction parameter set (S6S1002). When the best weight prediction parameter for the current block is selected, the encoding device can encode the information (e.g., index information) used to specify the selected weight prediction parameter.

[0182] When the optimal weight prediction parameters are selected, inter-frame prediction for the current block can be performed based on the selected weight prediction parameters (S6S1003). For example, inter-frame prediction for the current block can be performed by multiplying the predicted pixels obtained via motion compensation by a multiplication parameter and then adding an addition parameter.

[0183] If the prediction block is generated as an inter-frame prediction result for the current block, the current block can be reconstructed based on the prediction block and residual block generated after inverse quantization and inverse transform (S6S1004).

[0184] Assuming that the block to be encoded adjacent to the current block is encoded based on inter-frame prediction, the set of weight prediction parameters for the adjacent blocks can be generated using the weight prediction parameters of the current block. In this case, the weight prediction parameters of the current block that can be used for the adjacent blocks can be any one selected from the set of weight prediction parameters of the current block.

[0185] As another example, the weight prediction parameters of the current block that can be used for neighboring blocks can be estimated from the reconstructed block and the predicted block of the current block. In this case, the encoding device can perform weighted prediction parameter estimation on the current block in order to configure the set of weight prediction parameters for the neighboring blocks of the current block (S6S1005).

[0186] The reconstructed block has a quantization error relative to the original block, but the brightness variation characteristics in the reconstructed block have a pattern similar to that of the original block. Accordingly, in the decoding device, the weight prediction parameters for the current block can be estimated using the reconstructed block and the prediction block of the current block, so as to estimate the weight prediction parameters in the same way as in the encoding device.

[0187] Specifically, if a reconstructed block is generated for the current block, the weight prediction parameters of the current block can be estimated for neighboring blocks by using regression analysis based on the reconstructed block and the predicted block without applied weight prediction parameters. An example of regression analysis is shown in Equation (1).

[0188] In equation (1), Y represents the data of the reconstructed block, X represents the data of the prediction block, w represents the slope of the regression line, o represents the intercept value of the regression line, and e represents the error of the regression line prediction. Specifically, Y represents the pixel value of the reconstructed block, and X represents the pixel value of the prediction block.

[0189] The weight prediction parameters for the current block can be obtained by partially dividing equation (1) into w and o respectively. For example, when equation (1) is partially divided into w and o respectively, w and o that minimize the square of error e can be derived as multiplication parameters and addition parameters respectively.

[0190] The weight prediction parameters calculated based on Equation (1) can have real values. The weight prediction parameters for the current block can be set to real values ​​calculated based on Equation (1), or they can be set to integer values ​​obtained by integerizing the real values ​​calculated based on Equation (1). In one example, the weight prediction parameters can be derived as integer values ​​derived by multiplying the real values ​​calculated based on Equation (1) by 2N. The variable N used to integerize the weight prediction parameters can be encoded via a bitstream. Alternatively, the weight prediction parameters can be integerized by using a predetermined value of N with the same value in both the encoding and decoding devices.

[0191] In Equation (1), all pixels in the reconstruction block are set as the input range for X, and all pixels in the prediction block can be set as the input range for Y. As another example, only some of the subsampled pixels (instead of using the entire block) can be used to estimate the weighted prediction parameters of the current block.

[0192] For example, Figure 8 and Figure 12 This is a diagram used to illustrate an example where only a few pixels are used to estimate the weighted prediction parameters for the current block. Assuming the reconstructed and predicted blocks of the current block are subsampled by 1 / 2 along the horizontal and vertical directions, one of the four samples (2x2) can be used as the input to X or Y. For example, in... Figure 8 and Figure 12 In the example shown, only black pixels can be used for regression analysis.

[0193] exist Figure 8 and Figure 12In the example shown, although the top-left pixel of the 2x2 sample is shown as being used for regression analysis, the top-right, bottom-left, or bottom-right pixels within the 2x2 pixel array could also be set to be used for regression analysis. Optionally, index information can be encoded via a bitstream. The index information can indicate which of the aforementioned locations was used for regression analysis.

[0194] The encoding and decoding devices can perform subsampling based on predefined values, or based on values ​​obtained under predefined conditions. Optionally, the encoding device can encode the subsampling unit M and can encode the subsampling unit M via a bit stream, thereby notifying the decoding device that 1 / M subsampling has been performed.

[0195] The weight prediction parameters estimated for the current block can be used to construct a set of weight prediction parameters for neighboring blocks.

[0196] The set of weight prediction parameters for the current block may include at least one weight prediction parameter. The maximum number of weight prediction parameters included in the set of weight prediction parameters for the current block may be a fixed number, or it may be a variable number depending on the size or type of the current block or the information signaled via a bitstream. In one example, the maximum number P of weight prediction parameters that the set of weight prediction parameters may include can be encoded via a bitstream.

[0197] exist Figure 7 and Figure 11 In this context, although the blocks used to construct the set of weight prediction parameters for the current block are restricted to A through E and Col, blocks at other locations can also be used to construct the set of weight prediction parameters for the current block.

[0198] For example, the encoding device can use blocks at predetermined positions to generate the weight prediction parameter set for the current block. Alternatively, the encoding device can use blocks at arbitrary positions to generate the weight prediction parameter set for the current block. The encoding device can encode information about the position of the block specified for generating the weight prediction parameter set for the current block, or the block itself, via a bitstream. In this case, the information about the position of the specified block, or the block itself, can be encoded via a bitstream.

[0199] If the set of weight prediction parameters includes multiple weight prediction parameters, the weight prediction parameters can be arranged in a predetermined order. In one example, the weight prediction parameters can be ordered according to spatially adjacent blocks and temporally adjacent blocks. Accordingly, in Figure 7 and Figure 11In this process, after adding the weight prediction parameters of A through E to the weight prediction parameter set, the weight prediction parameters of Col can be added to the weight prediction parameter set. As another example, the weight prediction parameters can be arranged in order of their most similarity to the motion information (e.g., motion vectors) of the current block. Optionally, higher priority can be assigned to the weight prediction parameters derived from adjacent reconstructed blocks and prediction blocks, and these weight prediction parameters can be added to the weight prediction parameter set. Conversely, higher priority can also be assigned to the weight prediction parameters of adjacent blocks, and these weight prediction parameters can be added to the weight prediction parameter set.

[0200] The set of weight prediction parameters may include an initial set of weight prediction parameters. The initial weight prediction parameters refer to the multiplication and addition parameters being set to their initial values. Here, the initial values ​​of the multiplication and addition parameters can be 1 and 0, respectively. As another example, the initial value of the multiplication parameter can be set to 1 << N, and the initial value of the addition parameter can be set to zero.

[0201] Information about the initial weight prediction parameters can be encoded via a bitstream and transmitted to a decoding device. Optionally, the initial weight prediction parameters can be set to a fixed index in the weight prediction parameter set. In one example, the initial weight prediction parameters can be set to index 0 in the weight prediction parameter set.

[0202] The set of weight prediction parameters may include weight prediction parameters derived from images. Here, the set of weight prediction parameters derived on an image-by-image basis may be generated based on the brightness variation between the current image and a reference image of the current image. For example, the weight prediction parameters derived in an image cell can be derived using equation (1). In this case, all pixels in the current image are set as the input range for X, and all pixels in the reference image can be set as the input range for Y.

[0203] Information about the weight prediction parameters of an image unit can be encoded via a bitstream and transmitted to a decoding device. Here, the information about the weight prediction parameters may include weight prediction parameter values ​​or residual values ​​used for the weight prediction parameters. For example, in the decoding device, the weight prediction parameters of an image unit can be obtained by adding the predicted values ​​of the weight prediction parameters and the residual values. In this case, the predicted values ​​of the weight prediction parameters can be determined by the precision N used for the current block, and the residual values ​​can be determined based on the information decoded from the bitstream.

[0204] At this point, the precision N of the current block can be encoded via a bitstream and transmitted to the decoding device via the bitstream.

[0205] Alternatively, the precision N of the current block can be determined in the same manner in both the encoding and decoding devices. For example, the precision N of the current block can be adaptively determined based on the size, shape, etc., of the current block.

[0206] When the weight prediction parameters of a picture unit are included in a weight prediction parameter set, the indices of the weight prediction parameters assigned to the picture unit can be encoded via a bitstream or can have predetermined values. For example, assuming the weight prediction parameter set includes up to six weight prediction parameters, the initial weight prediction parameters can be set to have index 0, and the weight prediction parameters of the picture unit can be set to have index 1. Indices 2 through 5 can be used for weight prediction parameters derived from neighboring blocks.

[0207] When the number of weight prediction parameters derived from neighboring blocks is less than the maximum number that can be included in the weight prediction parameter set, the initial weight prediction parameter or the weight prediction parameter of the image unit can be added to the weight prediction parameter set. For example, if four weight prediction parameters are available, but four weight prediction parameters cannot be derived from neighboring blocks, then at least one of the initial weight prediction parameter or the weight prediction parameter of the image unit can be included in the weight prediction parameter set. Four weight prediction parameters cannot be derived from neighboring blocks if the neighboring blocks adjacent to the current block are encoded via intra-frame prediction or if the weight prediction parameters of the neighboring blocks are unavailable because the current block is at an image boundary.

[0208] The set of weight prediction parameters may include combined weight prediction parameters generated by combining two or more weight prediction parameters. For example, when the number of weight prediction parameters that can be derived from adjacent blocks is less than the number of available weight prediction parameters, combined weight prediction parameters generated by combining two or more weight prediction parameters may be added to the set of weight prediction parameters.

[0209] Assuming the weight prediction parameters consist of (w, o), then the prediction patterns or prediction directions of adjacent blocks can be considered to generate combined weight prediction parameters.

[0210] For example, Figure 9 and Figure 13 An example of the weighted prediction parameters for generating the combination is shown.

[0211] exist Figure 9 and Figure 13 In the diagram, A to D represent the neighboring blocks adjacent to the current block, and list 0 and list 1 represent the weight prediction parameters of the neighboring blocks when using the forward reference image and the backward reference image, respectively.

[0212] exist Figure 9 and Figure 13In the diagram, A has only forward weight prediction parameters by performing forward prediction, and D has only backward weight prediction parameters by performing backward prediction. Since C performs bidirectional prediction, C has both forward and backward weight prediction parameters. B is shown as having neither forward nor backward weight prediction parameters. In one example, if B is encoded via intra-frame prediction, then B will have no weight prediction parameters.

[0213] Accordingly, in Figure 9 and Figure 13 In the example shown, only the forward weight prediction parameters (i.e., the weight prediction parameters for list 0) can be obtained from blocks A and C, and only the backward weight prediction parameters (i.e., the weight prediction parameters for list 1) can be obtained from blocks C and D.

[0214] Assuming the set of weight prediction parameters for the current block may include four forward weight prediction parameters, the remaining two forward weight prediction parameters, besides the two derived from adjacent blocks, can be generated by combining two or more forward weight prediction parameters. Alternatively, when assuming the set of weight prediction parameters for the current block may include four backward weight prediction parameters, the remaining two backward weight prediction parameters, besides the two derived from adjacent blocks, can be generated by combining two or more backward weight prediction parameters. Figure 9 and Figure 13 In the above, the forward weight prediction parameters for B and D are generated by combining the forward weight prediction parameters for A and C. The backward weight prediction parameters for A and B are generated by combining the backward weight prediction parameters for C and D.

[0215] When combined weight prediction parameters are added to the weight prediction parameter set, the index assigned to the combined weight prediction parameters can be determined by a predetermined priority.

[0216] If combining the weight prediction parameters in adjacent blocks does not generate a set of weight prediction parameters, then all remaining weight prediction parameters can be set as the initial weight prediction parameters.

[0217] Figure 10 and Figure 14 An example of decoding weight prediction parameters in a decoding device is shown. (See reference...) Figure 10 and Figure 14 The decoding device can generate a set of weight prediction parameters based on the weight prediction parameters of neighboring blocks adjacent to the current block in blocks decoded before the current block (S10S1401). The generation of weight prediction parameters has already been described in detail in the operation of the encoding device, and a detailed description of the generation of weight prediction parameters will be omitted.

[0218] When the set of weight prediction parameters for the current block is generated, the decoding device can determine the weight prediction parameters for the current block based on the index information (S10S1402). The index information can specify the weight prediction parameters for the current block. The specified weight prediction parameter can be any one of the weight prediction parameters included in the set of weight prediction parameters. Here, the index information can be decoded from the bitstream.

[0219] When the weight prediction parameters for the current block are determined, inter-frame prediction for the current block can be performed using the determined weight prediction parameters (S10S1403). As an example, inter-frame prediction for the current block can be performed by multiplying the prediction sample obtained via motion compensation by the multiplication parameter and adding the addition parameter.

[0220] When a predicted block is generated as an inter-frame prediction result for the current block, the current block can be reconstructed based on the generated predicted block and residual block (S10S1404).

[0221] When the reconstruction of the current block is completed, the weight prediction parameters for the current block can be estimated so that the blocks to be decoded adjacent to the current block can be used (S10S1405). The estimation of the set of weight prediction parameters for the current block has been described in detail in the operation of the encoding device, and a detailed description of the estimation will be omitted.

[0222] Although the exemplary methods of this disclosure are represented by a series of actions for clarity, they are not intended to limit the order of the steps performed, and each step may be performed simultaneously or in a different order if necessary. To implement the methods according to this disclosure, the illustrative steps may additionally include other steps, or may include steps other than some of the steps.

[0223] The various embodiments of this disclosure are not intended to be all-encompassing, but are intended to illustrate representative aspects of this disclosure. The features described in the various embodiments may be applied independently or in combination of two or more.

[0224] Furthermore, the various embodiments of this disclosure can be implemented by hardware, firmware, software, or a combination thereof. In the case of a hardware implementation, the hardware can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, etc.

[0225] The scope of this disclosure includes software or machine-executable instructions (e.g., operating systems, applications, firmware, programs, etc.) that operate according to methods of various embodiments, the operations being performed on a device or computer and on a non-transitory computer-readable medium, wherein such software or instructions may be stored and are executable on the device or computer.

[0226] Industrial applicability

[0227] This invention can be used to encode / decode images.

Claims

1. A method for decoding a video signal using a decoding device, comprising: The residual block of the current block is decoded from the bitstream using the decoding device. The decoding device is used to obtain a first prediction sample of the current block by performing inter-frame prediction on the current block; The decoding device is used to determine one or more candidate weight prediction parameters for the current block; The decoding device is used to determine the weight prediction parameters of the current block from one or more weight prediction parameter candidates; The second prediction sample of the current block is obtained by applying the weight prediction parameters to the first prediction sample using the decoding device. The decoding device is used to obtain the predicted block of the current block based on the second prediction sample; as well as The current block is reconstructed using the decoding device based on the predicted block and the residual block of the current block. The number of the one or more candidate weight prediction parameters is variably determined based on the size of the current block, and The current block is reconstructed based on the scanning order determined by the scanning information.

2. A method for encoding video signals using an encoding device, comprising: The first prediction sample of the current block is obtained by performing inter-frame prediction on the current block using the encoding device; The encoding device is used to determine one or more candidate weight prediction parameters for the current block; The weight prediction parameters for the current block are determined from one or more weight prediction parameter candidates using the encoding device. The second prediction sample of the current block is obtained by applying the weight prediction parameters to the first prediction sample using the encoding device. The encoding device is used to obtain the predicted block of the current block based on the second prediction sample; The encoding device is used to obtain the residual block of the current block based on the predicted block of the current block and the original block of the current block; as well as The residual block is encoded into a bit stream using the encoding device. The number of the one or more candidate weight prediction parameters is variably determined based on the size of the current block. The residual blocks are encoded based on the scan order, and The scanning information is determined based on the scanning order.

3. A method for transmitting a bit stream generated by an encoding method, the encoding method comprising: The first prediction sample of the current block is obtained by performing inter-frame prediction on the current block using an encoding device; The encoding device is used to determine one or more candidate weight prediction parameters for the current block; The weight prediction parameters for the current block are determined from one or more weight prediction parameter candidates using the encoding device. The second prediction sample of the current block is obtained by applying the weight prediction parameters to the first prediction sample using the encoding device. The encoding device is used to obtain the predicted block of the current block based on the second prediction sample; The encoding device is used to obtain the residual block of the current block based on the predicted block of the current block and the original block of the current block; as well as The residual block is encoded into a bit stream using the encoding device. The number of the one or more candidate weight prediction parameters is variably determined based on the size of the current block. The residual blocks are encoded based on the scan order, and The scanning information is determined based on the scanning order.

Images