Video decoding method and apparatus, and video encoding method and apparatus based on partial transform coding

Abstract

A video decoding method of the present disclosure comprises the steps of: determining a transform area in the current block, on the basis of information about the transform area; reconstructing residual samples of the current block by performing inverse transform on the transform area of the current block; and reconstructing the current block on the basis of prediction samples of the current block and the residual samples, wherein the information about the transform area comprises information for determining an area in which transform has been performed in the current block.

Description

Video decoding method and device based on partial transformation coding, and video encoding method and device

[0001] The present disclosure relates to the field of video encoding and decoding. More specifically, the present disclosure relates to a conversion method and apparatus for performing conversion coding using conversion regions of various sizes and shapes, and efficiently signaling conversion region information used in the conversion coding.

[0002] With the development and widespread adoption of hardware capable of playing and storing high-resolution or high-quality video content, the need for codecs that can effectively encode or decode such content is increasing. Recently, methods to effectively compress this high-resolution or high-quality video content are being implemented. Representative examples of codecs include HEVC (High Efficiency Video Coding) and VVC (Versatile Video Coding).

[0003] In HEVC, a single picture is divided into one or more tiles / slices and then further divided into multiple Coding Tree Units (CTUs). VVC, on the other hand, can first divide a single picture into multiple sub-pictures. A sub-picture is defined as a group of rectangular slices and was added to VVC to support the ability to partially and independently encode, decode, and transmit the picture. A single sub-picture can be divided into tiles / slices, similar to HEVC. Additionally, VVC introduces the "brick" as a new picture division structure. Bricks are created by dividing tiles horizontally and serve as the basic unit for parallel processing. To process higher resolution video than HEVC, VVC uses Coding Tree Units (CTUs) of up to 256x256, which is 16 times larger than those in HEVC, as the basic unit for encoding and decoding.

[0004] In the case of the HEVC / H.265 standard, as a transformation and quantization technique, transformation is performed in TU units. First, the CU is divided into a Quad-tree structure through the RQT process, and then the transformation process is performed according to the size of the divided blocks. For example, a 32x32 CU is divided into blocks such as 32x32, 16x16, 8x8, and 4x4 for transformation. While the HEVC / H.265 standard enables a transformation process that considers residual characteristics through the RQT process, it has the disadvantage that it fails to produce an optimized division size because all division is centered on Quad-tree division.

[0005] In the case of the VVC / H.266 standard, transformations are basically performed using a block size identical to the CU size, except in special cases. Since the CU can be determined as a non-square block, the transformation block can also be rectangular rather than square, allowing for transformations that reflect the characteristics of the residuals. Although the VVC / H.266 standard supports adaptive subdivision block sizes for transformations, it cannot perform an optimized transformation process when the CU block size and the distribution of residuals differ. To overcome this, the Sub-Block Transform (SBT) technology was adopted in the VVC / H.266 standard.

[0006] Figure 1 is a diagram showing the partitioning type of SBT according to the conventional VVC standard and the description of the conversion kernel according to the partitioning type.

[0007] SBT is a transformation method that encodes only one of the two blocks divided horizontally or vertically in the residual prediction block, and is applied only to the inter-prediction block. When the current CU is an inter-prediction encoded block and cu_cbf is 1, i.e., the number of quantization coefficients is 1 or more, it is signaled using cb_sbt_flag whether it is encoded with SBT. Additionally, if SBT is used, the type of SBT and the location where SBT is applied are additionally signaled. Referring to Fig. 1, if the block is divided vertically, cb_sbt_horizontal_flag is signaled as 0, and if it is divided horizontally, cb_sbt_horizontal_flag is signaled as 1. Furthermore, if the block located on the left or top side of the divided block is encoded, cu_sbt_pos_flag is signaled as 0, and if the block located on the right or bottom side is encoded, cu_sbt_pos_flag is signaled as 1. Transformation kernels for horizontal and vertical directions are predefined according to cb_sbt_horizontal_flag and cu_sbt_pos_flag, and transformation is performed.

[0008] When partitioning in the horizontal and vertical directions, the partitioning ratio can be half the horizontal or vertical length of the CU, or in the case of an Asymmetric Binary Tree (ABT), a 1:3 / 3:1 ratio. In SBT, if the size of one side of the transform block is 32 or greater, a DCT-2 kernel is applied for the horizontal and vertical directions. The maximum size of the transform kernel applied to SBT (max-SbtSize) is signaled in the SPS and is set to 64 for HD and 4K video, and 32 for low-resolution video. Additionally, SBT is not applied to Inter- and Intra Mixed Prediction (CIIP) mode and TPM mode.

[0009] However, these conventional SBT methods define the transformation regions within a block using only a limited number of fixed position patterns, such as top-left, top-right, bottom-left, and bottom-right. Due to these structural constraints, there are limitations in representing the locations of transformation regions of various sizes and shapes, and it is difficult to precisely represent the areas where residual energy is actually concentrated.

[0010] Furthermore, conventional SBT has limitations in selecting the optimal size of the transform area. If the distribution of residuals is non-uniform due to factors such as content characteristics within the image or the spatial correlation of prediction errors, the SBT method struggles to provide an optimal transform area size suitable for that distribution. In particular, due to limited flexibility regarding the size and orientation of the transform area, it fails to adequately reflect energy concentration in complex edges or texture regions. Consequently, transform efficiency is limited, and the improvement in coding gain falls short of expectations.

[0011] As such, existing SBT-based transform coding methods fail to accurately reflect the spatial distribution characteristics of residuals and cannot achieve optimal coding efficiency due to limited transform domain representation and signaling.

[0012] The present invention aims to provide a new method for setting and processing a transformation area that improves compression efficiency by more precisely reflecting the spatial distribution characteristics of residual signals in the transformation coding process applied in video coding standards, and overcomes the limitations of position representation in the existing SBT method.

[0013] To solve the above-mentioned technical problem, the present invention provides an adaptive transform region-based transform coding method and apparatus that adaptively sets a transform region by considering the spatial distribution characteristics of residual blocks during the encoding or decoding process of a video signal, and efficiently signals the set transform region information to perform a consistent transformation process between an encoder and a decoder.

[0014] According to the present invention, by precisely reflecting the spatial distribution characteristics of residual blocks during the image encoding process, the transformation of unnecessary areas is prevented, and the transformation can be performed only on areas where energy is substantially concentrated, thereby improving transformation efficiency and overall compression performance.

[0015] Furthermore, according to the present invention, the limitations of position representation that occurred in the existing Sub-Block Transform (SBT) method can be overcome, and a detailed and flexible transformation area setting corresponding to various residual distribution forms becomes possible.

[0016] In addition, since the present invention efficiently signals conversion area information, it is possible to ensure consistency of conversion area information between the encoder and the decoder while minimizing additional encoding overhead of the bitstream.

[0017] In addition, according to the present invention, by utilizing a partial transform process that considers complexity, the amount of computation of the encoder can be reduced while performing the transformation only on selected valid regions, thereby achieving the effect of simultaneously improving compression efficiency and computation efficiency.

[0018] Figure 1 is a diagram showing the partitioning type of SBT according to the conventional VVC standard and the description of the conversion kernel according to the partitioning type.

[0019] FIG. 2 illustrates a block diagram of an encoding device according to an embodiment of the present invention.

[0020] FIG. 3 illustrates a decoding device according to one embodiment of the present invention in a block diagram.

[0021] FIG. 4 is a reference diagram for explaining the conversion area setting according to one embodiment of the present invention.

[0022] FIGS. 5a and 5b show examples of transformation regions set according to the first to fourth parameters of an embodiment of the present invention.

[0023] FIGS. 6a to 6c are reference drawings illustrating examples of conversion regions set according to the first to fourth parameters and conversion mode according to an embodiment of the present invention.

[0024] FIGS. 7a to 7c are reference drawings illustrating other examples of conversion regions set according to the first to fourth parameters and conversion mode according to an embodiment of the present invention.

[0025] FIGS. 8a to 8c are reference diagrams for explaining the process of reconstructing a set transformation area into one or more transformation blocks in one embodiment of the present invention.

[0026] FIGS. 9a to 9c are reference diagrams for explaining the process of reconstructing a conversion region in one embodiment of the present invention.

[0027] FIG. 10 is a diagram showing candidate transformation regions according to one embodiment of the present invention.

[0028] FIG. 11 is a reference diagram illustrating the process of setting the size of a corner position transformation area according to one embodiment of the present invention.

[0029] FIG. 12 is a diagram illustrating a process for determining a transformation area based on slope according to another embodiment of the present invention.

[0030] FIG. 13 is a flowchart of a video encoding method based on partial conversion according to an embodiment of the present invention.

[0031] FIG. 14 is a flowchart of a video decoding method based on partial conversion according to one embodiment of the present invention.

[0032] A video decoding method according to one embodiment of the present invention comprises: a step of generating predicted samples of a current block; a step of determining a transformation area in the current block based on information regarding a transformation area; a step of restoring residual samples of the current block by performing an inverse transformation on the transformation area of ​​the current block; and a step of restoring the current block based on the predicted samples of the current block and the residual samples, wherein the information regarding the transformation area includes information for determining a region in the current block where a transformation was performed.

[0033] In one embodiment, the information regarding the transformation area includes at least one parameter among a first parameter (w_start) indicating the starting position of the transformation area in the horizontal axis direction in the current block, a second parameter (w_end) indicating the ending position of the transformation area in the horizontal axis direction, a third parameter (h_start) indicating the starting position of the transformation area in the vertical axis direction, and a fourth parameter (h_end) indicating the ending position of the transformation area in the vertical axis direction, and the transformation area is determined according to the at least one parameter.

[0034] In one embodiment, the step of determining a transformation area in the current block includes: a step of determining a first transformation area in the horizontal axis direction using at least one of the first parameter (w_start) and the second parameter (w_end) when the information regarding the transformation area includes at least one of the first parameter (w_start) and the second parameter (w_end); and a step of determining a second transformation area in the vertical axis direction using at least one of the third parameter (h_start) and the fourth parameter (h_end) when the information regarding the transformation area includes at least one of the third parameter (h_start) and the fourth parameter (h_end).

[0035] In one embodiment, the information regarding the transformation area further includes transformation mode information indicating one of the following transformation modes: a first transformation mode determining the area where the first transformation area in the horizontal axis direction and the second transformation area in the vertical axis direction overlap as the transformation area; a third transformation area including the area up to the vertical axis direction position pointed to by the fourth parameter (h_end) among the first transformation area in the horizontal axis direction; a fourth transformation area including the area up to the horizontal axis direction position pointed to by the second parameter (w_end) among the second transformation area in the vertical axis direction; a second transformation mode determining the area included in at least one of the third transformation area and the fourth transformation area as the transformation area; and a third transformation mode determining the area included in at least one of the first transformation area in the horizontal axis direction and the second transformation area in the vertical axis direction as the transformation area. The transformation area is determined according to the transformation mode information indicating one of the first to third transformation modes.

[0036] In one embodiment, the first to fourth parameters and the transformation mode information are each signaled using a predetermined index, or signaled using a horizontal axis direction index pointing to one element pointing to the first parameter and the second parameter applied to the current block among a grouped horizontal axis direction transformation area table having elements combining the first parameter and the second parameter, a vertical axis direction index pointing to one element pointing to the third parameter and the fourth parameter applied to the current block among a grouped vertical axis direction transformation area table having elements combining the third parameter and the fourth parameter, and an index pointing to the transformation mode, or signaled using an index pointing to one element pointing to the first to fourth parameter applied to the current block among a grouped transformation area table having elements combining the first to fourth parameters, and an index pointing to the transformation mode, or a grouped table having elements combining the first to fourth parameters and the transformation mode information pointing to one element pointing to the first to fourth parameters and the transformation mode applied to the current block. Signaling is performed using an index.

[0037] In one embodiment, the step of restoring residual samples of the current block divides the transformation area of ​​the determined current block into one or more sub-blocks and rearranges the one or more sub-blocks according to a predetermined scan order to generate one or more transformation blocks.

[0038] In one embodiment, the step of restoring residual samples of the current block performs a first-order inverse transformation based on one or more rearranged transformation blocks, and performs a second-order inverse transformation on a portion of one or more of the first-order inversely transformed transformation blocks.

[0039] In one embodiment, the transformation area of ​​the current block is determined to be one of predetermined transformation area patterns including at least one of the upper-left, upper-right, lower-left, lower-right corner transformation area and the center transformation area of ​​the current block.

[0040] In one embodiment, the transformation area of ​​the current block includes at least one of the upper-left, upper-right, lower-left, and lower-right corner transformation areas of the current block, and information regarding the transformation area includes corner transformation area information for pointing to the upper-left, upper-right, lower-left, and lower-right corner areas and size setting index information for determining the size of the corner transformation area.

[0041] In one embodiment, the information regarding the transformation area includes the corner transformation area information, which includes a first corner transformation area information indicating the upper / lower position of the corner transformation area and a second corner transformation area information indicating the left / right position of the corner transformation area, and the size setting index information includes information indicating whether the width or height of the corner transformation area is a size that divides the width or height of the current block into 1 / 2 or 1 / 4.

[0042] In one embodiment, the information regarding the transformation area includes information indicating whether partial transformation is applied, and the step of determining the transformation area in the current block determines the transformation area by determining a transformation area derived from a preset candidate transformation area of ​​the current block using a gradient when the information indicating whether partial transformation is applied indicates that partial transformation is applied.

[0043] In one embodiment of the present invention, a video decoding device for performing the aforementioned video decoding method and a computer-readable recording medium having a program for execution on a computer are disclosed.

[0044] A video encoding method according to an embodiment of the present invention comprises: a step of generating predicted samples of a current block; a step of generating residual samples of a current block using the predicted samples of the current block and the samples of the current block; a step of determining a transformation region in the current block; a step of encoding residual samples of the current block by performing a transformation on the transformation region of the current block; and a bitstream including transformation region information for determining a region where a transformation is performed in the current block and the encoded residual sample information.

[0045] In one embodiment of the present invention, a video encoding device for performing the aforementioned video encoding method and a computer-readable recording medium having a program for execution on a computer are disclosed.

[0046] Hereinafter, a video encoding, decoding method, and apparatus according to an embodiment of the present invention will be described in detail with reference to the attached drawings.

[0047] In the following description of the present invention, specific descriptions of related known functions or configurations will be omitted if it is determined that such detailed descriptions could unnecessarily obscure the essence of the invention. Furthermore, the terms described below are defined in consideration of their functions within the present invention, and these definitions may vary depending on the intentions or practices of the user or operator. Therefore, their definitions should be based on the overall content of the present invention.

[0048] In addition, the preferred embodiments of the present invention described below will focus on explaining the functional configurations that must be additionally provided for the present invention, while omitting as much as possible the system functional configurations that are already provided in each system functional configuration or are ordinarily provided in the technical field to which the present invention belongs, in order to efficiently explain the technical components constituting the present invention.

[0049] If a person skilled in the art to which the present invention pertains can easily understand the function of a component that has been used in the past among the functional configurations that are omitted and not illustrated below, and can also clearly understand the relationship between the component that was omitted as above and the component added for the present invention.

[0050] In this specification, a device that encodes an image to generate a video signal bitstream is referred to as an encoding device, an encoding device, or an encoder, and a device that decodes a video signal bitstream to restore an image is referred to as a decoding device, a decoding device, or a decoder.

[0051] A pixel or pel refers to the smallest unit that constitutes an image, and the terms pixel and sample may be used interchangeably. Generally, a sample can represent a pixel or its value, and it may represent only the pixel or its value of the luminance component, or only the pixel or its value of the chroma component.

[0052] Furthermore, the term "unit" is used to refer to a basic unit of image processing or a specific location within a picture, representing an image region containing at least one of the luminance component and the chrominance component. Specifically, the term "unit" can be used as a concept encompassing the Coding Tree Unit (CTU), Coding Unit (CU), Prediction Unit (PU), and Transform Unit (TU). Additionally, the term "block" represents an image region containing a specific component among the luminance and chrominance components, and an MxN block can represent a set of samples or transform coefficients consisting of M columns and N rows. Here, terms such as unit, block, partition, signal, and region may be used interchangeably.

[0053] Meanwhile, the term "picture" refers to a field or a frame, and these terms may be used interchangeably. For example, if the image is interlaced, a single frame is separated into an odd (or top) field and an even (or bottom) field, and each field is configured as a single picture unit for encoding or decoding. And if the image is sequential, a single frame is configured as a picture for encoding or decoding.

[0054] FIG. 2 is a block diagram illustrating an encoding device according to an embodiment of the present invention, and is intended to explain the configuration and operation of a video encoder for encoding images.

[0055] Referring to FIG. 2, the video encoder (100) may be configured to include a conversion unit (110), a quantization unit (120), an inverse quantization unit (130), an inverse conversion unit (140), a filtering unit (150), a prediction unit (160), a DPB (Decoded Picture Buffer, 170), and an entropy coding unit (180).

[0056] The conversion unit (110) converts the residual signal, which is the difference between the input video signal and the prediction signal generated by the prediction unit (160), to obtain a conversion coefficient value.

[0057] For example, the Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), or Wavelet Transform may be used.

[0058] The transformation kernel used for the transformation of a residual block may be a transformation kernel having separable characteristics of vertical transformation and horizontal transformation. In this case, the transformation of the residual block can be performed by separating it into a vertical transformation and a horizontal transformation. For example, an encoder may perform a vertical transformation by applying a transformation kernel in the vertical direction of the residual block. Additionally, an encoder may perform a horizontal transformation by applying a transformation kernel in the horizontal direction of the residual block.

[0059] Meanwhile, a transformation kernel can be used as a term referring to a set of parameters used for transforming a residual signal, such as a transformation matrix, transformation array, transformation function, or transformation; it can be any one of a plurality of available kernels, and transformation kernels based on different transformation types may be used for vertical transformation and horizontal transformation, respectively.

[0060] Transformation coefficients may be distributed such that higher coefficients are found towards the top-left corner of the block, and coefficients close to '0' are found towards the bottom-right corner. Additionally, as the current block size increases, there is a higher likelihood of '0' coefficients existing in the bottom-right region; to reduce the transformation complexity of large blocks, only an arbitrary top-left region may be retained, while the remaining regions may be reset to '0'.

[0061] Additionally, a residual signal may exist only in some areas of a residual block, and in this case, a transformation process may be performed only on some areas (hereinafter referred to as "transformation areas"). For example, in a residual block of size 2Nx2N, an error signal may exist only in the first 2NxN block, and in this case, a transformation process may be performed only on the first 2NxN block, but the second 2NxN block may not be encoded or decoded without a transformation process being performed.

[0062] As described below, according to embodiments of the present invention, a transformation region in which transformation coding of various sizes and shapes is performed can be established to precisely reflect the spatial distribution characteristics of residual samples, prevent transformation of unnecessary regions, and perform transformation only on regions where energy is substantially concentrated. According to embodiments of the present invention, by efficiently signaling the transformation region in which transformation coding is performed in various ways, the limitations of position representation that occurred in the existing Sub-Block Transform (SBT) method can be overcome, and a detailed and flexible transformation region setting corresponding to various residual distribution forms becomes possible.

[0063] Meanwhile, the encoder may perform an additional transformation before the transformation coefficients are quantized. The transformation method described above is referred to as a primary transform, and the additional transformation may be referred to as a secondary transform.

[0064] The second-order conversion can be optional for each residual block. For example, the encoder can improve coding efficiency by performing a second-order conversion for regions where it is difficult to concentrate energy in the low-frequency region with only the first-order conversion.

[0065] Specifically, a second transformation may be additionally performed on blocks where residual values ​​appear significantly in directions other than the horizontal or vertical direction of the residual block, and unlike the first transformation, the second transformation may not be performed separately into a vertical transformation and a horizontal transformation. The second transformation described above may be referred to as a Low Frequency Non-Separable Transform (LFNST).

[0066] The quantization unit (120) quantizes the conversion coefficient value output from the conversion unit (110).

[0067] In order to increase coding efficiency, instead of coding the picture signal as is, a method is used to predict the picture using an area that has already been coded through a prediction unit (160), and to obtain a restored picture by adding the residual value between the original picture and the predicted picture to the predicted picture.

[0068] To prevent mismatches from occurring between the encoder and decoder, when performing prediction in the encoder, information available in the decoder must be used; to this end, the encoder may perform a process of restoring the currently encoded block.

[0069] The inverse quantization unit (130) inversely quantizes the conversion coefficient value, and the inverse conversion unit (140) restores the residual value using the inversely quantized conversion coefficient value.

[0070] The filtering unit (150) performs filtering operations using a deblocking filter, a Sample Adaptive Offset (SAO), an Adaptive Loop Filter (ALF), etc., to improve the quality of the restored picture and enhance encoding efficiency.

[0071] A deblocking filter is a filter for removing distortion within blocks generated at the boundaries between blocks in a restored picture, and the encoder can determine whether to apply the deblocking filter to a boundary based on the distribution of pixels included in a few columns or rows relative to an arbitrary boundary within a block.

[0072] When a deblocking filter is applied, the filtering unit (150) can apply a long filter, a strong filter, or a weak filter depending on the deblocking filtering strength, and can process horizontal filtering and vertical filtering in parallel.

[0073] Sample Adaptive Offset (SAO) can be used to correct the offset from the original image on a pixel-by-pixel basis for a residual block to which a deblocking filter has been applied. The filtering unit (150) may use a Band Offset method to correct the offset for a specific picture by dividing the pixels included in the image into a certain number of regions, determining the region to perform offset correction, and applying the offset to the region. Additionally, the filtering unit (150) may use an Edge Offset method to apply the offset by considering the edge information of each pixel.

[0074] An Adaptive Loop Filter (ALF) is a method that divides pixels contained in an image into specific groups, determines a single filter to apply to each group, and performs differential filtering for each group. Information regarding whether to apply an Adaptive Loop Filter can be signaled at the coding unit level, and the shape and filter coefficients of the ALF filter to be applied may vary depending on each block. Additionally, the same type of Adaptive Loop Filter may be applied regardless of the characteristics of the target block.

[0075] The filtered picture can be stored in DPB (170) to be used as a reference picture.

[0076] The prediction unit (160) includes an intra / IBC prediction unit (161) and an inter prediction unit (165). The intra / IBC prediction unit (161) performs intra prediction and IBC prediction within the current picture, and the inter prediction unit (165) performs inter prediction by predicting the current picture using a reference picture stored in the DPB (170).

[0077] The intra / IBC prediction unit (161) performs intra prediction from the restored regions within the current picture and transmits intra encoding information to the entropy coding unit (180). Here, the intra encoding information may include at least one of an intra prediction mode, an MPM (Most Probable Mode) flag, an MPM index, and information regarding a reference sample.

[0078] Additionally, the intra / IBC prediction unit (161) can perform IBC (Intra Black Copy) prediction from the restored samples in the current picture and transmit the IBC encoding information to the entropy coding unit (180).

[0079] In this case, the intra / IBC prediction unit (161) obtains a block vector value indicating a reference area used for prediction of the current area by referring to a specific area within the current picture, and can perform IBC prediction using the obtained block vector value.

[0080] The intra / IBC prediction unit (161) can transmit IBC encoding information, including at least one of the size information of the reference area and block vector information (index information for predicting the block vector of the current block within the motion candidate list and block vector difference information), to the entropy coding unit (160).

[0081] The inter prediction unit (165) refers to a specific area of ​​the restored reference picture to find the part most similar to the current area and obtains a motion vector value which is the distance between the areas, and transmits motion information for the obtained reference area (reference direction indicator information (L0 prediction, L1 prediction, bidirectional prediction), reference picture index, motion vector information, etc.) to the entropy coding unit (180).

[0082] Additionally, the inter prediction unit (165) performs motion compensation using motion information to generate a prediction block for the current block, and transmits inter encoding information containing motion information for a reference area to the entropy coding unit (180).

[0083] Meanwhile, quantized transformation coefficients in the form of a two-dimensional array can be rearranged into a one-dimensional array for entropy coding.

[0084] The method for scanning quantized transform coefficients can be determined by the size of the transform block and the intra-prediction mode, and diagonal, vertical, or horizontal scans can be applied, and the scan information can be signaled in block units or derived from a decoder according to a defined rule.

[0085] The entropy coding unit (180) generates a bitstream by entropy coding information representing quantized conversion coefficients, intra-coding information, and inter-coding information, etc. For this purpose, a variable length coding (VLC) method and an arithmetic coding method may be used.

[0086] Variable Length Coding (VLC) converts input symbols into a sequence of codewords, the length of which can be variable. For example, frequently occurring symbols can be represented by short codewords, while infrequently occurring symbols can be represented by long codewords.

[0087] As a variable-length coding method, Context-based Adaptive Variable Length Coding (CAVLC) can be used.

[0088] Arithmetic coding utilizes the probability distribution of each data symbol to convert consecutive data symbols into a single prime number, thereby obtaining the optimal prime bit required to represent each symbol.

[0089] As an arithmetic coding method, the Context-based Adaptive Binary Arithmetic Code (CABAC) method can be used.

[0090] CABAC is a method of binary arithmetic encoding using multiple context models generated based on probabilities obtained through experiments. A context model can also be referred to as a context model. First, if a symbol is not in binary form, the encoder binarizes each symbol using exp-Golomb or similar methods. Binarized 0s or 1s can be described as bins.

[0091] The CABAC initialization process is divided into context initialization and arithmetic coding initialization. Context initialization is the process of initializing the occurrence probability of each symbol, which is determined by the type of symbol, quantization parameters (QP), and slice type (whether it is I, P, or B). A context model containing this initialization information can use probability-based values ​​obtained through experiments.

[0092] The context model provides the probability of occurrence of LPS (Least Probable Symbol) or MPS (Most Probable Symbol) for the symbol currently to be coded, and information (valMPS) on which empty value between 0 and 1 corresponds to the MPS.

[0093] One of several context models is selected through the context index (ctxIdx), and the context index can be derived from information about the block currently to be encoded or information about surrounding blocks.

[0094] Initialization for binary arithmetic coding is performed based on the probability model selected from the context model. Binary arithmetic coding proceeds by dividing into probability intervals based on the occurrence probabilities of 0 and 1, and then the probability interval corresponding to the bin to be processed becomes the entire probability interval for the next bin to be processed.

[0095] The location information within the probability interval where the last bin was processed is output. However, since the probability interval cannot be divided indefinitely, if it shrinks to within a certain size, a renormalization process is performed to widen the probability interval and output the corresponding location information. Additionally, after each bin is processed, a probability update process may be performed to newly set the probability for the next bin to be processed based on the information of the processed bin.

[0096] As mentioned above, the generated bitstream is encapsulated with a Network Abstraction Layer (NAL) unit as the basic unit.

[0097] NAL units are classified into VCL (Video Coding Layer) NAL units containing video data and non-VCL NAL units containing parameter information for decoding video data, and various types of VCL or non-VCL NAL units may exist.

[0098] A NAL unit consists of NAL header information and data, which is a Raw Byte Sequence Payload (RBSP), and the NAL header information includes summary information about the RBSP. The RBSP of a VCL NAL unit contains encoded integer coding tree units.

[0099] In order to decode a bitstream in a decoder, the bitstream must first be separated into NAL units, and then each separated NAL unit must be decoded. Meanwhile, the information required for decoding the bitstream can be transmitted in the Picture Parameter Set (PPS), Sequence Parameter Set (SPS), Video Parameter Set (VPS), etc.

[0100] Meanwhile, the configuration and operation of the video encoder described with reference to FIG. 2 are according to an embodiment of the present invention, and some configurations may be omitted or added as needed.

[0101] FIG. 3 is a block diagram illustrating a decoding device according to one embodiment of the present disclosure, and is intended to explain the configuration and operation of a video decoder for decoding video.

[0102] Referring to FIG. 3, the video decoder (200) may be configured to include an entropy decoding unit (210), an inverse quantization unit (220), an inverse transform unit (230), a filtering unit (240), a prediction unit (260), and a DPB (Decoded Picture Buffer, 270).

[0103] The entropy decoding unit (210) entropies decodes the bitstream to extract conversion coefficient information, intra-coding information, inter-coding information, etc. for each region.

[0104] For example, the entropy decoding unit (210) can obtain a binary code for conversion coefficient information of a specific region from the bitstream, and can obtain quantized conversion coefficients by debinding the binary code.

[0105] The inverse quantization unit (220) inversely quantizes the quantized conversion coefficients, and the inverse conversion unit (230) restores the residual value using the inversely quantized conversion coefficients.

[0106] Meanwhile, the residual value obtained from the inverse conversion unit (230) is added to the predicted value obtained from the prediction unit (260) to restore the original pixel value.

[0107] The filtering unit (240) performs filtering operations using a deblocking filter, a sample adaptive offset, an adaptive loop filter, etc., to improve the quality of the restored picture, and the filtered picture can be stored in the DPB (270) to be output or used as a reference picture for the next picture.

[0108] The prediction unit (260) includes an intra / IBC prediction unit (261) and an inter prediction unit (265), and generates a prediction picture by utilizing the encoding type decoded through the entropy decoding unit (210), the conversion coefficient for each region, intra / inter encoding information, etc.

[0109] To restore the current block being decrypted, the decrypted regions of the current picture containing the current block or other pictures may be utilized. A picture (or tile / slice) that performs intra prediction or intra-BC prediction using only the current picture for restoration is called an intra picture or I picture (or tile / slice), and a picture (or tile / slice) capable of performing intra prediction, inter prediction, and intra-BC prediction is called an inter picture (or tile / slice).

[0110] Meanwhile, a picture (or tile / slice) that uses at most one motion vector and reference picture index to predict sample values ​​of each block among the inter-pictures (or tiles / slices) is called a predictive picture or P picture (or tile / slice), and a picture (or tile / slice) that uses at most two motion vectors and reference picture indices is called a bi-predictive picture or B picture (or tile / slice).

[0111] That is, the P picture (or tile / slice) uses at most one set of motion information to predict each block, and the B picture (or tile / slice) uses at most two sets of motion information to predict each block. Here, a set of motion information may include one or more motion vectors and one reference picture index.

[0112] The intra / IBC prediction unit (261) generates a prediction block using intra-coding information and restored samples within the current picture, and the intra-coding information may include at least one of an intra-prediction mode, an MPM (Most Probable Mode) flag, and an MPM index.

[0113] The intra / IBC prediction unit (261) can predict the sample values ​​of the current block by using the restored samples located to the left and / or above the current block as reference samples.

[0114] For example, the reference samples may be samples adjacent to the left boundary of the current block and / or samples adjacent to the upper boundary, and among the samples of the surrounding blocks of the current block, samples located on a line within a preset distance from the left boundary of the current block and / or samples located on a line within a preset distance from the upper boundary of the current block. In this case, the surrounding blocks of the current block may include at least one of the left (L) block, upper (A) block, lower-left (BL) block, upper-right (AR) block, or upper-left (AL) block adjacent to the current block.

[0115] Additionally, the intra / IBC prediction unit (261) can restore the current region by referring to a specific region containing restored samples within the current picture. To this end, the intra / IBC prediction unit (261) can perform IBC prediction using IBC encoding information obtained from the entropy decoding unit (210), and the IBC encoding information may include block vector information.

[0116] The inter prediction unit (265) generates a prediction block using reference picture and inter encoding information stored in the DPB (270), and the inter encoding information may include a set of motion information of the current block for the reference block (reference picture index, motion vector information, etc.).

[0117] Meanwhile, inter-prediction can include L0 prediction, L1 prediction, and bi-prediction.

[0118] L0 prediction means a prediction using one reference picture included in the L0 picture list, and L1 prediction means a prediction using one reference picture included in the L1 picture list. To do this, a set of motion information (e.g., motion vector and reference picture index) may be required.

[0119] In the pair prediction method, up to two reference regions can be used, and the two reference regions may exist in the same reference picture or in different pictures. Accordingly, in the up to two sets of motion information used in the pair prediction method, two motion vectors may correspond to the same reference picture index or to different reference picture indices.

[0120] At this time, the reference pictures are pictures located temporally before or after the current picture, and can be completed pictures that have already been restored, and the two reference regions used in the pair prediction method can be regions selected from the L0 picture list and the L1 picture list, respectively.

[0121] The inter prediction unit (265) can obtain a reference block of the current block using a motion vector and a reference picture index, and the reference block exists within a reference picture corresponding to the reference picture index.

[0122] In addition, a sample value of a block specified by a motion vector or an interpolated value thereof can be used as a predictor for the current block. For motion prediction with subpel-level pixel accuracy, an 8-tap interpolation filter can be used for the luminance signal and a 4-tap interpolation filter can be used for the chrominance signal.

[0123] Meanwhile, the configuration and operation of the decoder described with reference to FIG. 2 are in accordance with an embodiment of the present disclosure, and some configurations may be omitted or added as necessary, and the decoder can decode an image by performing the reverse process of the encoding method of the encoder described above.

[0124] Hereinafter, a partial transform process according to one embodiment of the present disclosure is described. The partial transform process according to the present disclosure is performed in the aforementioned video encoder (100), and information related to the partial transform can be encoded in the entropy coding unit (180) and transmitted to the video decoder (200) in the form of a bitstream. The video encoder (100) can perform the transformation process and subsequent encoding procedure based on a determined transformation area.

[0125] The entropy decoding unit (210) of the video decoder (200) can parse a bitstream to obtain information related to partial conversion, determine a conversion region where conversion coding is performed in the current block, and perform an inverse conversion process. Information related to partial conversion may also be derived from other information obtained from the bitstream. The video decoder (200) can perform an inverse conversion process and subsequent decoding procedures based on the determined conversion region.

[0126] A video encoding and decoding method according to one embodiment of the present invention may set a transform region to efficiently determine the area where actual transformation is performed within the current block. The transform region may be adaptively determined according to the energy distribution of the residual signal within the current block, the prediction mode, etc. The video encoder (100) may select an optimal transform region based on Rate-Distortion Optimization (RDO) among candidate transform regions of various sizes and shapes within the current block, and generate a bitstream by encoding relevant information for determining the transform region.

[0127] Embodiments of the present invention disclose a method for defining a transformation region in which a transformation is actually performed within a current block, and for efficiently encoding and decoding information related to this transformation region. In one embodiment, the determination of the transformation region may be performed based on position parameters in the width and height directions of the block and a transformation mode indicating a combination method of the corresponding region.

[0128] A transformation area according to one embodiment of the present invention may be configured to include boundary parameters in the horizontal axis direction (width) and / or the vertical axis direction (height) to define an area where actual transformation is performed within a current block. For example, the transformation area may be determined using at least one of a first parameter (w_start) indicating the starting position of the transformation area in the horizontal axis direction within the current block, a second parameter (w_end) indicating the ending position of the transformation area in the horizontal axis direction, a third parameter (h_start) indicating the starting position of the transformation area in the vertical axis direction, and a fourth parameter (h_end) indicating the ending position of the transformation area in the vertical axis direction.

[0129] FIG. 4 is a reference diagram for explaining the conversion area setting according to one embodiment of the present invention.

[0130] The first to fourth parameters may be expressed as relative indices indicating the location of a sub-block according to the sub-division unit of the current block in advance (e.g., 2 divisions, 4 divisions, 8 divisions, etc.). For example, referring to FIG. 4, when the current block is divided into 4 horizontal and 4 vertical divisions, the first to fourth parameters indicating the start or end position of the transformation area in the horizontal axis direction and the vertical axis direction, respectively, may be expressed as indices indicating the boundary positions of the sub-block, such as {0, 1, 2, 3, 4}, as illustrated. Not limited thereto, the first to fourth parameters may be set in pixel units indicating the start or end position of the transformation area within the current block.

[0131] FIGS. 5a and 5b show examples of transformation regions set according to the first to fourth parameters of an embodiment of the present invention.

[0132] The transformation area may be set using at least one of the first to fourth parameters to specify the spatial range in which the transformation is performed in the current block. Depending on the situation, the video encoder (100) or video decoder (200) may adaptively configure a transformation area that considers only the horizontal axis direction, a transformation area that considers only the vertical axis direction, or a two-dimensional transformation area that considers both the horizontal and vertical axes simultaneously.

[0133] If the transformation area information is set only by the first parameter (w_start), the transformation area can be determined as the area from the width position of the current block pointed to by the first parameter (w_start) to the right boundary. If the transformation area information is set only by the second parameter (w_end), the transformation area can be determined as the area from the left boundary of the current block to the width position of the current block pointed to by the second parameter (w_end). Similarly, if the transformation area information is set only by the third parameter (h_start), the transformation area can be determined as the area from the height position of the current block pointed to by the third parameter (h_start) to the bottom boundary. If the transformation area information is set only by the fourth parameter (h_end), the transformation area can be determined as the area from the upper boundary of the current block to the height position of the current block pointed to by the fourth parameter (h_end).

[0134] For example, referring to FIG. 5a, when only the first parameter (w_start) is set to 1, the gray shaded area (501) from the horizontal axis position indicated by index 1 among the positions in which the horizontal axis direction (width direction) of the current block is divided into four parts to the right boundary is determined as the transformation area. When only the second parameter (w_end) is set to 3, the gray shaded area (502) from the left boundary of the current block to the horizontal axis position indicated by index 3 among the positions in which the horizontal axis direction (width direction) is divided into four parts is determined as the transformation area.

[0135] In a similar example, when only the third parameter (h_start) is set to 1, the gray shaded area (503) from the vertical axis position pointed to by index 1 among the positions in which the vertical axis direction (height direction) of the current block is divided into four parts is determined as the transformation area. Also, when only the fourth parameter (h_end) is set to 2, the gray shaded area (504) from the upper boundary of the current block to the vertical axis position pointed to by index 2 among the positions in which the vertical axis direction (height direction) is divided into four parts is determined as the transformation area.

[0136] When the transformation area information is set to the first parameter (w_start) and the second parameter (w_end), the transformation area can be determined as the area from the width position of the current block pointed to by the first parameter (w_start) to the width position of the current block pointed to by the second parameter (w_end). When the transformation area information is set to the third parameter (h_start) and the fourth parameter (h_end), the transformation area can be determined as the area from the height position of the current block pointed to by the third parameter (h_start) to the height position of the current block pointed to by the fourth parameter (h_end).

[0137] For example, referring to FIG. 5b, when the first parameter (w_start) is set to 0 and the second parameter (w_end) is set to 3, the gray shaded area (505) from the horizontal axis position pointed to by index 0 to the position pointed to by index 3 among the positions in which the horizontal axis direction (width direction) of the current block is divided into four parts is determined as the transformation area. When the third parameter (h_start) is set to 1 and the fourth parameter (h_end) is set to 3, the gray shaded area (506) from the vertical axis position pointed to by index 1 to the position pointed to by index 3 among the positions in which the vertical axis direction (height direction) of the current block is divided into four parts is determined as the transformation area.

[0138] In one embodiment of the present invention, a transformation mode may be set to determine a final transformation area by combining a transformation area in the horizontal direction determined using at least one of a first parameter (w_start) and a second parameter (w_end), and a transformation area in the vertical direction determined using at least one of a third parameter (h_start) and a fourth parameter (h_end).

[0139] FIGS. 6a to 6c are reference drawings illustrating examples of conversion regions set according to the first to fourth parameters and conversion mode according to an embodiment of the present invention.

[0140] The transformation modes may include a first transformation mode (And mode), a second transformation mode (Sub-or mode), and a third transformation mode (Full-or mode) depending on how the final transformation area is determined by combining the transformation area in the horizontal direction determined using at least one of the first parameter (w_start) and the second parameter (w_end) and the transformation area in the vertical direction determined using at least one of the third parameter (h_start) and the fourth parameter (h_end).

[0141] Specifically, the first transformation mode (And mode) is a transformation mode that determines the intersection area where the first transformation area in the horizontal direction determined using at least one of the first parameter (w_start) and the second parameter (w_end) and the second transformation area in the vertical direction determined using at least one of the third parameter (h_start) and the fourth parameter (h_end) overlap as the transformation area. For example, referring to FIG. 6a, if the first parameter (w_start) is set to 1 and the second parameter (w_end) is set to 3 to set a first transformation area in the horizontal axis direction such as reference numeral 601, and the third parameter (h_start) is set to 1 and the fourth parameter (h_end) is set to 3 to set a second transformation area in the vertical axis direction such as reference numeral 602, the first transformation mode (And mode) determines the area where the first transformation area (601) in the horizontal axis direction and the second transformation area (602) in the vertical axis direction overlap as the final transformation area (603). In this way, the first transformation mode (And mode) takes the area where the transformation area in the horizontal axis direction determined using [w_start, w_end] and the transformation area in the vertical axis direction determined using [h_start, h_end] overlap as the target of transformation.

[0142] Additionally, the second transformation mode (Sub-or mode) is a transformation mode that determines a third transformation area including a region up to a position in the vertical axis direction pointed to by the fourth parameter (h_end) among the first transformation areas in the horizontal axis direction, and a fourth transformation area including a region up to a position in the horizontal axis direction pointed to by the second parameter (w_end) among the second transformation areas in the vertical axis direction, and determines a region included in at least one of the third transformation area and the fourth transformation area as a transformation area. For example, referring to FIG. 6b, when the first parameter (w_start) is set to 1 and the second parameter (w_end) is set to 3 to set a first transformation area in the horizontal direction such as reference numeral 611, and the third parameter (h_start) is set to 1 and the fourth parameter (h_end) is set to 3 to set a second transformation area in the vertical direction such as reference numeral 613, the second transformation mode (Sub-or) determines a third transformation area by excluding the end portion (612) from the first transformation area so that only the portion up to the vertical direction index 3 position indicated by the fourth parameter (h_end) of the first transformation area (611) is included in the transformation area. Additionally, the second transformation mode (Sub-or) determines the fourth transformation area by excluding the end portion (614) from the second transformation area so that only the portion up to the index 3 position in the horizontal axis direction indicated by the second parameter (w_end) in the second transformation area (613) is included in the transformation area. Then, the second transformation mode (Sub-or) determines the area (615) included in at least one of the third transformation area and the fourth transformation area as the final transformation area. In this way, the second transformation mode (Sub-or) takes as the target of transformation the area where the start portions of the horizontal axis transformation area determined using [w_start, w_end] and the vertical axis transformation area determined using [h_start, h_end] do not need to overlap, but the end portions overlap.

[0143] Additionally, the third transformation mode (Full-or mode) is a transformation mode that determines a transformation area as a region included in at least one of the first transformation area in the horizontal axis direction and the second transformation area in the vertical axis direction. For example, referring to FIG. 6c, if the first parameter (w_start) is set to 1 and the second parameter (w_end) is set to 3 to set the first transformation area in the horizontal axis direction as shown in reference numeral 621, and the third parameter (h_start) is set to 1 and the fourth parameter (h_end) is set to 3 to set the second transformation area in the vertical axis direction as shown in reference numeral 622, the third transformation mode (Full-or mode) determines the union area included in at least one of the first transformation area in the horizontal axis direction (621) and the second transformation area in the vertical axis direction (622) as the final transformation area (623). In this way, the third transformation mode (Full-or mode) takes as the target of transformation the area formed by adding both the transformation area in the horizontal direction determined using [w_start, w_end] and the transformation area in the vertical direction determined using [h_start, h_end].

[0144] The selected conversion mode is signaled as conversion mode information, enabling the encoder and decoder to reproduce the same conversion region structure.

[0145] Meanwhile, in one embodiment of the present invention, the first to fourth parameters (w_start, w_end, h_start, h_end) for setting the transformation area can be set regardless of the size of the transformation area, and the first parameter (w_start) indicating the starting position of the transformation area in the horizontal direction is not set smaller than the second parameter (w_end) indicating the ending position of the transformation area in the horizontal direction, nor is it necessary for the third parameter (h_start) indicating the starting position of the transformation area in the vertical direction to be set smaller than the fourth parameter (h_end) indicating the ending position of the transformation area in the vertical direction. That is, the first parameter (w_start) indicating the starting position of the transformation area in the horizontal direction can be set larger than the second parameter (w_end) indicating the ending position of the transformation area in the horizontal direction, or the third parameter (h_start) indicating the starting position of the transformation area in the vertical direction can be set larger than the fourth parameter (h_end) indicating the ending position of the transformation area in the vertical direction.

[0146] FIGS. 7a to 7c are reference drawings illustrating other examples of transformation regions set according to the first to fourth parameters (w_start, w_end, h_start, h_end) and transformation mode according to one embodiment of the present invention.

[0147] Referring to FIG. 7a, the first parameter (w_start) is set to 3, the second parameter (w_end) to 1, the third parameter (h_start) to 3, and the fourth parameter (h_end) to 1, so that the first parameter (w_start) has an index value greater than the second parameter (w_end) and the third parameter (h_start) has an index value greater than the fourth parameter (h_end). Similar to the embodiments described above, in the case of the first transformation mode (And mode), the transformation area in the horizontal axis direction is determined as the area (701) from the current block width position indicated by the first parameter (w_start) to the current block width position indicated by the second parameter (w_end), the transformation area in the vertical axis direction is determined as the area (702) from the current block height position indicated by the third parameter (h_start) to the current block height position indicated by the fourth parameter (h_end), and the area (703) where the transformation area in the horizontal axis direction (701) and the transformation area in the vertical axis direction (702) overlap can be determined as the final transformation area.

[0148] Referring to FIG. 7b, the first parameter (w_start) is set to 3, the second parameter (w_end) to 1, the third parameter (h_start) to 3, and the fourth parameter (h_end) to 1, so that the first parameter (w_start) has an index value greater than the second parameter (w_end) and the third parameter (h_start) has an index value greater than the fourth parameter (h_end). Similar to the embodiments described above, in the case of the second transformation mode (Sub-or mode), the remaining area excluding the end area (712) from the position in the vertical direction pointed to by the fourth parameter (h_end) to the upper boundary among the transformation area (711) in the horizontal direction from the current block width position pointed to by the first parameter (w_start) to the current block width position pointed to by the second parameter (w_end) is determined as the third transformation area. Additionally, among the vertical axis direction transformation area (713) from the current block height position pointed to by the third parameter (h_start) to the current block height position pointed to by the fourth parameter (h_end), the remaining area excluding the end portion area (714) from the horizontal axis direction position pointed to by the second parameter (w_end) to the left boundary is determined as the fourth transformation area. Then, an area included in at least one of the third transformation area and the fourth transformation area is determined as the final transformation area (715) according to the second transformation mode (Sub-or mode).

[0149] Referring to FIG. 7c, the first parameter (w_start) is set to 3, the second parameter (w_end) to 1, the third parameter (h_start) to 3, and the fourth parameter (h_end) to 1, so that the first parameter (w_start) has an index value greater than the second parameter (w_end) and the third parameter (h_start) has an index value greater than the fourth parameter (h_end). Similar to the embodiments described above, in the case of the third transformation mode (Full-or mode), the transformation area in the horizontal axis direction is determined as the area (721) from the current block width position indicated by the first parameter (w_start) to the current block width position indicated by the second parameter (w_end), the transformation area in the vertical axis direction is determined as the area (722) from the current block height position indicated by the third parameter (h_start) to the current block height position indicated by the fourth parameter (h_end), and the area (723) corresponding to the union of the transformation area in the horizontal axis direction (721) and the transformation area in the vertical axis direction (722) can be determined as the final transformation area.

[0150] Meanwhile, the first to fourth parameters and transformation mode information for determining the transformation area may be set independently by i) the first to fourth parameters and transformation mode information, or ii) the first and second parameter combinations in the horizontal direction and the third and fourth parameter combinations in the vertical direction are grouped and set based on an index, or iii) the index and transformation mode information pointing to the combination of the first to fourth parameters applied to the current block among a table having the combination of the first to fourth parameters as elements, or iv) the index and transformation mode information pointing to the combination of the first to fourth parameters and transformation mode applied to the current block among a table having the combination of the first to fourth parameters and transformation mode information as elements.

[0151] Specifically, i) the first to fourth parameters and transformation mode information can be signaled using a predetermined index, respectively. For example, as in the example above, if the current block is divided into four parts in the horizontal and vertical directions, the first to fourth parameters can be set using index values ​​from 0 to 4, respectively, according to the determined transformation area, and the transformation mode can be set to an index (e.g., 0, 1, 2) pointing to one of And / Sub-Or / Full-Or. If only one of the first and second parameters is used in the horizontal direction, only the one parameter used among the first and second parameters is set. Similarly, if only one of the third and fourth parameters is used in the vertical direction, only the one parameter used among the third and fourth parameters can be set. Encoding efficiency can be increased by applying a variable-length code that varies the index length according to the frequency of occurrence.

[0152] Additionally, ii) a horizontal axis direction index pointing to one element pointing to the first parameter and the second parameter applied to the current block among a grouped horizontal axis direction transformation area table having elements combining the first parameter and the second parameter, a vertical axis direction index pointing to one element pointing to the third parameter and the fourth parameter applied to the current block among a grouped vertical axis direction transformation area table having elements combining the third parameter and the fourth parameter, and an index pointing to the transformation mode can be used to set the first to fourth parameters and transformation mode information. For example, when the information used to set the transformation area of ​​the current block is w_start=0, w_end=1, h_start=1, h_end=2, and AND mode, horizontal axis direction index information pointing to the element corresponding to [w_start=0, w_end=1] in a preset horizontal axis direction table having combinations of the first parameter (w_start) and the second parameter (w_end) as elements, vertical axis direction index information pointing to the element corresponding to [h_start=1, h_end=2] in a preset vertical axis direction table having combinations of the third parameter (h_start) and the fourth parameter (h_end) as elements, and AND mode information are signaled. The indices of the preset tables can be ordered by considering the frequency of occurrence of the area, and in such cases, area information can be transmitted with fewer bits. That is, even in this case, encoding efficiency can be increased by applying a variable-length code that varies the index length according to the frequency of occurrence.

[0153] Additionally, iii) among the grouped transformation area tables having elements combining the first to fourth parameters, the first to fourth parameters and transformation mode information may be signaled using an index pointing to an element pointing to the first to fourth parameters applied to the current block, and an index pointing to the transformation mode. For example, if the information used to set the transformation area of ​​the current block is w_start=0, w_end=1, h_start=1, h_end=2, and the And mode, then among the pre-set tables having combinations of the first to fourth parameters as elements, index information pointing to an element corresponding to [w_start=0, w_end=1, h_start=1, h_end=2] and index information pointing to the And mode information are signaled. The indexes of the pre-set tables may be ordered considering the frequency of occurrence of the areas, and in such cases, area information can be transmitted with fewer bits. For index coding, a variable length coding technique may be utilized considering the frequency of occurrence. For example, techniques such as unary, truncated unary, and golomb-rice can be used depending on the frequency of occurrence.

[0154] Additionally, iv) the information regarding the first to fourth parameters and transformation modes may be set using an index pointing to a single element pointing to the first to fourth parameters and transformation modes applied to the current block among a grouped table having elements combining the information regarding the first to fourth parameters and transformation modes. For example, if the information used to set the transformation area of ​​the current block is w_start=0, w_end=1, h_start=1, h_end=2, and And mode, then index information pointing to an element corresponding to [w_start=0, w_end=1, h_start=1, h_end=2, and And mode] among a pre-set table having elements combining all information regarding the first to fourth parameters and transformation modes is signaled. In this case as well, the index of the pre-set table may be ordered by considering the frequency of occurrence of the area. In such a case, information for all combinations can be transmitted with a small number of bits. For index coding, a variable length coding technique may be utilized by considering the frequency of occurrence. For example, techniques such as unary, truncated unary, and golomb-rice can be used depending on the frequency of occurrence.

[0155] Meanwhile, in one embodiment of the present invention, in order to perform a transformation (inverse transformation) process for a set transformation area, the transformation area may be reconfigured into one or more transformation blocks. At this time, when the transformation area is reconfigured into transformation blocks, the transformation process may be performed sequentially in blocks.

[0156] FIGS. 8a to 8c are reference diagrams for explaining the process of reconstructing a set transformation area into one or more transformation blocks in one embodiment of the present invention.

[0157] The transformation area set through the first transformation mode (And mode) is defined as the intersection of the horizontal and vertical axis directions, so it basically has a rectangular shape. For example, as shown in FIG. 6a above, the transformation area (603) set according to [w_start=1, w_end=3], [h_start=1, h_end=3] and the first transformation mode (And mode) can be set as a transformation block as is without a separate additional reconstruction process, and the transformation can be performed.

[0158] Additionally, if necessary, the transformation area set through the first transformation mode (And mode) may be subdivided into smaller transformation blocks or reconstructed according to the scan order. For example, as shown in FIG. 8a, sub-blocks included in a rectangular transformation area (801) may be reconstructed according to a predetermined scan order into a square transformation block (802), and a transformation may be performed based on the reconstructed square transformation block (802).

[0159] Since the transformation area set by the second transformation mode (Sub-Or mode) or the third transformation mode (Full-Or mode) does not have a complete intersection of the horizontal and vertical axis directions, there may be cases where the transformation area does not form a rectangular shape. For example, an L-shaped or T-shaped non-rectangular transformation area may be set according to the second transformation mode (Sub-Or mode) or the third transformation mode (Full-Or mode). In this case, to perform the transformation process, the transformation area may be rearranged into one or more square or rectangular blocks or divided into multiple blocks. Referring to FIG. 8b, the transformation area (811) set based on the second transformation mode (Sub-Or mode) may have an L-shape as illustrated, and such a non-rectangular transformation area may be reconfigured into rectangular blocks (812, 813).

[0160] Meanwhile, the reconstruction of the transformation area can be performed by dividing the transformation area into sub-blocks and rearranging the sub-blocks according to a predetermined scan order.

[0161] FIGS. 9a to 9c are reference diagrams for explaining the process of reconstructing a conversion region in one embodiment of the present invention.

[0162] Referring to FIG. 9a, an L-shaped transformation area (901) is divided into sub-blocks, and each sub-block can be rearranged into a rectangular shape (902) according to a predefined up-right scan order. Subsequently, the transformation process can be performed using the entire rearranged transformation area as a single transformation block, or the transformation process can be performed by setting the sub-blocks included in the rearranged transformation area as individual transformation blocks. Additionally, referring to FIG. 9b, an L-shaped transformation area (921) is divided into a plurality of rectangular blocks (922 or 923), and the transformation process for each of the divided plurality of rectangular blocks can be performed sequentially.

[0163] The rearrangement process may be performed according to a scan order that considers the characteristics of the original residual block, namely the energy distribution or spatial correlation within the residual block, rather than a predetermined scan order. Referring to FIG. 9c, the rearrangement order of each sub-block included in the L-shaped transformation region (911) is determined by considering the characteristics of the residual samples constituting the transformation region, and according to the determined rearrangement order, the L-shaped transformation region (911) may be rearranged into a rectangular shape (912 or 913). For example, the rearrangement order of the sub-blocks may be determined so that significant residual signals that are not zero are spatially adjacent. Subsequently, the transformation process may be performed using the entire rearranged transformation region as a single transformation block, or the transformation process may be performed on a sub-block unit included in the rearranged transformation region.

[0164] The rearranged transformation block can be in the shape of a square or a rectangle, and can be applied equally to both the second transformation mode (sub-Or) and the third transformation mode (Full-Or mode).

[0165] Information regarding whether the final transformation block is configured in a square or rectangular shape during rearrangement, the configuration order of sub-blocks mapped to a square or rectangular shape during rearrangement, and the number of final rectangles into which the transformation area is divided can be adaptively determined in a video encoder (100) based on at least one of the width and height of the transformation area, the area ratio (width / height ratio), the selected transformation mode (And / Sub-Or / Full-Or), the prediction mode, the QP (Quantization Parameter), or the block type, the width and height of the current block, the residual block, and the area ratio of the residual block (width / height ratio of residual block). The video decoder (200) can arbitrarily derive information regarding whether the final transformed block will be configured in a square or rectangular shape during rearrangement, the configuration order of sub-blocks mapped to a square or rectangular shape during rearrangement, and the number of final rectangles in which the transformed area is divided, based on at least one of the following: the width and height of the transformed area, the width / height ratio, the selected transformed mode (And / Sub-Or / Full-Or), the prediction mode, the QP (Quantization Parameter), or the block type, the width and height of the residual block which is the current block, and the width / height ratio of the residual block.

[0166] The video encoder (100) may explicitly include in the bitstream information regarding whether the final conversion block will be configured in a square or rectangular shape during rearrangement, information regarding the configuration order of sub-blocks mapped to a square or rectangular shape during rearrangement, and information regarding the number of final rectangles into which the conversion area is divided, and transmit this information to the video decoder (200). When transmitting the explicit information, the information may be configured into a pre-set table, and an index corresponding to the selected information may be transmitted. This information may be transmitted in units such as GOP (Group of Pictures), Intra Period, Picture, Slice, Tile, CTU, CU, TU, etc. Based on the received information, the video decoder (200) reconstructs the conversion area using the rearranged conversion block shape and sub-blocks to configure the conversion block, and then performs inverse conversion.

[0167] As described above, once the transformation region is finally determined, the transformation process is performed on one or more transformation blocks included within the transformation region. Within the current block, which is a residual block, only the residual signals in the region included in the transformation region are transformed, and residual signals not included in the transformation region may be set to 0. Each transformation block is transformed into the frequency domain by sequentially applying horizontal and vertical transformation kernels. Kernels such as DCT-8 (Discrete Cosine Transform, type-8), DST-7 (Discrete Sine Transform, type-7), DCT-4, DST-1, and DCT-2 may be used as transformation kernels; additionally, other normalized orthogonal transformation kernels may be applied depending on the system implementation or the codec standard.

[0168] The type of transformation kernel applied to the vertical and horizontal directions can be arbitrarily determined based on at least one of the following: the width and height of the transformation block, width and height position information, the width / height ratio (aspect ratio) of the transformation block, the width and height of the transformation area, width and height position information of the transformation area, the width / height ratio of the transformation area, the transformation mode used (And / Sub-Or / Full-Or), the prediction mode, the Quantization Parameter (QP), the block type, and the width, height, and ratio of the residual block. For example, a DST-series kernel may be more effective for a vertically elongated transformation area (height > width), while applying a DCT-series kernel may be advantageous in terms of minimizing distortion and RDO performance for a horizontally expanded block (width > height). As such, the kernel selection can be determined based on RDO by considering the morphological characteristics and additional information of the transformation area or the transformation block.

[0169] Additionally, the video encoder (100) may explicitly include information regarding the types of conversion kernels applied to the vertical and horizontal directions in the bitstream and transmit it to the video decoder (200). When transmitting the information explicitly, the information may be organized into a pre-set table, and an index corresponding to the selected information may be transmitted. This information may be transmitted in units such as Group of Pictures (GOP), Intra Period, Picture, Slice, Tile, CTU, CU, TU, etc. The video decoder (200) may determine the conversion kernels applied to the vertical and horizontal directions based on the received information and perform an inverse conversion process for the determined conversion area.

[0170] According to one embodiment of the present invention, a secondary transform process may be performed after a primary transform process. In a conventional secondary transform process, a secondary transform is performed on the coefficients of the low-frequency region in the upper left corner after the primary transform of the residual block. However, in the present invention, since the primary transform is performed based on a transformation region that is a part of the current block, it is necessary to set the transformation region where the secondary transform is performed.

[0171] In one embodiment of the present invention, the second transformation may be performed based on the transformation area where the first transformation was performed. In one embodiment of the present invention, since the transformation is performed only on the transformation area which is a part of the residual block rather than the entire area of ​​the residual block, the transformation area or the transformation block that reconstructs the transformation area is used as the input unit for the second transformation.

[0172] The target of the second transformation is the transformation coefficient block obtained as a result of the first transformation, which can be defined within the coordinate system of the reconstructed transformation block regardless of the original residual block position. For example, in the first transformation mode (And mode), the top-left corner of the reconstructed rectangular transformation block is considered as (0,0), and a predetermined size area at the top-left corner of the reconstructed rectangular transformation block is used as the input for the second transformation process. In the case of the second transformation mode (Sub-Or mode) or the third transformation mode (Full-Or mode), since non-rectangular transformation areas such as L-shapes or T-shapes are rearranged into rectangular shapes, the output coefficients of the first transformation are uniformly distributed within the rectangle. Therefore, in this case as well, the top-left corner of the reconstructed rectangular block is considered as (0,0), and the entire first transformation coefficient inside it is taken as the target of the second transformation process.

[0173] Meanwhile, as shown in FIG. 8b above, when the reconstructed transformation area is divided into two rectangular blocks, each rectangular block is considered an independent transformation unit, and the output coefficients of the first transformation also exist separated into the two blocks. Therefore, the second transformation is performed on each block individually, and if there are two blocks, two second transformations are performed. The top-left corner of each block is considered as (0,0), and the first transformation coefficients of the corresponding block are used as the input for the second transformation.

[0174] The second transformation may take the same two-stage structure as the first transformation, which is a vertical transformation followed by a horizontal transformation (or the reverse order). The applied kernel may be one or more of DCT-8, DST-7, DCT-4, DST-1, and DCT-2, and a combination of kernels of the same or different types as the kernel used in the first transformation may also be used.

[0175] The type of kernel applied to the second conversion and whether to apply it are determined by the video encoder (100) and may be explicitly transmitted to the video decoder (200). The transmission unit may be one or more of the GOP, Picture, Slice, Tile, CTU, CU, and TU units. The video encoder (100) can improve the encoding efficiency of the bitstream by configuring the second conversion kernel combination and application flags into a predefined table and transmitting an index corresponding to the selected item.

[0176] Meanwhile, according to one embodiment of the present invention, when a transformation region is set using the first to fourth parameters, the computational complexity for determining the optimal transformation region in the video encoder (100) may increase, and the amount of information required to signal information regarding the transformation region may also increase. Accordingly, in one embodiment of the present invention, a transformation region may be determined among candidate transformation region patterns that are predetermined based on statistical frequency of occurrence.

[0177] FIG. 10 is a diagram showing candidate transformation regions according to one embodiment of the present invention.

[0178] Referring to FIG. 10, the candidate transformation regions may include at least one of the upper-left corner transformation region (a,b), upper-right corner transformation region (c,d), lower-left corner transformation region (e,f), lower-right corner transformation region (g,f) and center transformation region (i) of the current block. Specifically, at least one of the following may be used as a candidate transformation area: an upper-left corner transformation area (a) having a width and height that are each 1 / 4 the size of the width and height of the current block; an upper-left corner transformation area (b) having a size that is 1 / 2 the size of the width and height of the current block; an upper-right corner transformation area (c) having a size that is 1 / 4 the size of the width and height of the current block; an upper-right corner transformation area (d) having a size that is 1 / 2 the size of the width and height of the current block; a lower-left corner transformation area (e) having a size that is 1 / 2 the size of the width and height of the current block; a lower-left corner transformation area (f) having a size that is 1 / 4 the size of the width and height of the current block; a lower-right corner transformation area (g) having a size that is 1 / 2 the size of the width and height of the current block; and a center transformation area (i). The size of the center transformation area (i) is not limited to having a width and height that are 1 / 2 the size of the current block as illustrated, but may be set to a predetermined size.

[0179] When candidate transformation regions are used as exemplified in FIG. 10, information related to the transformation region applied to the current block can be signaled through a simplified syntax structure in order to simultaneously reduce the complexity of the encoding process and the amount of transmitted bits.

[0180] In one embodiment, the corner transformation regions (a) to (h) among the candidate transformation regions of FIG. 10 can be signaled using three pieces of information related to "up / down application," "left / right application," and "size setting (½ or ¼)".

[0181] The top / bottom_flag syntax indicates whether the transformation area is included in the top or bottom of the current block when the current block is divided into two along the vertical axis. For example, if the top / bottom_flag value is '1', it indicates that the transformation area is located in the bottom area, and if it is '0', it indicates that the transformation area is located in the top area. It is not limited to this and can be set in the opposite way.

[0182] The left / right application syntax (left_right_flag) can indicate whether the transformation area is included on the left or right side of the current block when the current block is divided into two along the horizontal axis. For example, if the left / right application syntax (left_right_flag) value is '0', it indicates that the transformation area is located on the left side of the current block, and if it is '1', it indicates that it is located on the right side. It is not limited to this and can be set in the opposite way.

[0183] FIG. 11 is a reference diagram illustrating the process of setting the size of a corner position transformation area according to one embodiment of the present invention.

[0184] The size setting syntax (size_idx) is information indicating the width and height of the transformation area at the corner position, and to reduce the amount of information, it can be set as index information indicating a specific ratio of the width and height of the current block, for example, one of 1 / 2 size or 1 / 4 size. That is, the width and height of the transformation area can be determined as one of 1 / 2 size and 1 / 4 size of the width and height of the current block, as shown in FIG. 11. If the size setting syntax (size_idx) value is '0', it may indicate that the width and height of the transformation area are 1 / 2 of the width and height of the current block, and if it is '1', it may indicate that the width and height of the transformation area are 1 / 4 of the width and height of the current block. It is not limited to this and can be set in the opposite way. In the case of the center transformation area, the width and height may be pre-set, or a size setting syntax indicating the width and height of the center transformation area may be signaled separately.

[0185] In this way, eight corner transformation regions (a) to (h) can be signaled by combining three syntaxes (whether to apply up / down), "whether to apply left / right," and "size setting (½ or ¼)"). In the example described above, the eight corner transformation regions (a) to (f) of FIG. 10 can be set using three syntaxes as shown in Table 1 below.

[0186] Candidate transformation area top_bottom_flag(0 is top, 1 is bottom) left_right_flag(0 is left, 1 is right) size_idx(0 is 1 / 2, 1 is 1 / 4) (a)101 (b)100 (c)111 (d)110 (e)001 (f)000 (g)011 (h)010

[0187] In the case of the center transformation area of ​​(i) in FIG. 10, since the up-down and left-right information are both unnecessary, it can be signaled by a separate syntax (center_tr_flag) that indicates only whether it is applied. For example, if center_tr_flag is '0', it may indicate that the center transformation area is not applied, and if center_tr_flag is '1', it may indicate that the center transformation area is applied. When center_tr_flag is '0', that is, when the center transformation area is not applied, the eight corner transformation areas of (a) to (h) can be signaled by the combination of the three syntaxes mentioned above. When center_tr_flag is '1', indicating that the center transformation area is applied, the three syntaxes mentioned above may not be signaled.

[0188] Meanwhile, the nine patterns (a) to (i) can also be directly composed into a single syntax. In this case, one of the nine patterns (a) to (i) can be represented as a single index value through a predefined table-based partial transformation index (partial_tr_idx). For example, the partial transformation index (partial_tr_idx) can be set as shown in Table 2 below.

[0189] Partial Transformation Index (partial_tr_index) Candidate Transformation Regions 0(a) 1(b) 2(c) 3(d) 4(e) 5(f) 6(g) 7(h) 8(i)

[0190] The partial transformation index (partial_tr_idx) can be represented by values ​​from 0 to 8 (a total of 9), and frequently occurring corner patterns (0 to 3) can be assigned to short codewords (VLC, variable-length code) to improve signaling efficiency.

[0191] At least one of the above syntax information (top_bottom_flag, left_right_flag, size_idx, center_tr_flag, partial_tr_idx) can be transmitted in units of Group of Pictures (GOP), Intra Period, Picture, Slice, Tile, Coding Tree Unit (CTU), Coding Unit (CU), or Transform Unit (TU). In particular, when transmitted in units of TU, ​​different partial transformation forms can be applied to each transformation unit, enabling fine adaptive transform control.

[0192] When transmitting, the video encoder (100) can encode the syntax value into an index of a pre-configured table. The table is sorted by considering the frequency of occurrence of the pattern, and bit efficiency is maximized by assigning shorter codewords (unary, truncated-unary, Golomb-Rice, etc.) to patterns with high frequency.

[0193] In the simplified partial transformation process exemplified in Fig. 10, the form of the block and the method of applying the kernel for performing transformation processing on a partially set transformation area can be performed as follows.

[0194] (1) Shape of the transformation application block

[0195] In the simplified partial transformation process, the shape of the block to which the transform is applied is the same as the setting method of the first transformation mode (AND Mode) described above, so the set transformation area can be used as is in the transformation process. That is, in the case of the first transformation mode (AND Mode), since the transformation area is defined in a rectangular shape, the area can be set as the input block of the transformation process without an additional rearrangement process.

[0196] In addition, the shape of the block to which the transformation is applied may use the aforementioned reconstructed block shape as is. That is, even if the transformation area of ​​Sub-Or or Full-Or mode is defined as a non-rectangular area, the sub-blocks can be rearranged according to a predefined scan order to be reconstructed into a rectangular or rectangular shape, and then the reconstructed block can be set as the input block for the transformation process.

[0197] Accordingly, in the simplified partial transformation process, the structure of the complex transformation region is mapped into a single rectangular block to perform the transformation, thereby significantly reducing the computational complexity of the encoder while maintaining transformation consistency.

[0198] (2) Method of applying the conversion kernel

[0199] In the simplified partial transformation process, the kernel applied to the transformation process is applied sequentially to the vertical and horizontal directions, respectively, in the same manner as described above. That is, it operates in a structure that first performs a first transformation based on the vertical axis (column), and then performs a second transformation based on the horizontal axis (row).

[0200] Applicable transformation kernels include DCT-8 (Discrete Cosine Transform Type 8), DST-7 (Discrete Sine Transform Type 7), DCT-4, DST-1, DCT-2, etc. These kernels can be selectively applied depending on the characteristics of the transformation block, and it is also possible to apply different kernels to the same block.

[0201] (3) Criteria for kernel selection

[0202] The transformation kernels applied to the vertical and horizontal directions, respectively, can be determined based on the width and height of the transformation block, the position of the width and height of the transformation block, the ratio of the width and height of the transformation block, the width and height of the transformation area, the position of the width and height of the transformation area, the ratio of the width and height of the transformation area, the prediction mode, the quantization parameter (QP), the block type, the width and height of the current block which is the residual block, and the ratio of the width and height of the current block.

[0203] The types of transformation kernels applied to the vertical and horizontal directions may be explicitly determined by the encoder and transmitted to the decoder. That is, the encoder encodes and transmits the selected kernel information in the form of an index, and the decoder interprets this to apply the same kernel combination, thereby ensuring consistency in the transformation results between the encoder and the decoder. The transmission of this kernel information can be done at the GOP (Group of Pictures) level, Intra-Period level, Picture level, Slice level, Tile level, CTU (Coding Tree Unit) level, CU (Coding Unit) level, or TU (Transform Unit) level. Signaling efficiency can be maximized by defining the kernel information in the form of a pre-configured kernel mapping table and transmitting the index value corresponding to the selected kernel combination. The decoder can reconstruct the transformation kernels for the vertical and horizontal directions from the index value by referring to the same table.

[0204] In another embodiment of the present invention, when a partial transform is applied, the size, location, and orientation of the transform area may be adaptively determined based on the gradient. According to another embodiment of the present invention, the location and orientation of the transform area may be arbitrarily derived using the sum of gradients in the same manner in the video encoder (100) and the video decoder (200). Therefore, separate signaling for the location and orientation of the transform area is unnecessary. The video encoder (100) transmits information regarding whether the partial transform is used for the current block unit by adding it to the bitstream, and may additionally add information regarding the size of the transform area to the bitstream. If only the use of the partial transform is signaled, the size, location, and orientation of the transform area may be determined from the initial transform area using the sum of gradients. If the use of the partial transform and the size are signaled, the location and orientation of the transform area may be determined from the initial transform area. The video encoder (100) determines the final transform area according to the RDO result and adds information regarding the use of the partial transform and / or the size to the bitstream. When the video decoder (200) determines that partial conversion has been applied based on whether partial conversion is applied to the received bitstream, it performs the same gradient-based procedure to derive the optimal position and direction of the conversion area within the current block in the same way as the video encoder (100), and can configure the conversion area based on the position and direction.

[0205] That is, in another embodiment of the present invention, the location and orientation of the transformation region are arbitrarily determined by the encoder and decoder through the same gradient sum calculation procedure, and only information regarding whether partial transformation is used and / or its size is signaled based on the RDO results. Accordingly, while minimizing signaling overhead and maintaining consistency between the encoder and decoder, it is possible to efficiently determine a transformation region that is adaptive to the actual residual energy distribution. For example, the location and orientation of the transformation region are determined by calculating the gradient sum while sliding the boundary in pixel units based on the initial transformation regions, determining the location where the gradient sum is maximum, thereby determining candidate transformation regions, determining one candidate transformation region having the optimal RDO based on the RDO of the candidate transformation regions, and determining the determined single candidate transformation region as the final transformation region. In another embodiment of the present invention, when it is determined that partial transformation has been applied, the video encoder (100) and the video decoder (200) can determine the transformation region by deriving and determining the transformation region based on the gradient.

[0206] FIG. 12 is a diagram illustrating a process for determining a transformation area based on slope according to another embodiment of the present invention.

[0207] Referring to FIG. 12, the video encoder (100) derives and determines candidate transformation regions from the initial transformation regions by calculating the sum of the gradients while moving the boundaries from the initial points of each initial transformation region based on preset initial transformation regions (1201, 1202) and initial transformation regions (1203, 1204) of a first size, and determining the position of the point where the sum of the gradients is maximum. The first size may be half the width and height of the current block (TU), and the second size may be one-fourth the width and height of the current block (TU), but is not limited thereto. Additionally, the first size initial transformation areas (1201, 1202) may be set as a first size initial transformation area (Half-ver) (1201) in the vertical axis direction and a first size initial transformation area (Half-hor) (1202) in the horizontal axis direction, and the second size initial transformation areas (1203, 1204) may be set as a second size initial transformation area (quad-hor) (1203) in the horizontal axis direction and a second size initial transformation area (quad-ver) (1204) in the vertical axis direction. The initial transformation areas (1201, 1202, 1203, 1204) may be set based on a sub-block used in the SBT or determined according to a preset rule.

[0208] Specifically, the video encoder (100) determines the point where the sum of the gradients is maximized by sliding the boundaries of the initial transformation areas in specific units (e.g., 1 pixel unit, 2 pixel unit, 4 pixel unit, etc.) at the initial point of each initial transformation area (1201 to 1204), and determines candidate transformation areas of each size based on the point where the sum of the gradients is maximized. For example, the video encoder (100) calculates the sum of the gradients while moving the boundaries of the first size initial transformation area (Half-ver) (1201) in the vertical axis direction and the first size initial transformation area (Half-hor) (1202) in the horizontal axis direction in units of a predetermined number of pixels, determines the point where the sum of the gradients is maximum, determines candidate transformation areas from each of the first size initial transformation area (Half-ver) (1201) in the vertical axis direction and the first size initial transformation area (Half-hor) (1202) in the horizontal axis direction, and determines one first size candidate transformation area (1205) among the first size candidate transformation areas where the sum of the gradients is maximum. Similarly, the video encoder (100) calculates the sum of the gradients while moving the boundaries of the second size initial transformation area (quad-hor) (1203) in the horizontal axis direction and the second size initial transformation area (quad-ver) (1204) in the vertical axis direction in units of a predetermined number of pixels, determines the point where the sum of the gradients is maximum, determines two candidate transformation areas from each of the second size initial transformation area (quad-hor) (1203) in the horizontal axis direction and the second size initial transformation area (quad-ver) (1204) in the vertical axis direction, and determines one second size candidate transformation area (1206) among the second size candidate transformation areas where the sum of the gradients is maximum.

[0209] Then, the video encoder (100) compares the RDOs of two candidate transformation regions (1205, 1206) to determine one candidate transformation region (1205) having the optimal RDO, and the determined candidate transformation region is determined as the final transformation region.

[0210] The video encoder (100) can add size information of an initial conversion area used to derive a final conversion area based on RDO among preset initial conversion areas (1201, 1202) of a first size and initial conversion areas (1203, 1204) to the bitstream. For example, as shown in FIG. 12, when a final conversion area (1205) is derived and determined from the initial conversion areas (1201, 1202) of the first size, information about the first size can be added to the bitstream and transmitted. In this case, the video decoder (200) can derive a final conversion area by determining the position and direction of the conversion area from the initial conversion areas (1201, 1202) of the first size as described above, without the process of deriving conversion area candidates for the initial conversion areas (1203, 1204) of the second size.

[0211] FIG. 13 is a flowchart of a video encoding method based on partial conversion according to an embodiment of the present invention.

[0212] A video encoding method according to one embodiment of the present invention relates to a method for simultaneously improving conversion efficiency and encoding efficiency by restrictively setting a conversion area where conversion is actually performed within a residual block that is the target of conversion.

[0213] Referring to FIG. 13, the video encoder (100) generates prediction samples of the current block according to the prediction mode of the current block (S1310). The video encoder (100) generates residual samples of the current block by calculating the difference between the prediction samples of the current block and the samples of the current block (S1320).

[0214] Then, the video encoder (100) determines a conversion area in the current block according to the embodiments described above (S1330). The video encoder (100) performs a conversion on the conversion area of ​​the current block to encode the residual samples of the current block (S1340). The conversion process for residual samples in areas not included in the conversion area of ​​the current block is skipped, and the corresponding residual samples can be set to have a value of 0.

[0215] The video encoder (100) generates a bitstream containing conversion area information and encoded residual sample information to determine the area where conversion is performed in the current block (S1350).

[0216] As in the above-described embodiment, the video encoder (100) may set at least one parameter suitable for indicating a transformation area of ​​the current block determined among the first parameter (w_start), the second parameter (w_end), the third parameter (h_start), and the fourth parameter (h_end) in the current block, and may add information about the set at least one parameter to the bitstream. The video encoder (100) may add information about the first transformation mode (And mode), the second transformation mode (Sub-or mode), and the third transformation mode (Full-or mode) applied to the transformation area of ​​the current block to the bitstream. As the signaling method of the first to fourth parameters and transformation modes has been described above, a detailed explanation is omitted.

[0217] In the case where the conversion region is induced and determined as in another embodiment of the present invention described above, the video encoder (100) may add only information indicating whether the partial conversion region is applied to the bitstream.

[0218] FIG. 14 is a flowchart of a video decoding method based on partial conversion according to one embodiment of the present invention.

[0219] Referring to FIG. 14, the video decoder (200) generates prediction samples of the current block according to the prediction mode of the current block (S1410). The video decoder (200) obtains information related to a conversion area from the bitstream and determines a conversion area in the current block according to the embodiments described above based on the obtained information related to the conversion area (S1420). Then, the video decoder (200) performs an inverse conversion on the determined conversion area to obtain residual samples for the conversion area (S1430). For areas not included in the conversion area of ​​the current block, the inverse conversion process is skipped, and the values ​​of the residual samples of the areas not included in the conversion area can be set to 0. Then, the video decoder (200) restores the current block by adding the prediction samples of the current block and the residual samples of the current block (S1440).

[0220] According to the video encoding and decoding method based on partial conversion of the present invention described above, since the conversion is applied only to the area where it is actually needed, the generation of unnecessary coefficients is reduced, thereby improving encoding efficiency, and since the conversion area is limited, the amount of conversion computation for the entire TU is reduced and hardware pipeline efficiency is improved.

[0221] The methods described above in this specification may be performed through a processor of a video encoder or a video decoder. Additionally, the encoder may generate a bitstream that is decoded by a video signal processing method, and the bitstream generated by the encoder may be stored in a computer-readable non-transient storage medium (recording medium).

[0222] The embodiments of the present invention described above may be implemented through various means. For example, the embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof.

[0223] Some embodiments may also be implemented in the form of a recording medium containing computer-executable instructions, such as program modules executed by a computer. A computer-readable medium may be any available medium accessible by a computer and includes both volatile and non-volatile media, and both removable and non-removable media.

[0224] Additionally, computer-readable media may include both computer storage media and communication media. Computer storage media include both volatile and non-volatile, removable and non-removable media implemented by any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data. Communication media typically include other data of modulated data signals, such as computer-readable instructions, data structures, or program modules, or other transmission mechanisms, and include any information transmission media.

[0225] Furthermore, although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the invention as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present invention.

Claims

1. A step of generating prediction samples for the current block; A step of determining a transformation area in the current block based on information regarding the transformation area; A step of restoring residual samples of the current block by performing an inverse transformation on the transformation region of the current block; and The method includes the step of restoring the current block based on the predicted samples of the current block and the residual samples, and A video decoding method comprising information regarding the above-mentioned transformation area, wherein the information includes information for determining the area where transformation was performed in the above-mentioned current block.

2. In Paragraph 1, Information regarding the above transformation area It includes at least one parameter among a first parameter (w_start) indicating the start position of the transformation area in the horizontal axis direction in the current block, a second parameter (w_end) indicating the end position of the transformation area in the horizontal axis direction, a third parameter (h_start) indicating the start position of the transformation area in the vertical axis direction, and a fourth parameter (h_end) indicating the end position of the transformation area in the vertical axis direction. A video decoding method in which the transformation region is determined according to at least one parameter.

3. In Paragraph 2, The step of determining the transformation area in the current block above If the information regarding the transformation region includes at least one of the first parameter (w_start) and the second parameter (w_end), the method comprises the step of determining a first transformation region in the horizontal axis direction using at least one of the first parameter (w_start) and the second parameter (w_end); and A video decoding method comprising the step of determining a second transformation region in the vertical axis direction using at least one of the third parameter (h_start) and the fourth parameter (h_end) when the information regarding the transformation region includes at least one of the third parameter (h_start) and the fourth parameter (h_end).

4. In Paragraph 2, Information regarding the above transformation area A first transformation mode that determines the area where the first transformation area in the horizontal axis direction and the second transformation area in the vertical axis direction overlap as the transformation area, A second transformation mode that determines a third transformation area including an area up to a position in the vertical axis direction pointed to by the fourth parameter (h_end) among the first transformation area in the horizontal axis direction, and a fourth transformation area including an area up to a position in the horizontal axis direction pointed to by the second parameter (w_end) among the second transformation area in the vertical axis direction, and determines an area included in at least one of the third transformation area and the fourth transformation area as a transformation area, and It further includes conversion mode information pointing to one of a third conversion mode that determines a region included in at least one of the first conversion region in the horizontal axis direction and the second conversion region in the vertical axis direction as a conversion region, and A video decoding method in which the above conversion region is determined according to the conversion mode information indicating one of the first to third conversion modes.

5. In Paragraph 4, The above first to fourth parameters and the transformation mode information are each signaled using a predetermined index, or Signaling using a horizontal axis direction index pointing to a single element pointing to the first parameter and the second parameter applied to the current block among a grouped horizontal axis direction transformation area table having an element combining the first parameter and the second parameter, a vertical axis direction index pointing to a single element pointing to the third parameter and the fourth parameter applied to the current block among a grouped vertical axis direction transformation area table having an element combining the third parameter and the fourth parameter, and an index pointing to the transformation mode, Signaling using an index pointing to an element pointing to the first to fourth parameters applied to the current block among a grouped transformation area table having elements combining the first to fourth parameters, and an index pointing to the transformation mode, A video decoding method that is signaled using an index pointing to one element pointing to the first to fourth parameters and the conversion mode applied to the current block among a grouped table having elements combining the first to fourth parameters and the conversion mode information.

6. In Paragraph 1, The step of restoring the residual samples of the current block mentioned above A video decoding method that divides the transformation area of ​​the current block determined above into one or more sub-blocks and rearranges the one or more sub-blocks according to a predetermined scan order to generate one or more transformation blocks.

7. In Paragraph 6, The step of restoring the residual samples of the current block mentioned above A video decoding method that performs a first inverse transformation based on one or more rearranged transformation blocks and performs a second inverse transformation on a portion of one or more of the first inversely transformed transformation blocks.

8. In Paragraph 1, A video decoding method in which the transformation area of ​​the current block is determined as one of predetermined transformation area patterns including at least one of the upper-left, upper-right, lower-left, lower-right corner transformation area and center transformation area of ​​the current block.

9. In Paragraph 1, The transformation area of ​​the current block includes at least one of the upper-left, upper-right, lower-left, and lower-right corner transformation areas of the current block, and Information regarding the above transformation area A video decoding method comprising corner transformation area information for indicating the upper-left, upper-right, lower-left, and lower-right corner areas, and size setting index information for determining the size of the corner transformation area.

10. In Paragraph 9, Information regarding the above transformation area The above corner transformation area information includes first corner transformation area information indicating the upper / lower position of the corner transformation area and second corner transformation area information indicating the left / right position of the corner transformation area, and A video decoding method comprising the above-mentioned size setting index information including information indicating whether the width or height of the corner transformation area is a size divided by 1 / 2 or a size divided by 1 / 4 of the width or height of the current block.

11. In Paragraph 1, The information regarding the above-mentioned transformation area includes information indicating whether partial transformation is applied, The step of determining the transformation area in the current block above A video decoding method that determines a transformation region by determining a transformation region derived from a preset candidate transformation region of the current block using a gradient when information indicating whether the partial transformation is applied indicates that the partial transformation is applied.

12. A video decoding device that performs the method of any one of paragraphs 1 to 11.

13. A computer-readable recording medium storing a program for executing the method of any one of paragraphs 1 through 11 on a computer.

14. Step of generating prediction samples for the current block; A step of generating residual samples of the current block using the predicted samples of the current block and the samples of the current block; A step of determining a transformation region in the current block above; A step of encoding residual samples of the current block by performing a transformation on the transformation region of the current block; and A video encoding method for generating a bitstream including conversion area information for determining the area where conversion is performed in the current block and the encoded residual sample information.

15. A video encoding device that performs the method of paragraph 14.

16. A computer-readable recording medium storing a program for executing the method of paragraph 14 on a computer.

Images