Method and apparatus for video coding based on sign prediction of transform skip block

Abstract

The present embodiment discloses a video coding method and apparatus based on sign prediction of a transform skip block. In the present embodiment, if it is determined to perform a transform skip on the current block, an image decoding apparatus decodes an absolute value and sign information of a residual sample, and acquires a residual signal of the current block on the basis of the absolute value and sign information of the residual sample. If the sign information is determined as a sign error flag, the image decoding apparatus predicts a sign of the residual sample on the basis of pre-decoded neighboring samples of the residual sample and a virtual prediction sample, determines the sign of the residual sample on the basis of the predicted sign and the sign error flag, and generates a quantized residual signal by combining absolute values and determined signs of residual samples.

Description

Method and device for video coding based on sign prediction of transform skip block

[0001] The present disclosure relates to a video coding method and device based on sign prediction of a transform skip block.

[0002] The content described below merely provides background information related to the present invention and does not constitute prior art.

[0003] Since video data has a large amount of data compared to voice data or still image data, it requires a lot of hardware resources, including memory, to store or transmit it without processing for compression.

[0004] Therefore, when storing or transmitting video data, an encoder is used to compress the video data and store or transmit it, and a decoder receives the compressed video data, decompresses it, and plays it back. Examples of such video compression technologies include H.264 / AVC, HEVC (High Efficiency Video Coding), and VVC (Versatile Video Coding), which improves encoding efficiency by about 30% compared to HEVC.

[0005] However, as the size, resolution, and frame rate of images are gradually increasing, and the amount of data that needs to be encoded is also increasing, a new compression technology that has better encoding efficiency and better image quality improvement than existing compression technologies is required.

[0006] In Regular Residual Coding (RRC) of VVC, bypass coding is applied to the sign. Beyond VVC, Enhanced Compression Model (ECM) adopts a coding method based on sign prediction. For example, instead of bypass coding the sign, sign prediction of RRC improves the performance of sign coding by predicting the sign of a given region / coefficient and context coding the prediction error of the sign. Bypass coding is applied to the remaining codes except for the code of the given region / coefficient. The code is predicted based on hypothesis border reconstruction. To predict n codes, hypothesis border reconstruction is 2 n For each possible sign combination of a branch, a virtual reconstruction value is generated based on the inverse transformation. The virtual boundary reconstruction generates a predicted value according to a predefined method using the pixel values ​​of the upper and left boundaries of the TU (Transform Unit), and a cost function is calculated based on the difference between the predicted value and the virtual reconstruction value. The virtual boundary reconstruction predicts n signs using the set of signs that minimizes the cost function.

[0007] As mentioned above, virtual boundary restoration is applied to residual blocks to which transformations have been applied. Therefore, similar to residual blocks to which transformations have been applied, sign prediction also needs to be considered for blocks to which transformation skips have been applied.

[0008] The present disclosure aims to provide a video coding method and device for predicting a sign of a residual signal based on a surrounding restoration signal-based sign prediction in relation to residual signal encoding of a transform skip block, and context-encoding a prediction error of the sign.

[0009] According to an embodiment of the present disclosure, a method for restoring a current block, performed by an image decoding device, is provided, comprising: a step of decoding a transform skip flag from a bitstream; and a step of determining whether to perform a transform skip for the current block based on the transform skip flag, and in a case where it is determined to perform the transform skip, a step of decoding an absolute value and sign information of a residual sample from the bitstream and obtaining a residual signal of the current block based on the absolute value of the residual sample and the sign information, and in a case where the sign information is determined to be a sign error flag, a step of predicting a sign of the residual sample based on decoded neighboring samples of the residual sample and a virtual prediction sample; a step of determining a sign of the residual sample based on the predicted sign and the sign error flag; and a step of generating the quantized residual signal by combining the absolute values ​​of the residual samples and the determined signs.

[0010] According to another embodiment of the present disclosure, a method for encoding a current block, performed by a video encoding device, is provided, comprising: obtaining a transform skip flag for the current block; and determining whether to perform transform skip for the current block based on the transform skip flag, and, in a case where it is determined to perform the transform skip, further comprising: generating parameters of a residual signal of the current block based on an absolute value and sign information of a residual sample, and, in a case where the sign information is a sign error flag, predicting a sign of the residual sample based on decoded neighboring samples of the residual sample and a virtual prediction sample; and determining a sign error flag of the residual sample based on the predicted sign and an original sign of the residual sample.

[0011] According to another embodiment of the present disclosure, a method for providing video data to a video decoding device is provided, comprising: encoding the video data into a bitstream; and transmitting the bitstream to the video decoding device, wherein the encoding the video data comprises: obtaining a transform skip flag for a current block; and determining whether to perform a transform skip for the current block based on the transform skip flag; and, in a case where it is determined to perform the transform skip, further comprising: generating parameters of a residual signal of the current block based on an absolute value and sign information of a residual sample; and, in a case where the sign information is a sign error flag, predicting a sign of the residual sample based on decoded neighboring samples of the residual sample and a virtual prediction sample; and determining a sign error flag of the residual sample based on the predicted sign and an original sign of the residual sample.

[0012] As described above, according to the present embodiment, in relation to residual signal encoding of a transform skip block, a video coding method and device are provided that predicts the sign of a residual signal based on a surrounding restoration signal-based sign prediction and context-encodes the prediction error of the sign, thereby making it possible to improve objective video encoding efficiency and subjective video quality.

[0013] FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the techniques of the present disclosure.

[0014] Figure 2 is a drawing for explaining a method of dividing a block using the QTBTTT (QuadTree plus BinaryTree TernaryTree) structure.

[0015] FIGS. 3A and 3B are diagrams illustrating multiple intra prediction modes, including wide-angle intra prediction modes.

[0016] Figure 4 is an example diagram of the surrounding blocks of the current block.

[0017] FIG. 5 is an exemplary block diagram of an image decoding device capable of implementing the techniques of the present disclosure.

[0018] Figure 6 is an example diagram showing virtual boundary restoration.

[0019] FIG. 7 is an exemplary diagram showing restoration of a conversion skip block according to one embodiment of the present disclosure.

[0020] FIG. 8 is an exemplary diagram showing the scanning order of subblocks according to one embodiment of the present disclosure.

[0021] FIG. 9 is an exemplary diagram showing a scanning order within a subblock according to one embodiment of the present disclosure.

[0022] FIG. 10 is an exemplary diagram showing a scanning order within a subblock according to another embodiment of the present disclosure.

[0023] FIG. 11 is an exemplary diagram showing a scanning order within a subblock according to another embodiment of the present disclosure.

[0024] FIG. 12 is an exemplary diagram showing a group of syntax parameters according to one embodiment of the present disclosure.

[0025] FIG. 13 is an exemplary diagram showing surrounding samples for sign prediction according to another embodiment of the present disclosure.

[0026] FIG. 14 is an exemplary diagram showing a group of syntax parameters according to another embodiment of the present disclosure.

[0027] FIG. 15 is an exemplary diagram showing signs of surrounding samples according to one embodiment of the present disclosure.

[0028] FIG. 16 is an exemplary diagram showing a group of syntax parameters according to one embodiment of the present disclosure.

[0029] FIG. 17 is an exemplary diagram showing a group of syntax parameters according to one embodiment of the present disclosure.

[0030] FIG. 18 is a flowchart illustrating a method for an image encoding device to encode a current block according to an embodiment of the present disclosure.

[0031] FIG. 19 is a flowchart illustrating a method for an image decoding device to restore a current block according to one embodiment of the present disclosure.

[0032] Hereinafter, embodiments of the present invention will be described in detail with reference to exemplary drawings. When designating components in each drawing, it should be noted that, where possible, identical components are given the same reference numerals, even if they appear in different drawings. Furthermore, in describing the present embodiments, detailed descriptions of related known structures or functions will be omitted if they are deemed to obscure the gist of the present embodiments.

[0033] FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the techniques of the present disclosure. Hereinafter, the image encoding device and its subcomponents will be described with reference to the illustration in FIG. 1.

[0034] The video encoding device may be configured to include a picture segmentation unit (110), a prediction unit (120), a subtractor (130), a transformation unit (140), a quantization unit (145), a reordering unit (150), an entropy encoding unit (155), an inverse quantization unit (160), an inverse transformation unit (165), an adder (170), a loop filter unit (180), and a memory (190).

[0035] Each component of the video encoding device may be implemented in hardware, software, or a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, with a microprocessor executing the software functions corresponding to each component.

[0036] A single image (video) is composed of one or more sequences containing multiple pictures. Each picture is divided into multiple regions, and encoding is performed for each region. For example, a single picture is divided into one or more tiles and / or slices. Here, one or more tiles can be defined as a tile group. Each tile or slice is divided into one or more Coding Tree Units (CTUs). Each CTU is then divided into one or more Coding Units (CUs) by a tree structure. Information applied to each CU is encoded as the syntax of the CU, and information commonly applied to CUs included in a CTU is encoded as the syntax of the CTU. In addition, information commonly applied to all blocks within a single slice is encoded as the syntax of the slice header, and information applied to all blocks constituting one or more pictures is encoded in the Picture Parameter Set (PPS) or the picture header. Furthermore, information commonly referenced by multiple pictures is encoded in a Sequence Parameter Set (SPS). And, information commonly referenced by one or more SPS is encoded in a Video Parameter Set (VPS). In addition, information commonly applied to one tile or tile group may be encoded as syntax of a tile or tile group header. Syntaxes included in an SPS, PPS, slice header, tile or tile group header may be referred to as high level syntax.

[0037] The picture segmentation unit (110) determines the size of the CTU. Information about the size of the CTU (CTU size) is encoded as the syntax of SPS or PPS and transmitted to the image decoding device.

[0038] The picture segmentation unit (110) divides each picture constituting an image into a plurality of CTUs having a predetermined size, and then recursively divides the CTUs using a tree structure. A leaf node in the tree structure becomes a CU, which is a basic unit of encoding.

[0039] The tree structure may be a QuadTree (QT) in which an upper node (or parent node) is divided into four lower nodes (or child nodes) of the same size, a BinaryTree (BT) in which an upper node is divided into two lower nodes, or a TernaryTree (TT) in which an upper node is divided into three lower nodes in a 1:2:1 ratio, or a structure that mixes two or more of the QT structures, BT structures, and TT structures. For example, a QTBT (QuadTree plus BinaryTree) structure may be used, or a QTBTTT (QuadTree plus BinaryTree TernaryTree) structure may be used. Here, BTTT may be combined and referred to as a MTT (Multiple-Type Tree).

[0040] Figure 2 is a drawing for explaining a method of dividing a block using the QTBTTT structure.

[0041] As illustrated in FIG. 2, a CTU may first be split into a QT structure. The quadtree splitting may be repeated until the size of the splitting block reaches the minimum block size (MinQTSize) of the leaf node allowed in the QT. A first flag (QT_split_flag) indicating whether each node of the QT structure is split into four nodes of the lower layer is encoded by the entropy encoding unit (155) and signaled to the image decoding device. If the leaf node of the QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in the BT, it may be further split into one or more of the BT structure or the TT structure. There may be multiple splitting directions in the BT structure and / or the TT structure. For example, there may be two directions in which the block of the corresponding node is split horizontally and two directions in which the block is split vertically. As illustrated in FIG. 2, when MTT splitting begins, a second flag (mtt_split_flag) indicating whether nodes have been split, and if splitting has occurred, a flag indicating the splitting direction (vertical or horizontal) and / or a flag indicating the splitting type (Binary or Ternary) are encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0042] Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node is split into four nodes of a lower layer, a CU split flag (split_cu_flag) indicating whether the node is split may be encoded. If the CU split flag (split_cu_flag) value indicates that the node is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a CU (coding unit), which is a basic unit of encoding. If the CU split flag (split_cu_flag) value indicates that the node is split, the video encoding device starts encoding from the first flag in the above-described manner.

[0043] As another example of a tree structure, when QTBT is used, there may be two types: a type that horizontally splits the block of the corresponding node into two blocks of the same size (i.e., symmetric horizontal splitting) and a type that vertically splits it (i.e., symmetric vertical splitting). A split flag (split_flag) indicating whether each node of the BT structure is split into blocks of a lower layer and split type information indicating the type of split are encoded by the entropy encoding unit (155) and transmitted to the image decoding device. Meanwhile, there may additionally be a type that splits the block of the corresponding node into two blocks of an asymmetrical shape. The asymmetric shape may include a shape that splits the block of the corresponding node into two rectangular blocks with a size ratio of 1:3, or a shape that splits the block of the corresponding node in a diagonal direction.

[0044] A CU can have various sizes depending on the QTBT or QTBTTT partitioning from the CTU. Hereinafter, the block corresponding to the CU to be encoded or decoded (i.e., the leaf node of the QTBTTT) is referred to as the "current block." Depending on the QTBTTT partitioning employed, the current block may be rectangular as well as square.

[0045] The prediction unit (120) predicts the current block and generates a prediction block. The prediction unit (120) includes an intra prediction unit (122) and an inter prediction unit (124).

[0046] In general, each current block within a picture can be predictively coded. Prediction of the current block can typically be performed using either intra-prediction (using data from the picture containing the current block) or inter-prediction (using data from a picture coded before the picture containing the current block). Inter-prediction encompasses both unidirectional and bidirectional prediction.

[0047] The intra prediction unit (122) predicts pixels within the current block using pixels (reference pixels) located around the current block within the current picture including the current block. There are multiple intra prediction modes depending on the prediction direction. For example, as shown in Fig. 3a, the multiple intra prediction modes may include two non-directional modes including the Planar mode and the DC mode, and 65 directional modes. The surrounding pixels to be used and the calculation formula are defined differently depending on each prediction mode.

[0048] For efficient directional prediction for a rectangular current block, directional modes (intra prediction modes 67 to 80 and -1 to -14) indicated by dotted arrows in Fig. 3b may be additionally used. These may be referred to as "wide-angle intra-prediction modes." In Fig. 3b, the arrows point to corresponding reference samples used for prediction, and do not indicate the prediction direction. The prediction direction is opposite to the direction indicated by the arrows. Wide-angle intra-prediction modes are modes that perform prediction in the opposite direction of a specific directional mode without additional bit transmission when the current block is rectangular. At this time, among the wide-angle intra-prediction modes, some wide-angle intra-prediction modes available for the current block may be determined based on the ratio of the width and height of the rectangular current block. For example, wide-angle intra prediction modes (intra prediction modes 67 to 80) having an angle less than 45 degrees are available when the current block is a rectangular shape whose height is smaller than its width, and wide-angle intra prediction modes (intra prediction modes -1 to -14) having an angle greater than -135 degrees are available when the current block is a rectangular shape whose width is larger than its height.

[0049] The intra prediction unit (122) can determine an intra prediction mode to be used to encode the current block. In some examples, the intra prediction unit (122) can encode the current block using multiple intra prediction modes and select an appropriate intra prediction mode to be used from the tested modes. For example, the intra prediction unit (122) can calculate bit-rate distortion values ​​using rate-distortion analysis for multiple tested intra prediction modes and select an intra prediction mode with the best bit-rate distortion characteristics among the tested modes.

[0050] The intra prediction unit (122) selects one intra prediction mode from among multiple intra prediction modes and predicts the current block using surrounding pixels (reference pixels) and an operation formula determined according to the selected intra prediction mode. Information about the selected intra prediction mode is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

[0051] The inter prediction unit (124) generates a prediction block for the current block using a motion compensation process. The inter prediction unit (124) searches for a block most similar to the current block within reference pictures that were encoded and decoded before the current picture, and generates a prediction block for the current block using the searched block. Then, a motion vector (MV) corresponding to the displacement between the current block within the current picture and the prediction block within the reference picture is generated. Generally, motion estimation is performed on the luma component, and the motion vector calculated based on the luma component is used for both the luma component and the chroma component. The motion information including information on the reference picture used to predict the current block and information on the motion vector is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

[0052] The inter prediction unit (124) may perform interpolation on a reference picture or a reference block to improve prediction accuracy. That is, subsamples between two consecutive integer samples are interpolated by applying filter coefficients to a plurality of consecutive integer samples including the two integer samples. When a process of searching for a block most similar to the current block is performed on the interpolated reference picture, the motion vector can be expressed up to a precision in decimal units rather than a precision in integer sample units. The precision or resolution of the motion vector can be set differently for each target region to be encoded, such as a slice, tile, CTU, CU, etc. When such adaptive motion vector resolution (AMVR) is applied, information on the motion vector resolution to be applied to each target region must be signaled for each target region. For example, when the target region is a CU, information on the motion vector resolution applied to each CU is signaled. Information on the motion vector resolution may be information indicating the precision of a differential motion vector, which will be described later.

[0053] Meanwhile, the inter prediction unit (124) can perform inter prediction using bi-prediction. In the case of bi-prediction, two reference pictures and two motion vectors indicating the block position most similar to the current block within each reference picture are used. The inter prediction unit (124) selects a first reference picture and a second reference picture from reference picture list 0 (RefPicList0) and reference picture list 1 (RefPicList1), respectively, and searches for a block similar to the current block within each reference picture to generate a first reference block and a second reference block. Then, the first reference block and the second reference block are averaged or weighted averaged to generate a prediction block for the current block. Then, motion information including information on two reference pictures used to predict the current block and information on two motion vectors is transmitted to the entropy encoding unit (155). Here, reference picture list 0 may be composed of pictures that are before the current picture in display order among the restored pictures, and reference picture list 1 may be composed of pictures that are after the current picture in display order among the restored pictures. However, this is not necessarily limited to this, and restored pictures that are after the current picture in display order may be additionally included in reference picture list 0, and conversely, restored pictures that are before the current picture may be additionally included in reference picture list 1.

[0054] Various methods can be used to minimize the number of bits required to encode motion information.

[0055] For example, if the reference picture and motion vector of the current block are identical to those of a neighboring block, the motion information of the current block can be transmitted to the image decoding device by encoding information that can identify the neighboring block. This method is called 'merge mode'.

[0056] In merge mode, the inter prediction unit (124) selects a predetermined number of merge candidate blocks (hereinafter referred to as 'merge candidates') from the surrounding blocks of the current block.

[0057] As the surrounding blocks for deriving merge candidates, all or part of the left block (A0), the lower left block (A1), the upper block (B0), the upper right block (B1), and the upper left block (B2) adjacent to the current block within the current picture may be used, as illustrated in FIG. 4. In addition, a block located within a reference picture (which may or may not be the same as the reference picture used to predict the current block) other than the current picture in which the current block is located may be used as a merge candidate. For example, a block co-located with the current block within the reference picture or blocks adjacent to the block at the co-located block may be additionally used as a merge candidate. If the number of merge candidates selected by the method described above is less than a preset number, a 0 vector is added to the merge candidates.

[0058] The inter prediction unit (124) uses these surrounding blocks to construct a merge list containing a predetermined number of merge candidates. Among the merge candidates included in the merge list, the merge candidate to be used as motion information of the current block is selected and merge index information for identifying the selected candidate is generated. The generated merge index information is encoded by the entropy encoding unit (155) and transmitted to the video decoding device.

[0059] Merge Skip mode is a special case of merge mode. After quantization, when all transform coefficients for entropy encoding are close to zero, only neighboring block selection information is transmitted without transmitting residual signals. By utilizing merge skip mode, relatively high encoding efficiency can be achieved for low-motion images, still images, and screen content images.

[0060] Hereinafter, merge mode and merge skip mode are collectively referred to as merge / skip mode.

[0061] Another method for encoding motion information is Advanced Motion Vector Prediction (AMVP) mode.

[0062] In AMVP mode, the inter prediction unit (124) derives predicted motion vector candidates for the motion vector of the current block using neighboring blocks of the current block. As neighboring blocks used to derive predicted motion vector candidates, all or some of the left block (A0), the lower left block (A1), the upper block (B0), the upper right block (B1), and the upper left block (B2) adjacent to the current block in the current picture as shown in FIG. 4 may be used. In addition, a block located in a reference picture (which may or may not be the same as the reference picture used to predict the current block) other than the current picture in which the current block is located may be used as the neighboring block used to derive predicted motion vector candidates. For example, a block co-located with the current block in the reference picture or blocks adjacent to the block in the co-located block may be used. If the number of motion vector candidates is less than a preset number by the method described above, a 0 vector is added to the motion vector candidates.

[0063] The inter prediction unit (124) derives predicted motion vector candidates using the motion vectors of these surrounding blocks, and determines a predicted motion vector for the motion vector of the current block using the predicted motion vector candidates. Then, the predicted motion vector is subtracted from the motion vector of the current block to produce a differential motion vector.

[0064] The predicted motion vector can be obtained by applying a predefined function (e.g., median, mean, etc.) to the predicted motion vector candidates. In this case, the image decoding device also knows the predefined function. In addition, since the surrounding blocks used to derive the predicted motion vector candidates are blocks that have already been encoded and decoded, the image decoding device also already knows the motion vectors of the surrounding blocks. Therefore, the image encoding device does not need to encode information to identify the predicted motion vector candidates. Therefore, in this case, information about the differential motion vector and information about the reference picture used to predict the current block are encoded.

[0065] Alternatively, the predicted motion vector can be determined by selecting one of the predicted motion vector candidates. In this case, information for identifying the selected predicted motion vector candidate is additionally encoded, along with information about the differential motion vector and the reference picture used to predict the current block.

[0066] The subtractor (130) subtracts the prediction block generated by the intra prediction unit (122) or inter prediction unit (124) from the current block to generate a residual block.

[0067] The transformation unit (140) transforms residual signals within a residual block having pixel values ​​in a spatial domain into transform coefficients in a frequency domain. The transformation unit (140) may transform the residual signals within the residual block using the entire size of the residual block as a transformation unit, or may divide the residual block into a plurality of sub-blocks and use the sub-blocks as transformation units to perform the transformation. Alternatively, the residual signals may be transformed using only the transformation domain sub-block as a transformation unit by dividing the sub-blocks into two sub-blocks, that is, a transformation domain and a non-transform domain. Here, the transformation domain sub-block may be one of two rectangular blocks having a size ratio of 1:1 with respect to the horizontal axis (or vertical axis). In this case, a flag (cu_sbt_flag) indicating that only a sub-block has been converted, directionality (vertical / horizontal) information (cu_sbt_horizontal_flag), and / or position information (cu_sbt_pos_flag) are encoded by the entropy encoding unit (155) and signaled to the image decoding device. In addition, the size of the conversion area sub-block may have a size ratio of 1:3 with respect to the horizontal axis (or vertical axis), and in this case, a flag (cu_sbt_quad_flag) distinguishing the corresponding division is additionally encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0068] Meanwhile, the transformation unit (140) can individually perform transformations on the residual block in the horizontal and vertical directions. For the transformation, various types of transformation functions or transformation matrices can be used. For example, a pair of transformation functions for horizontal transformation and vertical transformation can be defined as a Multiple Transform Set (MTS). The transformation unit (140) can select one transformation function pair with the best transformation efficiency among the MTS and transform the residual block in the horizontal and vertical directions, respectively. Information (mts_idx) on the transformation function pair selected among the MTS is encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0069] The quantization unit (145) quantizes the transform coefficients output from the transform unit (140) using quantization parameters and outputs the quantized transform coefficients to the entropy encoding unit (155). The quantization unit (145) may directly quantize a related residual block without transformation for a certain block or frame. The quantization unit (145) may also apply different quantization coefficients (scaling values) according to the positions of the transform coefficients within the transform block. The quantization matrix applied to the quantized transform coefficients arranged in two dimensions may be encoded and signaled to an image decoding device.

[0070] The rearrangement unit (150) can perform rearrangement of coefficient values ​​for quantized residual values.

[0071] The reordering unit (150) can change a two-dimensional coefficient array into a one-dimensional coefficient sequence by using coefficient scanning. For example, the reordering unit (150) can output a one-dimensional coefficient sequence by scanning from the DC coefficient to the coefficients of the high-frequency region by using a zig-zag scan or a diagonal scan. Depending on the size of the transformation unit and the intra prediction mode, a vertical scan that scans the two-dimensional coefficient array in the column direction or a horizontal scan that scans the two-dimensional block-shaped coefficients in the row direction may be used instead of the zig-zag scan. That is, depending on the size of the transformation unit and the intra prediction mode, the scanning method to be used may be determined among the zig-zag scan, the diagonal scan, the vertical scan, and the horizontal scan.

[0072] The entropy encoding unit (155) generates a bitstream by encoding a sequence of one-dimensional quantized transform coefficients output from the rearrangement unit (150) using various encoding methods such as CABAC (Context-based Adaptive Binary Arithmetic Code) and Exponential Golomb.

[0073] In addition, the entropy encoding unit (155) encodes information related to block division, such as CTU size, CU division flag, QT division flag, MTT division type, and MTT division direction, so that the image decoding device can divide the block in the same manner as the image encoding device. In addition, the entropy encoding unit (155) encodes information about the prediction type indicating whether the current block is encoded by intra prediction or inter prediction, and encodes intra prediction information (i.e., information about the intra prediction mode) or inter prediction information (information about the encoding mode of motion information (merge mode or AMVP mode), a merge index in the case of the merge mode, and a reference picture index and a differential motion vector in the case of the AMVP mode) according to the prediction type. In addition, the entropy encoding unit (155) encodes information related to quantization, that is, information about quantization parameters and information about a quantization matrix.

[0074] The inverse quantization unit (160) inversely quantizes the quantized transform coefficients output from the quantization unit (145) to generate transform coefficients. The inverse transform unit (165) transforms the transform coefficients output from the inverse quantization unit (160) from the frequency domain to the spatial domain to restore the residual block.

[0075] An adder (170) adds the restored residual block and the predicted block generated by the prediction unit (120) to restore the current block. The pixels within the restored current block are used as reference pixels when intra-predicting the next block.

[0076] The loop filter unit (180) performs filtering on restored pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. that occur due to block-based prediction and transformation / quantization. The loop filter unit (180) may include all or part of a deblocking filter (182), a sample adaptive offset (SAO) filter (184), and an adaptive loop filter (ALF, 186) as an in-loop filter.

[0077] The deblocking filter (182) filters the boundaries between restored blocks to remove blocking artifacts caused by block-based encoding / decoding, and the SAO filter (184) and the ALF (186) perform additional filtering on the deblocking-filtered image. The SAO filter (184) and the ALF (186) are filters used to compensate for the differences between restored pixels and original pixels caused by lossy coding. The SAO filter (184) improves not only subjective image quality but also encoding efficiency by applying an offset in units of CTUs. In contrast, the ALF (186) performs block-based filtering, and compensates for distortion by applying different filters by distinguishing the edges and degrees of variation of the corresponding block. Information on filter coefficients to be used in the ALF can be encoded and signaled to an image decoding device.

[0078] The restored blocks filtered through the deblocking filter (182), SAO filter (184), and ALF (186) are stored in the memory (190). When all blocks within a picture are restored, the restored picture can be used as a reference picture for inter-predicting blocks within a picture to be encoded later.

[0079] The video encoding device can store the bitstream of encoded video data on a non-transitory storage medium or transmit it to the video decoding device using a communication network.

[0080] FIG. 5 is an exemplary block diagram of an image decoding device capable of implementing the techniques of the present disclosure. Hereinafter, the image decoding device and its subcomponents will be described with reference to FIG. 5.

[0081] The video decoding device may be configured to include an entropy decoding unit (510), a rearrangement unit (515), an inverse quantization unit (520), an inverse transformation unit (530), a prediction unit (540), an adder (550), a loop filter unit (560), and a memory (570).

[0082] Similar to the video encoding device of FIG. 1, each component of the video decoding device may be implemented in hardware, software, or a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, with a microprocessor executing the software functions corresponding to each component.

[0083] The entropy decoding unit (510) decodes the bitstream generated by the image encoding device to extract information related to block division, thereby determining the current block to be decoded, and extracts prediction information, information on residual signals, etc. required to restore the current block.

[0084] The entropy decoding unit (510) extracts information about the CTU size from the Sequence Parameter Set (SPS) or the Picture Parameter Set (PPS), determines the size of the CTU, and divides the picture into CTUs of the determined size. Then, the CTU is determined as the top layer of the tree structure, i.e., the root node, and the CTU is divided using the tree structure by extracting division information about the CTU.

[0085] For example, when splitting a CTU using the QTBTTT structure, first, the first flag (QT_split_flag) related to the splitting of QT is extracted, and each node is split into four nodes of the lower layer. Then, for the nodes corresponding to the leaf nodes of QT, the second flag (mtt_split_flag) related to the splitting of MTT and the split direction (vertical / horizontal) and / or split type (binary / ternary) information are extracted, and the corresponding leaf nodes are split into the MTT structure. Accordingly, each node below the leaf nodes of QT are split recursively into the BT or TT structure.

[0086] As another example, when splitting a CTU using the QTBTTT structure, the CU split flag (split_cu_flag) indicating whether the CU is split is first extracted, and if the block is split, the first flag (QT_split_flag) may be extracted. During the splitting process, each node may undergo zero or more repeated QT splits followed by zero or more repeated MTT splits. For example, a CTU may undergo an MTT split right away, or conversely, may undergo only multiple QT splits.

[0087] As another example, when splitting a CTU using the QTBT structure, the first flag (QT_split_flag) related to the splitting of QT is extracted, and each node is split into four nodes of the lower layer. Furthermore, for nodes corresponding to leaf nodes of QT, a split flag (split_flag) indicating whether to further split into BTs and splitting direction information are extracted.

[0088] Meanwhile, when the entropy decoding unit (510) determines the current block to be decoded by using the division of the tree structure, it extracts information on the prediction type indicating whether the current block is intra-predicted or inter-predicted. If the prediction type information indicates intra-prediction, the entropy decoding unit (510) extracts syntax elements for intra-prediction information (intra-prediction mode) of the current block. If the prediction type information indicates inter-prediction, the entropy decoding unit (510) extracts syntax elements for inter-prediction information, i.e., information indicating a motion vector and a reference picture referenced by the motion vector.

[0089] Additionally, the entropy decoding unit (510) extracts information about the quantized transform coefficients of the current block as information related to quantization and information about residual signals.

[0090] The rearrangement unit (515) can change the sequence of one-dimensional quantized transform coefficients entropy-decoded in the entropy decoding unit (510) back into a two-dimensional coefficient array (i.e., block) in the reverse order of the coefficient scanning performed by the image encoding device.

[0091] The inverse quantization unit (520) inversely quantizes the quantized transform coefficients and inversely quantizes the quantized transform coefficients using the quantization parameters. The inverse quantization unit (520) may also apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in two dimensions. The inverse quantization unit (520) may perform inverse quantization by applying a matrix of quantized coefficients (scaling values) from an image encoding device to a two-dimensional array of quantized transform coefficients.

[0092] The inverse transform unit (530) inversely transforms the inverse quantized transform coefficients from the frequency domain to the spatial domain to restore residual signals, thereby generating a residual block for the current block.

[0093] In addition, when the inverse transform unit (530) inversely transforms only a portion of a transform block (sub-block), it extracts a flag (cu_sbt_flag) indicating that only a sub-block of the transform block has been transformed, directionality (vertical / horizontal) information (cu_sbt_horizontal_flag) of the sub-block, and / or position information (cu_sbt_pos_flag) of the sub-block, and inversely transforms the transform coefficients of the corresponding sub-block from the frequency domain to the spatial domain to restore residual signals, and fills “0” values ​​with residual signals for areas that have not been inversely transformed, thereby generating a final residual block for the current block.

[0094] In addition, when MTS is applied, the inverse transform unit (530) determines a transform function or a transform matrix to be applied in the horizontal and vertical directions using MTS information (mts_idx) signaled from the image encoding device, and performs inverse transform on the transform coefficients within the transform block in the horizontal and vertical directions using the determined transform function.

[0095] The prediction unit (540) may include an intra prediction unit (542) and an inter prediction unit (544). The intra prediction unit (542) is activated when the prediction type of the current block is intra prediction, and the inter prediction unit (544) is activated when the prediction type of the current block is inter prediction.

[0096] The intra prediction unit (542) determines the intra prediction mode of the current block among a plurality of intra prediction modes from the syntax elements for the intra prediction mode extracted from the entropy decoding unit (510), and predicts the current block using reference pixels around the current block according to the intra prediction mode.

[0097] The inter prediction unit (544) uses the syntax elements for the inter prediction mode extracted from the entropy decoding unit (510) to determine the motion vector of the current block and the reference picture referenced by the motion vector, and predicts the current block using the motion vector and the reference picture.

[0098] An adder (550) adds the residual block output from the inverse transform unit (530) and the predicted block output from the inter prediction unit (544) or the intra prediction unit (542) to restore the current block. The pixels within the restored current block are used as reference pixels when intra-predicting a block to be decoded later.

[0099] The loop filter unit (560) may include a deblocking filter (562), an SAO filter (564), and an ALF (566) as in-loop filters. The deblocking filter (562) deblocks the boundaries between restored blocks to remove blocking artifacts caused by block-by-block decoding. The SAO filter (564) and the ALF (566) perform additional filtering on restored blocks after deblocking filtering to compensate for differences between restored pixels and original pixels caused by lossy coding. The filter coefficients of the ALF are determined using information about filter coefficients decoded from the non-stream.

[0100] The restored blocks filtered through the deblocking filter (562), SAO filter (564), and ALF (566) are stored in the memory (570). When all blocks within a picture are restored, the restored picture is used as a reference picture for inter-predicting blocks within a picture to be encoded later.

[0101] The present embodiment relates to encoding and decoding of images (video) as described above. More specifically, in relation to residual signal encoding of a transform skip block, a video coding method and device are provided that predicts the sign of a residual signal based on a surrounding restoration signal-based sign prediction, and context-encodes the prediction error of the sign.

[0102] The following embodiments may be performed by a conversion unit (140) and an inverse conversion unit (165) within a video encoding apparatus. In addition, the following embodiments may be performed by an inverse conversion unit (530) within a video decoding apparatus.

[0103] The video encoding device can generate signaling information related to the present embodiment in terms of rate distortion optimization in encoding the current block. The video encoding device can encode the signaling information using the entropy encoding unit (155) and then transmit it to the video decoding device. The video decoding device can decode the signaling information related to the decoding of the current block from the bitstream using the entropy decoding unit (510).

[0104] In the following description, the term "target block" may be used interchangeably with the current block or coding unit (CU). Alternatively, the term "target block" may also refer to a portion of a coding unit.

[0105] Also, a value of a flag being true indicates that the flag is set to 1. Also, a value of a flag being false indicates that the flag is set to 0.

[0106] I. Sign Prediction Technology

[0107] In Regular Residual Coding (RRC) of VVC, bypass coding is applied to the sign. Beyond VVC, Enhanced Compression Model (ECM) adopts a coding method based on sign prediction. For example, instead of bypass coding the sign, RRC's sign prediction improves the performance of sign coding by predicting the sign of a given region / coefficient and context-coding the prediction error of the sign. Bypass coding is applied to the remaining codes except for the sign of the given region / coefficient. Here, bypass coding represents binary arithmetic decoding based on equal probability.

[0108] The signs are predicted based on hypothetical border reconstruction. To predict n signs, hypothetical border reconstruction is 2 n For possible code combinations of branches, a virtual restoration value is generated by inverse transformation. Virtual boundary restoration generates a prediction value according to a predefined method using pixel values ​​of the upper and left boundaries of a TU (Transform unit), as in the example of Fig. 6, and calculates a cost function based on the difference between the prediction value and the virtual restoration value, as in Equation 1. Virtual boundary restoration predicts n codes using a code set that minimizes the cost function.

[0109]

[0110] According to mathematical equation 1, n=w+h is satisfied. 2p x,-1 - p x,-2 and 2p -1,y - p -2,y represents the predicted values, and p x,0 and p 0,y represents virtual restoration samples

[0111] As previously mentioned, conventional virtual boundary restoration is applied to residual blocks to which transformations have been applied. Below, we describe a method for applying sign prediction to transform skip blocks in addition to residual blocks to which transformations have been applied.

[0112] The following embodiments are described with a focus on an image decoding device, but can be implemented identically or similarly in an image encoding device.

[0113] II. Embodiments according to the present disclosure

[0114] FIG. 7 is an exemplary diagram showing restoration of a conversion skip block according to one embodiment of the present disclosure.

[0115] The video decoding device according to the present embodiment determines a prediction and transformation unit, and performs prediction and inverse transformation on the current block corresponding to the determined unit using the determined prediction technique and prediction mode, thereby finally generating a restoration block of the current block. The operation illustrated in FIG. 7 may be performed by the entropy decoding unit (510), the inverse quantization unit (520), the inverse transformation unit (530), and the adder (550) of the video decoding device. Meanwhile, the same operations as illustrated in FIG. 7 may be performed by the inverse quantization unit (160), the inverse transformation unit (165), and the adder (170) of the video encoding device. At this time, the video decoding device uses encoding information parsed from the bitstream, but the video encoding device may use encoding information set from a higher level in terms of minimizing rate distortion. Hereinafter, for convenience, the present embodiment will be described with reference to the video decoding device.

[0116] The entropy decoding unit (510) illustrated in FIG. 5 may include the transform skip determination unit (702), the residual signal decoding unit (704), and the transform coefficient decoding unit (710) illustrated in FIG. 7. The inverse quantization unit (520) illustrated in FIG. 5 may include the residual signal inverse quantization unit (706) and the transform coefficient inverse quantization unit (712) illustrated in FIG. 7. The inverse transform unit (530) illustrated in FIG. 5 may correspond to the inverse transform unit (714) illustrated in FIG. 7, and the adder (550) may correspond to the restoration performing unit (708) illustrated in FIG. 7.

[0117] The transformation skip determination unit (702) can implicitly or explicitly determine whether to skip the current transformation block based on information such as the upper level syntax, the size of the current transformation block, the prediction mode of the current block, etc.

[0118] As an example, the conversion skip determination unit (702) may receive a flag for determining whether conversion skip is permitted from a higher level (e.g., VPS, SPS, PPS, picture header, slice header, etc.) and determine whether conversion skip is permitted. If the higher level flag permits conversion skip, the conversion skip determination unit (702) may determine whether conversion skip of the current conversion block is permitted. For example, if one or more of the following conditions are satisfied, conversion skip for the current conversion block may be permitted.

[0119] - When no subblock transformation (such as Intra Sub-partitions (ISP) or Subblock Transform (SBT)) is applied to the current block.

[0120] - (tbWidth ≤ maxTSWidth) && (tbHeight ≤ maxTSHeight)

[0121] Here, tbWidth and tbHeight represent the width and height of the current transformation block, respectively, and maxTSWidth and maxTSHeight represent the maximum width and height of the block to which transformation skip is applied, respectively, and can be passed on after being parsed at a higher level.

[0122] ISP technology performs intra prediction on the current block, and additionally divides the current block into subblocks for intra prediction. ISP technology performs the division into subblocks using a flag that determines whether to divide in ISP mode and a flag that determines whether to divide horizontally or vertically. The divided subblocks are reconstructed using intra prediction, entropy restoration, inverse quantization, and inverse transformation from top to bottom in horizontal division mode and from left to right in vertical division mode.

[0123] SBT is a transform technology that divides the CU into smaller blocks and performs transform on a sub-block basis when encoding an inter-predicted block. At this time, the SBT type and SBT location information of the block can be signaled. In the case of SBT-V (or SBT-H), the width (or height) of the TU (Transform Unit) can be equal to half or 1 / 4 of the width (or height) of the CU. Depending on the SBT type, the horizontal and vertical transforms can be implicitly applied differently.

[0124] If one or more of the conditions described above are satisfied and conversion skipping for the current conversion block is allowed, the conversion skip determination unit (702) can determine whether conversion skipping of the current conversion block is allowed by parsing TS_flag, which is a 1-bit flag indicating whether conversion skipping is allowed.

[0125] For example, if the current block is encoded in BDPCM (Block Difference Pulse Code Modulation) mode, the transform skip determination unit (702) can omit parsing of TS_flag and implicitly derive TS_flag=1. BDPCM differentially encodes the residual signal in the horizontal or vertical direction when transform skip is applied.

[0126] If the TS_flag implicitly or explicitly determined in the transform skip determination unit (702) is 0, the transform coefficient decoding unit (710) parses the quantized transform coefficient. The transform coefficient inverse quantization unit (712) inversely quantizes the quantized transform coefficient. The inverse transformation unit (714) determines the transform kernel and inversely transforms the inverse quantized transform coefficient based on the determined transform kernel, thereby restoring the residual signal. The restoration performing unit (708) adds the residual signal to the prediction block transmitted from the prediction unit (540) to generate a restoration block of the current block.

[0127] The residual signal decoding unit (704) performs decoding on the quantized residual signal when TS_flag is 1.

[0128] The residual signal dequantization unit (712) dequantizes the quantized residual signal based on the quantization method, quantization parameters, etc. The restoration performing unit (708) adds the dequantized residual signal to the prediction block transmitted from the prediction unit (540) to generate a restoration block of the current block.

[0129] Below, the operation performed by the residual signal decoding unit (704) is described in detail.

[0130] As an example, the residual signal decoding unit (704) can decode the residual signal for each subblock. Here, the subblock has a size of n×m, and n and m are integers greater than 0. n and m may be defined in advance according to an agreement between the video encoding device and the video decoding device, or may be determined according to the size, aspect ratio, etc. of the current transform block. The subblock may be decoded according to a scanning order determined based on the agreement between the video encoding device and the video decoding device, the size, aspect ratio, etc. of the current transform block. As in the example of Fig. 8, a scanning order from the upper left to the lower right may be used. As another example, a diagonal scan from the lower right to the upper left may be used.

[0131] Each subblock can parse coded_subblock_flag to determine whether there is a residual signal for decoding within the subblock. If coded_subblock_flag=0, the residual signal decoding unit (704) can skip residual signal decoding for the corresponding subblock and induce the residual signal within the subblock to 0. If coded_subblock_flag=1, the residual signal decoding unit (704) can decode the residual signal for the corresponding subblock.

[0132] The residual signal decoding unit (704) can decode quantized residual signal values ​​according to a scanning order defined within the subblock when decoding the residual signal of the subblock. As an example, a diagonal scanning method from the upper left to the lower right, as in the example of FIG. 9, can be used as a scanning order for residual signal coding within the subblock. As another example, diagonal scanning from the lower right to the upper left can be used for residual signal coding.

[0133] The residual signal scanning order within a subblock may be as follows. For example, when using top and left information as surrounding context and / or restoration information, the scanning order may be determined based on the priorities illustrated in FIG. 10. That is, lower numbers in the scanning order should be restored before higher numbers. As another example, when using right and bottom information as surrounding context and / or restoration information, the scanning order may be determined based on the priorities illustrated in FIG. 11.

[0134] As an example, the residual signal decoding unit (704) can decode a quantized residual signal value based on the following syntax.

[0135] sig_level_flag is a flag indicating whether the absolute value (abs_level) of the residual signal at the current position is greater than 0. If sig_level_flag is 0, the absolute value is 0, and if sig_level_flag is 1, it indicates the residual signal at the current position.

[0136] sign_flag is a flag indicating the sign value of the residual signal at the current position. If sig_level_flag is 1, sign_flag is parsed.

[0137] abs_level_gtx_flag[0] is a flag indicating whether the absolute value of the residual signal at the current position is greater than 0. If abs_level_gtx_flag[0] is 0, the absolute value is 1, and if abs_level_gtx_flag[0] is 1, the absolute value is greater than 1. If sig_level_flag is 1, abs_level_gtx_flag[0] is parsed.

[0138] par_level_flag represents the parity bit of the absolute value of the residual signal at the current position. If abs_level_gtx_flag[0] is 1, par_level_flag is parsed.

[0139] abs_level_gtx_flag[1] is a flag indicating whether the absolute value of the residual signal at the current position is greater than 3. If abs_level_gtx_flag[1] is 0, the absolute value is 3, and if abs_level_gtx_flag[1] is 1, the absolute value is greater than 3. If abs_level_gtx_flag[0] is 1, abs_level_gtx_flag[1] is parsed.

[0140] abs_level_gtx_flag[k-1] is a flag indicating whether the absolute value of the residual signal at the current position is greater than 2k+1. If abs_level_gtx_flag[k-1] is 0, the absolute value is 2k+1, and if abs_level_gtx_flag[k-1] is 1, the absolute value is greater than 2k+1. If abs_level_gtx_flag[k-2] is 1, abs_level_gtx_flag[k-1] is parsed.

[0141] abs_remainder represents the remaining value after encoding the absolute value of the residual signal at the current position based on the aforementioned flags. If abs_level_gtx_flag is all 1, abs_remainder is parsed.

[0142] Based on the syntax parameters described above, the residual signal decoding unit (704) can decode the residual signal value, i.e., level, according to mathematical expressions 2 and 3.

[0143]

[0144]

[0145] As another example, the residual signal decoding unit (704) can decode the quantized residual signal value based on the following syntax.

[0146] sig_level_flag is a flag indicating whether the absolute value (abs_level) of the residual signal at the current position is greater than 0. If sig_level_flag is 0, the absolute value is 0, and if sig_level_flag is 1, it indicates the residual signal at the current position.

[0147] sign_flag is a flag indicating the sign value of the residual signal at the current position. If sig_level_flag is 1, sign_flag is parsed.

[0148] abs_level_gtx_flag[0] is a flag indicating whether the absolute value of the residual signal at the current position is greater than 1. If abs_level_gtx_flag[0] is 0, the absolute value is 1, and if abs_level_gtx_flag[0] is 1, the absolute value is greater than 1. If sig_level_flag is 1, abs_level_gtx_flag[0] is parsed.

[0149] abs_level_gtx_flag[1] is a flag indicating whether the absolute value of the residual signal at the current position is greater than 2. If abs_level_gtx_flag[1] is 0, the absolute value is 2, and if abs_level_gtx_flag[1] is 1, the absolute value is greater than 2. If abs_level_gtx_flag[0] is 1, abs_level_gtx_flag[1] is parsed.

[0150] abs_level_gtx_flag[k-1] is a flag indicating whether the absolute value of the residual signal at the current position is greater than k. If abs_level_gtx_flag[k-1] is 0, the absolute value is k, and if abs_level_gtx_flag[k-1] is 1, the absolute value is greater than k. If abs_level_gtx_flag[k-2] is 1, abs_level_gtx_flag[k-1] is parsed.

[0151] par_level_flag indicates the parity bit of the absolute value of the residual signal at the current position. If abs_level_gtx_flag is all 1, par_level_flag is parsed.

[0152] abs_remainder represents the remaining value after encoding the absolute value of the residual signal at the current position based on the aforementioned flags. If abs_level_gtx_flag is all 1, abs_remainder is parsed.

[0153] Based on the syntax parameters described above, the residual signal decoding unit (704) can decode the residual signal value according to mathematical expressions 4 and 5.

[0154]

[0155]

[0156] As an example, in order to effectively reduce the syntax parameters to be parsed, the video encoding device can change the residual signal values ​​as follows based on the encoded surrounding residual signal values.

[0157]

[0158] Here, abs_level is the absolute value of the level at the current position to be coded, and X0 and X1 represent the absolute values ​​of the surrounding residual signals that are encoded, respectively.

[0159] The video decoding device can restore the level value changed in the video encoding device to the original level value as follows. In the content below, "abs_level<=pred" is replaced with 0 or 1 depending on whether the condition is satisfied.

[0160]

[0161] As an example, the syntax described above may be divided into T groups according to an agreement between an image encoding device and an image decoding device, and the syntax within a subblock may be parsed according to the scanning order for each group.

[0162] FIG. 12 is an exemplary diagram showing a group of syntax parameters according to one embodiment of the present disclosure.

[0163] Each syntax parameter can be encoded / decoded by context-based general CABAC (Context-based Adaptive Binary Arithmetic Code) coding or bypass coding. The maximum number of context-codable bins (maxContextBin) can be determined based on the size of the current transform block (tbWidth×tbHeight). For example, maxContextBin is tbWidth×tbHeight×th ccb can be determined. ccb represents context coded bin, and th ccb is a parameter used to determine the maximum number of ccb for the current transformation block. th ccb is a value determined by an agreement between the video encoding device and the video decoding device, or can be determined by parsing at a higher level. As another example, maxContextBin is based on tb_zeroout_size × th ccb can be determined. Here, tb_zeroout_size represents the number of pixels generated by zeroed-out performed according to the agreement between the image encoding device and the image decoding device. tb_zeroout_size ≤ (tbWidth × tbHeight).

[0164] In Fig. 12, syntax parameters are decoded by dividing them into three groups, i.e., passes. The syntax parameters included in each pass may vary depending on the embodiment. For example, the first and second passes may be coded in a context-based general CABAC mode, and the third pass may be coded in a bypass mode. As another example, when the residual signal is decoded according to Equations 4 and 5, par_level_flag may be parsed after parsing all abs_level_gtx_flag. In Fig. 12, C i(0≤i≤n×m-1) represents the i-th residual sample.

[0165] Each syntax parameter is coded in the context-based general CABAC mode, and when a ccb occurs, the remCcb value is decreased by 1. remCcb represents the remaining ccb in the encoding process and is initialized with the maxContextBin value. In order to prevent the occurrence of ccb exceeding maxContextBin, before decoding the residual signals of the first and second passes, the remCcb value is compared with the number of syntax parameters coded in each pass (for example, in the case of the first pass in FIG. 12, the number of syntax parameters is 4). If the remCcb value is greater than the number of syntax parameters in each pass, decoding of the corresponding syntax parameter can be started. If the remCcb value is less than the number of syntax parameters of each pass (e.g., if the first pass and / or the second pass are not decoded), the current residual signal can be determined by parsing abs_remainder based on the bypass mode. For example, if the remCcb is less than the number of syntax parameters during coding of the first pass and thus the first pass is not decoded, the sign value (sign_flag) can be parsed by the bypass mode.

[0166] As an example, in coding the sign value of the residual signal, an image encoding device may perform sign prediction-based coding. For sign prediction-based coding, the image encoding device predicts the sign and context-based codes the error value of the sign. The image encoding device calculates the difference between pixel values ​​based on previously decoded surrounding pixels and virtually reconstructed pixels of the current pixels. The image encoding device predicts the sign of the current residual signal using the sign with the smallest difference between the pixel values. If the predicted sign value matches the actual sign value, the image encoding device sets sign_error_flag to 0, and if they do not match, the image encoding device sets sign_error_flag to 1. The image encoding device encodes sign_error_flag. The image decoding device also performs sign prediction according to the same method as the image encoding device. If 0 is decoded as the sign of the residual signal for which sign prediction has been performed, the image decoding device determines the predicted sign value as the sign value of the current residual signal. On the other hand, when 1 is decoded, the image decoding device determines the opposite sign of the predicted sign as the sign value of the current residual signal.

[0167] As an example, sign prediction can be performed based on the pixel values ​​of the upper and left (X0, X1) as shown on the left side of Fig. 13. By calculating the two cost functions shown in Equation 6, the sign corresponding to the smaller cost function can be predicted as the sign value of the current residual signal.

[0168]

[0169] In Equation 6, resi0(X) represents the restored residual signal when the sign of the residual signal at the current position is positive, and resi1(X) represents the restored residual signal when the sign is negative. pred() represents the predicted signal, and resi(), resi0(), and resi1() represent the residual signals respectively, so pred()+resi(), pred()+resi0(), and pred()+resi1() represent the restored signals. Therefore, cost0 represents the cost value when the sign of the current residual signal is positive, and cost1 represents the cost value when the sign of the current residual signal is negative.

[0170] As another example, sign prediction can be performed based on the pixel values ​​on the top and left (X0, X2, X1, X3) as shown on the right side of Fig. 13. By calculating the two cost functions shown in Equation 7, the sign corresponding to the smaller cost function can be predicted as the sign value of the current residual signal.

[0171]

[0172] In mathematical expression 7, cost0 represents the cost value when the sign of the current residual signal is positive, and cost1 represents the cost value when the sign of the current residual signal is negative.

[0173] FIG. 14 is an exemplary diagram showing a group of syntax parameters according to another embodiment of the present disclosure.

[0174] In Fig. 14, sign prediction is performed for all residual signals within a subblock.

[0175] The syntax parameters included in each pass may differ depending on the embodiment. If the predicted encoded actual code is the same, sign_error_flag may represent 0, and if different, sign_error_flag may represent 1. At the time of decoding the code of the current residual signal, depending on the scanning order, the image decoding device may restore the syntax used for code prediction using the previously decoded residual signal. When decoding the code of the current residual signal, if as many context-coded bins as maxContextBin are generated and there are no remaining ccbs, sign_error_flag represents the actual code value rather than the code error value (e.g., 0 is a positive code, 1 is a negative code). The actual code value can be decoded by bypass mode or context-based coding.

[0176] A sign prediction area in which sign prediction is performed within a block and / or a maximum number of coefficients capable of sign prediction can be defined at a higher level (e.g., VPS, SPS PPS, picture header, slice header, etc.). The sign prediction area and / or the maximum number of coefficients capable of sign prediction can be determined according to information such as a configuration (e.g., a coding structure such as all intra, random access, low delay, etc., or a structure of a picture group), a QP (quantization parameter) of a current picture, a QP of an I (Intra) picture of a picture group including the current picture, whether it is an intra / inter slice, a horizontal resolution of a picture, a vertical resolution of a picture, etc. The sign prediction area and / or the maximum number of coefficients capable of sign prediction can be determined according to an agreement between a video encoding device and a video decoding device. If the position of the current residual signal (i.e., coefficient) is outside the sign prediction region, or if sign prediction has been performed on the maximum number of coefficients for which sign prediction is possible at the time of decoding the sign of the current residual signal (i.e., coefficient), sign_error_flag indicates the actual sign value rather than the sign error value (e.g., 0 for a positive sign, 1 for a negative sign), and can be decoded in bypass mode or context-based coding.

[0177] As another example, when the residual signal is decoded according to Equations 4 and 5, par_level_flag can be parsed after all abs_level_gtx_flag are parsed.

[0178] As an example, whether the sign of the residual signal is predictable can be determined based on the sign value of the surrounding decoded residual signal. Here, whether the sign of the residual signal is predictable can be determined within the sign prediction region and / or within the maximum number of coefficients for which sign prediction is possible.

[0179] Depending on the sign value of the surrounding residual signal, three cases (Case 1, Case 2, Case 3) can be classified, as in the example of Fig. 15. Case 1 represents the case where none of the surrounding residual signals exist, the residual signal value is 0 and has no sign, or the number of positive and negative signs of the surrounding residual signals is the same. Case 2 represents the case where one of the surrounding residual signals does not exist and the other has a positive sign, one of the residual signals has no sign as the value is 0 and the other has a positive sign, or the signs of the surrounding residual signals are all positive. Case 3 represents the case where one of the surrounding residual signals does not exist and the other has a negative sign, one of the residual signals has no sign as the value is 0 and the other has a negative sign, or the signs of the surrounding residual signals are all negative.

[0180] For example, sign prediction can be performed for Case 1. In Case 1, after sign prediction, the sign error (sign_error_flag) can be decoded / decoded based on context. In Cases 2 and 3, sign prediction is not performed, and sign_flag can be CABAC coded using separate context information. When sign prediction is performed for Case 1, syntax parameters can be decoded according to FIG. 16. As another example, when the residual signal is decoded according to Equations 4 and 5, par_level_flag can be parsed after parsing all abs_level_gtx_flag.

[0181] As another example, if no sign prediction is performed for Case 1, sign_flag may be decoded in bypass mode rather than using context-based general CABAC coding. For Cases 2 and 3, where no sign prediction is performed, sign_flag may be CABAC coded using separate context information.

[0182] Syntax parameters can be decoded according to the scanning order illustrated in FIG. 12 or FIG. 17.

[0183] As an example, when syntax parameters are decoded according to FIG. 17, the code decoding for Case 1 may be omitted during the first pass decoding (e.g., C0 in FIG. 17), and the code decoding for Case 1 may be performed according to the bypass mode during the third pass decoding.

[0184] As another example, the remCcb value may be checked before starting the first pass, and if the remCcb value is greater than or equal to 4, the first pass decoding may be performed. On the other hand, if the remCcb value is less than 4, the decoding of abs_remainder may be performed in bypass mode in the third pass. If the current residual signal is Case 1 and the sign is coded according to the bypass mode (since it is not context-based coding), the remCcb value may not decrease.

[0185] As another example, if the sign value of the surrounding residual signal (top and left) satisfies Case 1 before starting the first pass decoding and the remCcb value is greater than or equal to 3 (e.g., C0 in FIG. 17), syntax decoding for the current residual signal can be performed according to the first pass. If the sign value of the surrounding residual signal (top and left) satisfies Case 2, 3 and the remCcb value is greater than or equal to 4 (e.g., C2 in FIG. 17), syntax decoding for the current residual signal can be performed according to the first pass.

[0186] Hereinafter, using the illustrations of FIGS. 18 and 19, a method for encoding / decoding a residual signal based on sign prediction of the residual signal is described.

[0187] FIG. 18 is a flowchart illustrating a method for an image encoding device to encode a current block according to an embodiment of the present disclosure.

[0188] The video encoding device generates a prediction signal of the current block and subtracts the prediction signal from the original signal of the current block to generate a residual signal of the current block.

[0189] If transformation skip is allowed for the current block at a higher level, the video encoding device can perform the following steps.

[0190] The video encoding device obtains a conversion skip flag for the current block (S1800).

[0191] The video encoding device determines whether to perform conversion skip for the current block based on the conversion skip flag (S1802).

[0192] If it is determined not to perform transform skip, the image encoding device transforms / quantizes the residual signal and encodes the quantized residual signal. On the other hand, if it is determined to perform transform skip, the image encoding device performs the following steps.

[0193] The image encoding device generates parameters of the residual signal of the current block based on the absolute value and sign information of the residual sample (S1804).

[0194] The image encoding device quantizes the parameters of the residual signal to generate a quantized residual signal (S1806).

[0195] A video encoding device encodes residual signals of sub-blocks constituting a current block, and can encode parameters of the residual signals of each sub-block according to a scanning order based on a predetermined agreement between the video encoding device and the video decoding device, the size of the current block, or the aspect ratio of the current block.

[0196] The video encoding device can encode parameters of a residual signal within each subblock according to a predefined scanning order.

[0197] The image encoding device can obtain a sign prediction area of ​​a residual sample from a higher level with respect to the current block.

[0198] An image encoding device encodes parameters of a residual signal based on a plurality of passes including parameters of the residual signal. When the sign information is a sign error flag, the image encoding device can encode a pass including the sign error flag according to context-based coding within a sign prediction region. When the sign information is a sign, the image encoding device can encode the sign of a residual sample according to bypass coding or context-based coding outside the sign prediction region.

[0199] As an example, an image encoding device may determine whether to perform sign prediction of a residual sample based on the signs of surrounding residual samples. Here, whether to perform sign prediction of a residual signal may be determined within a sign prediction region and / or within a maximum number of coefficients for which sign prediction is possible. For example, for Case 1 described above, the image encoding device may determine to perform sign prediction of the residual sample. For Cases 2 and 3, the image encoding device may determine not to perform sign prediction of the residual sample.

[0200] If it is decided to perform sign prediction of residual samples for Case 1, the video encoding device can decode the sign error flag as sign information according to context-based coding. As another example, if it is decided not to perform sign prediction of residual samples for Cases 1, 2, and 3, the video encoding device can encode the sign of the residual sample as sign information according to bypass coding or context-based coding.

[0201] The video encoding device can change the absolute value of the residual sample based on the absolute values ​​of the encoded surrounding samples of the residual sample.

[0202] As an example, if the sign information is a sign error flag, the image encoding device generates a sign error flag of the residual sample as follows.

[0203] The image encoding device predicts the sign of the residual sample based on the decoded surrounding samples of the residual sample and the virtual prediction sample (S1820).

[0204] The video encoding device generates a first cost function, for example, as in Equation 6 or Equation 7, based on at least one decoded top sample of a residual sample, at least one decoded left sample, and a virtual reconstruction sample based on a positive sign. The video encoding device generates a second cost function, for example, as in Equation 6 or Equation 7, based on at least one decoded top sample, at least one decoded left sample, and a virtual reconstruction sample based on a negative sign.

[0205] The video encoding device may determine the sign of the residual sample based on the smaller cost function among the first cost function and the second cost function. For example, if the first cost function is smaller, the sign of the residual sample may be set to positive. Conversely, if the second cost function is smaller, the sign of the residual sample may be set to negative.

[0206] The image encoding device determines a sign error flag of the residual sample based on the predicted sign and the original sign of the residual sample (S1822).

[0207] The video encoding device sets the code error flag to false if the original code and the predicted code are the same. The video encoding device sets the code error flag to true if the original code and the predicted code are different.

[0208] FIG. 19 is a flowchart illustrating a method for an image decoding device to restore a current block according to one embodiment of the present disclosure.

[0209] The video decoding device generates a prediction signal for the current block.

[0210] If transformation skip is allowed for the current block at the upper level, the image decoding device can perform the following steps.

[0211] The video decoding device decodes a conversion skip flag from a bitstream (S1900).

[0212] The video decoding device determines whether to perform conversion skip for the current block based on the conversion skip flag (S1902).

[0213] If it is determined not to perform transform skip, the image decoding device decodes / dequantizes / inversely transforms the quantized residual signal to restore the residual signal. The restored residual signal and the prediction signal are added to generate a restored block of the current block. On the other hand, if it is determined to perform transform skip, the image decoding device performs the following steps.

[0214] The video decoding device decodes the absolute value and sign information of the residual sample from the bitstream, and obtains a quantized residual signal of the current block based on the absolute value and sign information of the residual sample (S1904).

[0215] An image decoding device obtains residual signals of sub-blocks constituting a current block, and can obtain parameters of the residual signals of each sub-block according to a scanning order based on a preset agreement between an image encoding device and an image decoding device, the size of the current block, or the aspect ratio of the current block.

[0216] The video decoding device can obtain parameters of a residual signal within each subblock according to a predefined scanning order.

[0217] The image decoding device can obtain a sign prediction area of ​​a residual sample from a higher level with respect to the current block.

[0218] An image decoding device obtains parameters of a residual signal based on a plurality of passes including parameters of the residual signal. If the sign information is a sign error flag, the image decoding device can decode a pass including the sign error flag according to context-based coding within a sign prediction region. If the sign information is a sign, the image decoding device can decode the sign of the residual sample according to bypass coding or context-based coding outside the sign prediction region.

[0219] As an example, an image decoding device may determine whether to perform sign prediction of a residual sample based on the signs of surrounding residual samples. Here, whether to perform sign prediction of a residual signal may be determined within a sign prediction region and / or within a maximum number of coefficients for which sign prediction is possible. For example, for Case 1 described above, the image decoding device may determine to perform sign prediction of the residual sample. For Cases 2 and 3, the image decoding device may determine not to perform sign prediction of the residual sample.

[0220] If it is decided to perform sign prediction of residual samples for Case 1, the video decoding device can decode the sign error flag as sign information according to context-based coding. As another example, if it is decided not to perform sign prediction of residual samples for Cases 1, 2, and 3, the video decoding device can decode the sign of the residual sample as sign information according to bypass coding or context-based coding.

[0221] The video decoding device can change the absolute value of the residual sample based on the absolute values ​​of the decoded surrounding samples of the residual sample.

[0222] As an example, if the sign information is a sign error flag, the image decoding device determines the sign of the residual sample as follows and generates a quantized residual signal.

[0223] The image decoding device predicts the sign of the residual sample based on the decoded surrounding samples of the residual sample and the virtual prediction sample (S1920).

[0224] The video decoding device generates a first cost function, for example, as in Equation 6 or Equation 7, based on at least one decoded top sample of a residual sample, at least one decoded left sample, and a virtual reconstruction sample based on a positive sign. The video decoding device generates a second cost function, for example, as in Equation 6 or Equation 7, based on at least one decoded top sample, at least one decoded left sample, and a virtual reconstruction sample based on a negative sign.

[0225] The image decoding device may determine the sign of the residual sample based on the smaller cost function among the first cost function and the second cost function. For example, if the first cost function is smaller, the sign of the residual sample may be set to positive. Conversely, if the second cost function is smaller, the sign of the residual sample may be set to negative.

[0226] The image decoding device determines the sign of the residual sample based on the predicted sign and the sign error flag (S1922).

[0227] If the sign error flag is false, the image decoding device determines the sign of the residual sample as the predicted sign. If the sign error flag is true, the image decoding device determines the sign of the residual sample as the opposite sign of the predicted sign.

[0228] The image decoding device generates a quantized residual signal by combining the absolute values ​​of residual samples and the determined signs (S1924).

[0229] The image decoding device dequantizes the quantized residual signal (S1906). Thereafter, the image decoding device can add the dequantized residual signal and the prediction signal to generate a restored block of the current block.

[0230] Although the flowchart / timing diagram of this specification describes each process as being executed sequentially, this is merely an illustrative description of the technical idea of ​​one embodiment of the present disclosure. In other words, a person of ordinary skill in the art to which one embodiment of the present disclosure belongs may modify and apply various modifications and variations by changing the order described in the flowchart / timing diagram without departing from the essential characteristics of one embodiment of the present disclosure, or by executing one or more of the processes in parallel. Therefore, the flowchart / timing diagram is not limited to a chronological order.

[0231] It should be understood that the exemplary embodiments described above can be implemented in many different ways. The functions or methods described in one or more examples can be implemented in hardware, software, firmware, or any combination thereof. It should be understood that the functional components described herein are labeled as "units" to further emphasize their implementation independence.

[0232] Meanwhile, the various functions or methods described in this embodiment may be implemented as instructions stored on a non-transitory storage medium that can be read and executed by one or more processors. Non-transitory storage media include, for example, all types of storage devices that store data in a form readable by a computer system. For example, non-transitory storage media include storage media such as erasable programmable read-only memory (EPROM), flash drives, optical drives, magnetic hard drives, and solid-state drives (SSDs).

[0233] The above description is merely an example of the technical idea of ​​the present embodiment, and those skilled in the art will appreciate that various modifications and variations can be made without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of ​​the present embodiment, but rather to explain it, and the scope of the technical idea of ​​the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment should be interpreted by the claims below, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of rights of the present embodiment.

[0234]

[0235]

[0236] CROSS-REFERENCE TO RELATED APPLICATION

[0237] This patent application claims priority to Korean patent application No. 10-2023-0173970, filed in Korea on December 5, 2023, and Korean patent application No. 10-2024-0152351, filed in Korea on October 31, 2024, the entire contents of which are incorporated herein by reference.

Claims

1. A method for restoring a current block performed by a video decoding device, A step of decoding a conversion skip flag from a bitstream; and A step for determining whether to perform a conversion skip for the current block based on the conversion skip flag. Including, In the case where the above conversion skip is decided to be performed, A step of decoding the absolute value and sign information of the residual sample from the bitstream and obtaining the residual signal of the current block based on the absolute value of the residual sample and the sign information. Including more, In the case where the above sign information is determined as a sign error flag, A step of predicting a sign of the residual sample based on the decoded surrounding samples of the residual sample and the virtual prediction sample; A step of determining the sign of the residual sample based on the predicted sign and the sign error flag; and A step of generating the quantized residual signal by combining the absolute values ​​of the residual samples and the determined signs. A method further comprising:

2. In paragraph 1, A step of generating a prediction signal of the current block; A step of dequantizing the above quantized residual signal; and A step of generating a restoration block of the current block by adding the dequantized residual signal and the above prediction signal. A method further comprising:

3. In paragraph 1, The step of obtaining the above residual signal is: A method for obtaining residual signals of sub-blocks constituting the current block, wherein parameters of the residual signals of each sub-block are obtained according to a scanning order based on a preset promise, the size of the current block, or the aspect ratio of the current block.

4. In paragraph 3, The step of obtaining the above residual signal is: A method for obtaining parameters of the residual signal within each subblock according to a defined scanning order.

5. In paragraph 1, A method further comprising the step of obtaining a sign prediction region of the residual sample from a higher level in relation to the current block.

6. In paragraph 5, The step of obtaining the above residual signal is: A method for generating parameters of a residual signal based on a plurality of passes including parameters of the residual signal, wherein a pass including a sign error flag is decoded according to context-based coding within the sign prediction region.

7. In paragraph 1, The step of predicting the sign of the above residual sample is: A step of generating a first cost function based on a decoded top sample of the residual sample, a decoded left sample, and a virtual restored sample based on a positive sign; generating a second cost function based on the decoded top sample, the decoded left sample, and the virtual restored sample based on the negative sign; and A step of determining the sign of the residual sample based on the first cost function and the second cost function. A method comprising:

8. In paragraph 1, The step of determining the sign of the above residual sample is: A method wherein, if the sign error flag is false, the sign of the residual sample is determined as the predicted sign, and if the sign error flag is true, the sign of the residual sample is determined as the opposite sign of the predicted sign.

9. In paragraph 1, A method further comprising the step of changing the absolute value of the residual sample based on the absolute values ​​of surrounding samples of the residual sample.

10. In paragraph 1, Further comprising a step of determining whether to predict the sign of the residual sample based on the signs of the surrounding residual samples, The step of decoding the sign information of the above residual sample is: A method wherein, in a case where it is decided to perform sign prediction of the residual sample, the sign error flag is decoded as the sign information according to context-based coding, and in a case where it is decided not to perform sign prediction of the residual sample, the sign of the residual sample as the sign information is decoded according to context-based coding or bypass mode.

11. In paragraph 10, The step of determining whether to predict the sign of the residual sample above is: A method for determining to perform sign prediction of the residual sample when the above-mentioned surrounding residual samples do not exist, when the signs of the above-mentioned surrounding residual samples do not exist, or when the number of positive signs and negative signs of the above-mentioned surrounding residual samples is equal.

12. A method for encoding a current block performed by a video encoding device, A step of obtaining a conversion skip flag for the current block; and A step for determining whether to perform a conversion skip for the current block based on the conversion skip flag. Including, In the case where the above conversion skip is decided to be performed, A step of generating parameters of the residual signal of the current block based on the absolute value and sign information of the residual sample. Including more, If the above sign information is a sign error flag, A step of predicting the sign of the residual sample based on the decoded surrounding samples of the residual sample and the virtual prediction sample; and A step of determining a sign error flag of the residual sample based on the predicted sign and the original sign of the residual sample. A method further comprising:

13. In paragraph 11, A step of generating a prediction signal of the current block; A step of generating the residual signal by subtracting the predicted signal from the original signal of the current block; and A step of generating a quantized residual signal by quantizing the parameters of the residual signal; and A step of encoding the parameters of the above quantized residual signal. A method further comprising:

14. In paragraph 11, The step of predicting the sign of the above residual sample is: A step of generating a first cost function based on a decoded top sample of the residual sample, a decoded left sample, and a virtual restored sample based on a positive sign; generating a second cost function based on the decoded top sample, the decoded left sample, and the virtual restored sample based on the negative sign; and A step of determining the sign of the residual sample based on the first cost function and the second cost function. A method comprising:

15. In paragraph 11, The step of determining the above sign error flag is: A method wherein, if the original code and the predicted code are the same, the code error flag is set to false, and if the original code and the predicted code are different, the code error flag is set to true.

16. A method for providing video data to a video decoding device, A step of encoding the above video data into a bitstream; and A step of transmitting the above bitstream to the image decoding device Including, The step of encoding the above video data is: Step of obtaining a conversion skip flag for the current block; and A step for determining whether to perform a conversion skip for the current block based on the conversion skip flag. Including, In the case where the above conversion skip is decided to be performed, A step of generating parameters of the residual signal of the current block based on the absolute value and sign information of the residual sample. Including more, If the above sign information is a sign error flag, A step of predicting the sign of the residual sample based on the decoded surrounding samples of the residual sample and the virtual prediction sample; and A step of determining a sign error flag of the residual sample based on the predicted sign and the original sign of the residual sample. A method comprising:

Images