Entropy decoding method, encoding method, and method for transmitting bit stream

Abstract

The present invention relates to an entropy decoding method, an encoding method, and a method for transmitting a bitstream. The present embodiments provide a method for efficiently operating a binary data buffer to limit the binary data to bit ratio in entropy encoding and decoding related to generation and parsing of a bitstream. Further, a method for constructing a list including various entropy encoding / decoding methods and adaptively utilizing the entropy encoding / decoding methods for each basic entropy encoding / decoding unit is provided.

Description

[0001] This application is a divisional application of the PCT patent application entered into China with Chinese patent application number 202080055455.7, invention name “Entropy decoding, encoding method, method for sending bit stream”, and application date August 6, 2020.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims priority to Korean Patent Application No. 10-2019-0095448, filed on August 6, 2019, and Korean Patent Application No. 10-2019-0097803, filed on August 9, 2019, which are hereby incorporated by reference in their entirety. Technical Field

[0004] The present invention relates to encoding and decoding of a video, and more particularly, to a method of efficiently operating a binary data buffer (bin buffer) and adaptively utilizing various encoding / decoding methods so as to efficiently perform entropy encoding and decoding. Background Art

[0005] The statements in this section merely provide background information related to the present invention and may not constitute prior art.

[0006] Since the amount of video data is larger than that of voice data or still image data, storing or transmitting the video data without compression processing requires a large amount of hardware resources (including memory).

[0007] Accordingly, when storing or transmitting video data, an encoder is usually used to compress the video data for storage or transmission. Then, a decoder receives the compressed video data, decompresses and reproduces the video data. Compression technologies used for such videos include H.264 / AVC and High Efficiency Video Coding (HEVC), which has an approximately 40% improvement in coding efficiency over H.264 / AVC.

[0008] However, video size, resolution, and frame rate are gradually increasing, and accordingly, the amount of data to be encoded is also increasing. Therefore, a new compression technology with better encoding efficiency and higher image quality than existing compression technologies is needed.

[0009] In video encoding, entropy coding is used to form a bitstream in which quantized transform coefficients, information about quantization parameters, intra-frame prediction or inter-frame prediction information according to the prediction type, information about block partitioning, etc. are compressed. In addition, in video decoding, entropy decoding is used to parse the above information from the bitstream.

[0010] It is desirable to provide an efficient entropy encoding / decoding method. Summary of the invention

[0011] Technical issues

[0012] The present invention provides a method for efficiently operating a binary data buffer by limiting the bin-to-bit ratio in entropy encoding and decoding associated with the generation and parsing of a bit stream. In addition, the present invention provides such a method, which configuration includes a list of various entropy encoding / decoding methods and adaptively utilizes the entropy encoding / decoding method for each basic unit of entropy encoding / decoding.

[0013] Technical Solution

[0014] According to one aspect of the present invention, a method for entropy decoding is performed by a video decoding device, the method comprising: receiving a bit stream formed by encoding an image; performing arithmetic decoding processing to generate at least one binary string (bin string) by decoding the bit stream, each binary string including at least one binary data (bin); and generating syntax elements by inverse binarization of the binary string, wherein the number of binary data generated by decoding the bit stream satisfies a constraint that the number does not exceed a threshold, wherein the threshold is variably set according to the hierarchy or level of the image.

[0015] According to another aspect of the present invention, there is provided a method for entropy encoding syntax elements generated according to predictive encoding of each block constituting an image, the method comprising: binarizing each syntax element and generating at least one binary string, each binary string including at least one binary data; performing arithmetic coding processing to generate encoded data from the binary string; and generating a bit stream configured with one or more network abstraction layer (NAL) units from the encoded data, wherein the amount of binary data is limited to not exceed a threshold value relative to the length of the one or more NAL units, wherein the threshold value is variably set according to the hierarchy or level of the image.

[0016] According to another aspect of the present invention, a device for entropy decoding is provided, the device comprising: an arithmetic decoder and an inverse binarizer, the arithmetic decoder being configured to receive a bit stream formed by encoding an image, and to perform arithmetic decoding processing to generate at least one binary string by decoding the bit stream, each binary string comprising at least one binary data; the inverse binarizer being configured to generate a syntax element by inverse binarizing the binary string, wherein the number of binary data generated by decoding the bit stream satisfies a constraint that the number does not exceed a threshold, wherein the threshold is variably set according to the hierarchy or level of the image.

[0017] Beneficial Effects

[0018] As described above, according to the present invention, in entropy encoding and decoding related to generation and parsing of a bit stream, a method of efficiently operating a binary data buffer can be provided, thereby enabling the ratio of binary data to bits to be limited.

[0019] In addition, according to the present invention, in entropy coding and decoding related to the generation and parsing of a bit stream, a method is provided, wherein a configuration includes a list of various entropy coding / decoding methods and the entropy coding / decoding method is adaptively utilized for each basic unit of entropy coding / decoding. Accordingly, entropy coding / decoding can be performed according to application and signal characteristics. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is an exemplary block diagram of a video encoding device capable of implementing the technology of the present invention.

[0021] Figure 2 A block partition structure using a QTBTTT structure is exemplarily shown.

[0022] Figure 3a and Figure 3b A plurality of intra prediction modes including a wide-angle intra prediction mode are exemplarily shown.

[0023] Figure 4 Neighboring blocks around the current block are exemplarily shown.

[0024] Figure 5 is an exemplary block diagram of a video decoding device capable of implementing the techniques of the present invention.

[0025] Figure 6 is a flow chart of a method for entropy decoding according to an embodiment of the present invention.

[0026] Figure 7 is a flow chart of an adaptive binary arithmetic decoding process according to an embodiment of the present invention.

[0027] Figure 8is an exemplary diagram of a basic unit of entropy coding including a coding block.

[0028] Fig. 9 is a flow chart of a method for entropy decoding according to another embodiment of the present invention.

[0029] Fig.10 is a flowchart of a binary arithmetic decoding process according to another embodiment of the present invention.

[0030] Reference numerals

[0031] 155: Entropy Encoder

[0032] 510: Entropy decoder. DETAILED DESCRIPTION

[0033] Hereinafter, some embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be noted that when reference numerals are added to the constituent elements in the various drawings, the same reference numerals also represent the same elements, although the elements are shown in different drawings. In addition, in the following description of the embodiments of the present invention, the detailed description of the known functions and configurations incorporated herein will be omitted to avoid blurring the subject matter of the embodiments.

[0034] Figure 1 is an exemplary block diagram of a video encoding device capable of implementing the technology of the present invention. Figure 1 A video encoding apparatus and elements of the apparatus are described.

[0035] The video encoding device includes: an image divider 110, a predictor 120, a subtractor 130, a transformer 140, a quantizer 145, a rearrangement unit 150, an entropy encoder 155, an inverse quantizer 160, an inverse transformer 165, an adder 170, a loop filtering unit 180 and a memory 190.

[0036] Each element of the video encoding device may be implemented in hardware or software, or a combination of hardware and software. The functions of each element may be implemented as software, and a microprocessor may be implemented to implement the software functions corresponding to each element.

[0037] A video consists of one or more sequences including multiple images. Each image is divided into multiple regions, and encoding is performed on each region. For example, an image is divided into one or more tiles (tile) or / and slices (slice). One or more tiles can be defined as tile groups. Each tile or slice is divided into one or more coding tree units (coding tree unit, CTU). Each CTU is divided into one or more coding units (coding unit, CU) through a tree structure. The information applied to each CU is encoded as the syntax of the CU, and the information commonly applied to the CU included in a CTU is encoded as the syntax of the CTU. In addition, the information commonly applied to all blocks in a slice is encoded as the syntax of the slice header, and the information applied to all blocks constituting one or more images is encoded in the picture parameter set (Picture Parameter Set, PPS) or the picture header. In addition, the information commonly referenced by one or more SPSs is encoded in the video parameter set (Video Parameter Set, VPS). In addition, the information commonly referenced by multiple images is encoded in the sequence parameter set (Sequence Parameter Set, SPS). In addition, information commonly applied to one tile or tile group may be encoded as a syntax of a tile header or a tile group header.The syntax included in the SPS, PPS, slice header, and tile header or tile group header may be referred to as a high-level syntax.

[0038] The image divider 110 determines the size of a coding tree unit (CTU). Information on the size of the CTU (CTU size) is encoded as a syntax of an SPS or a PPS and transmitted to a video decoding apparatus.

[0039] The image divider 110 divides each image constituting a video into a plurality of CTUs having a predetermined size, and then recursively divides the CTUs using a tree structure. In the tree structure, a leaf node serves as a coding unit (CU), which is a basic unit of coding.

[0040] The tree structure may be a quadtree (QT), a binary tree (BT), a ternary tree (TT), or a structure formed by a combination of two or more QT structures, BT structures, and TT structures, wherein the quadtree (QT) is a node (or parent node) that is divided into four slave nodes (or child nodes) of the same size, the binary tree (BT) is a node that is divided into two slave nodes, and the ternary tree (TT) is a node that is divided into three slave nodes at a ratio of 1:2:1. For example, a quadtree plus a binary tree (QTBT) structure may be used, or a quadtree plus a binary tree ternary tree (QTBTTT) structure may be used. BTTT may be collectively referred to as a multiple-type tree (MTT).

[0041] Figure 2 is a schematic diagram showing a method for partitioning a block using a QTBTTT structure.

[0042] like Figure 2 As shown, the CTU can be first split into a QT structure. The QT splitting can be repeated until the size of the split block reaches the minimum block size MinQTSize of the leaf node allowed in the QT. The first flag (QT_split_flag) indicating whether each node of the QT structure is split into four nodes of the lower layer is encoded by the entropy encoder 155 and notified to the video decoding device with a signal. When the leaf node of the QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in the BT, it can be further split into one or more BT structures or TT structures. The BT structure and / or the TT structure can have multiple splitting directions. For example, there can be two directions, namely, a direction of splitting the blocks of nodes horizontally and a direction of splitting the blocks vertically. As Figure 2 As shown, when MTT splitting starts, the entropy encoder 155 encodes a second flag (mtt_split_flag) indicating whether the node is split, a flag indicating the splitting direction (vertical or horizontal) in the case of splitting, and / or a flag indicating the splitting type (binary or trifurcated), and notifies the video decoding device with a signal.

[0043] Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node is split into 4 nodes of the lower layer, the CU split flag (split_cu_flag) indicating whether the node is split may be encoded. When the value of the CU split flag (split_cu_flag) indicates that the split is not performed, the block of the node becomes a leaf node in the split tree structure and is used as a coding unit (CU), which is a basic unit of encoding. When the value of the CU split flag (split_cu_flag) indicates that the split is performed, the video encoding device starts encoding the flags from the first flag in the above-described manner.

[0044] When QTBT is used as another example of a tree structure, there may be two types of segmentation, namely, a type of horizontally segmenting a block into two blocks of the same size (i.e., symmetrical horizontal segmentation) and a type of vertically segmenting a block into two blocks of the same size (i.e., symmetrical vertical segmentation). A segmentation flag (split_flag) indicating whether each node of the BT structure is segmented into blocks of the lower layer and segmentation type information indicating the segmentation type are encoded by the entropy encoder 155 and transmitted to the video decoding device. There may be an additional type of segmenting a block of a node into two asymmetric blocks. An asymmetric segmentation type may include a type of segmenting a block into two rectangular blocks at a size ratio of 1:3, or a type of segmenting a block of a node diagonally.

[0045] A CU may have various sizes according to the QTBT or QTBTTT partitioning of a CTU. Hereinafter, a block corresponding to a CU to be encoded or decoded (i.e., a leaf node of a QTBTTT) is referred to as a "current block". When QTBTTT partitioning is adopted, the shape of the current block may be a square or a rectangle.

[0046] The predictor 120 predicts the current block to generate a predicted block. The predictor 120 includes an intra predictor 122 and an inter predictor 124.

[0047] Typically, each current block in an image can be predictively encoded separately. Typically, the prediction of the current block can be performed using an intra-frame prediction technique (which uses data from an image including the current block) or an inter-frame prediction technique (which uses data from an image encoded before the image including the current block). Inter-frame prediction includes unidirectional prediction and bidirectional prediction.

[0048] The intra-frame predictor 122 predicts pixels in the current block using pixels (reference pixels) located around the current block in the current image including the current block. There are multiple intra-frame prediction modes according to the prediction direction. For example, Figure 3aAs shown, the plurality of intra prediction modes may include 2 non-directional modes and 65 directional modes, wherein the 2 non-directional modes include a planar mode and a DC mode. Neighboring pixels and equations to be used are defined differently for each prediction mode.

[0049] In order to perform effective directional prediction for the current block of rectangular shape, the Figure 3b The dotted arrows in the figure indicate the directional modes (intra-prediction modes 67 to 80 and -1 to -14). These modes may be referred to as "wide angle intra-prediction modes". Figure 3b , the arrows indicate the corresponding reference samples used for prediction, rather than indicating the prediction direction. The prediction direction is opposite to the direction indicated by the arrow. The wide-angle intra prediction mode is a mode in which prediction is performed in a direction opposite to a specific direction mode without additional bit transmission when the current block has a rectangular shape. In this case, in the wide-angle intra prediction mode, some wide-angle intra prediction modes that can be used for the current block can be determined based on the ratio of the width to the height of the rectangular current block. For example, when the height of the rectangular shape of the current block is less than the width, a wide-angle intra prediction mode (intra prediction modes 67 to 80) with an angle less than 45 degrees can be utilized. When the width of the rectangular shape of the current block is greater than the height, a wide-angle intra prediction mode (intra prediction modes -1 to -14) with an angle greater than -135 degrees can be utilized.

[0050] The intra-frame predictor 122 may determine an intra-frame prediction mode to be used when encoding the current block. In some examples, the intra-frame predictor 122 may encode the current block using several intra-frame prediction modes and select an appropriate intra-frame prediction mode to be used from the tested modes. For example, the intra-frame predictor 122 may calculate a rate-distortion value using a rate-distortion analysis of several tested intra-frame prediction modes and may select an intra-frame prediction mode having the best rate-distortion characteristic from the tested modes.

[0051] The intra predictor 122 selects an intra prediction mode from a plurality of intra prediction modes and predicts the current block using adjacent pixels (reference pixels) and equations determined according to the selected intra prediction mode. Information about the selected intra prediction mode is encoded by the entropy encoder 155 and transmitted to the video decoding device.

[0052] The inter-frame predictor 124 generates a prediction block for the current block by motion compensation. The inter-frame predictor 124 searches for a block that is most similar to the current block in a reference image that has been encoded and decoded earlier than the current image, and uses the searched block to generate a prediction block for the current block. Then, the inter-frame predictor generates a motion vector (MV) corresponding to the displacement between the current block in the current image and the prediction block in the reference image. Typically, motion estimation is performed on the luminance (luma) component, and the motion vector calculated based on the luminance component is used for both the luminance component and the chrominance component. The motion information including information about the reference image and information about the motion vector used to predict the current block is encoded by the entropy encoder 155 and transmitted to the video decoding device.

[0053] The inter-frame predictor 124 may perform interpolation on a reference image or a reference block to increase prediction accuracy. That is, a subsample is interpolated between two consecutive integer samples by applying a filter coefficient to a plurality of consecutive integer samples including two integer samples. When the operation of searching for the block most similar to the current block is performed on the interpolated reference image, the motion vector may be represented at an accuracy level of fractional sample units, rather than at an accuracy level of integer sample units. For each target region to be encoded, for example, each unit such as a slice, tile, CTU or CU, the accuracy or resolution of the motion vector may be set differently. When such an adaptive motion vector resolution (AMVR) is applied, information about the motion vector resolution to be applied to each target region should be signaled for each target region. For example, when the target region is a CU, information about the motion vector resolution applied to each CU is signaled. The information about the motion vector resolution may be information indicating the accuracy of the motion vector difference, which will be described later.

[0054] The inter-frame predictor 124 may perform inter-frame prediction using bidirectional prediction. In bidirectional prediction, the inter-frame predictor 124 uses two reference images and two motion vectors representing the block positions most similar to the current block in each reference image. The inter-frame predictor 124 selects a first reference image and a second reference image from a reference image list 0 (RefPicList0) and a reference image list 1 (RefPicList1), respectively, searches for blocks similar to the current block in each reference image, and generates a first reference block and a second reference block. Then, the inter-frame predictor 124 generates a prediction block for the current block by averaging or weighting the first reference block and the second reference block. Then, the inter-frame predictor 124 transmits motion information including information about the two reference images and two motion vectors used to predict the current block to the encoder 150. RefPicList0 may consist of images that are before the current image in the display order in the reconstructed image, and RefPicList1 may consist of images that are after the current image in the display order in the reconstructed image. However, the embodiment is not limited thereto. A pre-reconstructed image subsequent to the current image in the display order may be further included in RefPicList0, and conversely, a pre-reconstructed image prior to the current image may be further included in RefPicList1.

[0055] Various methods can be utilized to minimize the number of bits required to encode motion information.

[0056] For example, when the reference image and motion vector of the current block are the same as those of the adjacent block, the motion information about the current block can be transmitted to the video decoding device through the encoding information for identifying the adjacent block. This method is called "merge mode".

[0057] In the merge mode, the inter predictor 124 selects a predetermined number of merge candidate blocks (hereinafter referred to as “merge candidates”) from neighboring blocks of the current block.

[0058] like Figure 4 As shown, all or part of the left block L, the upper block A, the upper right block AR, the lower left block BL, and the upper left block AL adjacent to the current block in the current image can be used as adjacent blocks for deriving merge candidates. In addition, in addition to the current image where the current block is located, blocks located in a reference image (which may be the same as or different from the reference image used to predict the current block) can be used as merge candidates. For example, a co-located block at the same position as the current block in the reference image or a block adjacent to the co-located block can be additionally used as a merge candidate.

[0059] The inter-frame predictor 124 configures a merge list including a predetermined number of merge candidates using such adjacent blocks. The inter-frame predictor 124 selects a merge candidate to be used as motion information about the current block from among the merge candidates included in the merge list, and generates merge index information for identifying the selected candidate. The generated merge index information is encoded by the encoder 155 and transmitted to the video decoding device.

[0060] Another method of encoding motion information is the advanced motion vector prediction (AMVP) mode.

[0061] In the AMVP mode, the inter-frame predictor 124 utilizes neighboring blocks of the current block to derive motion vector candidates for prediction of the motion vector of the current block. Figure 2 In the current image in the image, all or part of the left block L, the upper block A, the upper right block AR, the lower left block BL and the upper left block AL adjacent to the current block can be used as the adjacent blocks for deriving the predicted motion vector candidate. In addition, in addition to the current image including the current block, blocks located in a reference image (which may be the same as or different from the reference image used to predict the current block) can be used as the adjacent blocks for deriving the predicted motion vector candidate. For example, a co-located block at the same position as the current block in the reference image or a block adjacent to the co-located block can be used.

[0062] The inter-frame predictor 124 derives predicted motion vector candidates using motion vectors of neighboring blocks, and determines a predicted motion vector for the motion vector of the current block using the predicted motion vector candidates. Then, a motion vector difference is calculated by subtracting the predicted motion vector from the motion vector of the current block.

[0063] The predicted motion vector can be obtained by applying a predefined function (e.g., a function for calculating a median, an average, etc.) to the predicted motion vector candidate. In this case, the video decoding device also knows the predefined function. Since the neighboring blocks used to derive the predicted motion vector candidate have been encoded and decoded, the video decoding device also already knows the motion vectors of the neighboring blocks. Accordingly, the video encoding device does not need to encode information for identifying the predicted motion vector candidate. Therefore, in this case, information about the motion vector difference and information about the reference image used to predict the current block are encoded.

[0064] The predicted motion vector may be determined by selecting any one of the predicted motion vector candidates. In this case, information for identifying the selected predicted motion vector candidate is further encoded together with information about a motion vector difference to be used for predicting the current block and information about a reference image.

[0065] The subtractor 130 subtracts the prediction block generated by the intra predictor 122 or the inter predictor 124 from the current block to generate a residual block.

[0066] The transformer 140 transforms the residual signal in the residual block having pixel values ​​in the spatial domain into a transform coefficient in the frequency domain. The transformer 140 may transform the residual signal in the residual block using the entire size of the residual block as a transform unit. Alternatively, the residual block may be divided into a plurality of sub-blocks, and the transform may be performed using the sub-blocks as transform units. Alternatively, the residual signal may be transformed by dividing the block into two sub-blocks, namely a transform region and a non-transform region, and using only the transform region sub-block as a transform unit. The transform region sub-block may be one of two rectangular blocks having a size ratio of 1:1 based on a horizontal axis (or a vertical axis). In this case, the entropy encoding unit 155 encodes a flag (cu_sbt_flag) indicating that only the sub-block is transformed, direction (vertical / horizontal) information (cu_sbt_horizontal_flag) and / or position information (cu_sbt_pos_flag), and notifies the video decoding device with a signal. In addition, the size of the transform region subblock may have a size ratio of 1:3 based on the horizontal axis (or vertical axis). In this case, a flag (cu_sbt_quad_flag) distinguishing the corresponding partition is additionally encoded by the entropy encoder 155 and signaled to the video decoding apparatus.

[0067] The transformer 140 may transform the residual block separately in the horizontal direction and the vertical direction. For the transformation, various types of transformation functions or transformation matrices may be used. For example, the paired transformation functions for horizontal transformation and vertical transformation may be defined as a multiple transform set (MTS). The transformer 140 may select a pair of transformation functions with the best transformation efficiency in the MTS, and transform the residual block in the horizontal direction and the vertical direction, respectively. The information (mts_idx) about the transformation function pair selected in the MTS is encoded by the entropy encoder 155 and notified to the video decoding device with a signal.

[0068] The quantizer 145 quantizes the transform coefficients output from the transformer 140 using the quantization parameters, and outputs the quantized transform coefficients to the entropy encoder 155. For some blocks or frames, the quantizer 145 may directly quantize the relevant residual block without transformation. The quantizer 145 may apply different quantization coefficients (scaling values) according to the position of the transform coefficients in the transform block. The matrix of quantized coefficients applied to the quantized transform coefficients arranged in two dimensions may be encoded and signaled to the video decoding device.

[0069] The rearrangement unit 150 may reclassify coefficient values ​​of the quantized residual value.

[0070] The rearrangement unit 150 can change the 2D coefficient array into a 1D coefficient sequence by coefficient scanning. For example, the rearrangement unit 150 can scan the coefficients from the DC coefficient to the coefficients in the high frequency region by zig-zag scanning or diagonal scanning to output a 1D coefficient sequence. According to the size of the transform unit and the intra-frame prediction mode, the zig-zag scan can be replaced by vertical scanning or horizontal scanning, wherein the vertical scanning scans the 2D coefficient array in the column direction, and the horizontal scanning scans the coefficients in the 2D block shape in the row direction. That is, the scanning mode to be used can be determined among the zig-zag scan, diagonal scan, vertical scan and horizontal scan according to the size of the transform unit and the intra-frame prediction mode.

[0071] The entropy encoder 155 encodes the one-dimensional quantized transform coefficient output from the rearrangement unit 150 using various encoding techniques such as Context-based Adaptive Binary Arithmetic Code (CABAC) and exponential Golomb to generate a bitstream.

[0072] The entropy encoder 155 encodes information related to block segmentation (e.g., CTU size, CU segmentation flag, QT segmentation flag, MTT segmentation type, and MTT segmentation direction) so that the video decoding device can segment the block in the same manner as the video encoding device. In addition, the entropy encoder 155 encodes information about the prediction type indicating whether the current block is encoded by intra-frame prediction or inter-frame prediction, and encodes intra-frame prediction information (i.e., information about the intra-frame prediction mode) or inter-frame prediction information (merge index for merge mode, information about reference image index and motion vector difference for AMVP mode) according to the prediction type. In addition, the entropy encoder 155 also encodes information related to quantization (i.e., information about quantization parameters and information about quantization matrices).

[0073] The inverse quantizer 160 inversely quantizes the quantized transform coefficient output from the quantizer 145 to generate a transform coefficient. The inverse transformer 165 transforms the transform coefficient output from the inverse quantizer 160 from the frequency domain to the spatial domain, and reconstructs a residual block.

[0074] The adder 170 adds the reconstructed residual block and the prediction block generated by the predictor 120 to reconstruct the current block. Pixels in the reconstructed current block are used as reference pixels when performing intra prediction of a subsequent block.

[0075] The loop filter unit 180 filters the reconstructed pixels to reduce blocking artifacts, ringing artifacts, and blurring artifacts caused by block-based prediction and transform / quantization. The loop filter unit 180 may include at least one of a deblocking filter 182, a sample adaptive offset (SAO) filter 184, and an adaptive loop filter (ALF) 186.

[0076] The deblocking filter 182 filters the boundaries between the reconstructed blocks to remove block artifacts caused by block-by-block encoding / decoding, and the SAO filter 184 performs additional filtering on the deblocking filtered video. The SAO filter 184 is a filter for compensating for the difference between the reconstructed sample and the original sample caused by lossy coding, and performs filtering in a manner of adding a corresponding offset to each reconstructed sample. The ALF 186 performs filtering on the target sample to be filtered by applying a filter coefficient to the target sample and the adjacent samples of the target sample. The ALF 186 can divide the samples included in the image into predetermined groups, and then determine a filter to be applied to the corresponding group to perform filtering on each group differentially. Information about the filter coefficient to be used for the ALF can be encoded and notified to the video decoding device with a signal.

[0077] The reconstructed blocks filtered by the loop filtering unit 180 are stored in the memory 190. Once all blocks in one image are reconstructed, the reconstructed image may be used as a reference image for inter-frame prediction of blocks in a subsequent image to be encoded.

[0078] Figure 5 is an exemplary functional block diagram of a video decoding device capable of implementing the technology of the present invention. Figure 5 A video decoding apparatus and elements of the apparatus are described below.

[0079] The video decoding apparatus may include an entropy decoder 510 , a rearrangement unit 515 , an inverse quantizer 520 , an inverse transformer 530 , a predictor 540 , an adder 550 , a loop filtering unit 560 , and a memory 570 .

[0080] Similar to Figure 1 In the video encoding device of the present invention, each element of the video decoding device can be implemented by hardware, software or a combination of hardware and software. In addition, the function of each element can be implemented by software, and the microprocessor can be implemented to perform the software function corresponding to each element.

[0081] The entropy decoder 510 determines a current block to be decoded by decoding a bit stream generated by a video encoding device and extracting information related to block partitioning, and extracts prediction information required to reconstruct the current block, information about a residual signal, and the like.

[0082] The entropy decoder 510 extracts information about the CTU size from a sequence parameter set (SPS) or a picture parameter set (PPS), determines the size of the CTU, and divides the image into CTUs of the determined size. Then, the decoder determines the CTU as the highest level (i.e., the root node) of the tree structure, and extracts partition information about the CTU to partition the CTU using the tree structure.

[0083] For example, when the CTU is segmented using the QTBTTT structure, the first flag (QT_split_flag) related to the segmentation of the QT is extracted to segment each node into four nodes of the sublayer. For the node corresponding to the leaf node of the QT, the second flag (MTT_split_flag) related to the segmentation of the MTT and the information about the segmentation direction (vertical / horizontal) and / or the segmentation type (binary / trifurcated) are extracted to segment the corresponding leaf node with the MTT structure. Thus, each node below the leaf node of the QT is recursively segmented with the BT or TT structure.

[0084] As another example, when a CTU is split using a QTBTTT structure, a CU split flag (split_cu_flag) indicating whether the CU is split may be extracted. When the corresponding block is split, a first flag (QT_split_flag) may be extracted. In a split operation, after zero or more recursive QT splits, zero or more recursive MTT splits may occur for each node. For example, a CTU may directly undergo MTT splitting without undergoing QT splitting, or may only undergo QT splitting multiple times.

[0085] As another example, when the CTU is segmented using the QTBT structure, the first flag (QT_split_flag) related to the QT segmentation is extracted, and each node is segmented into four nodes of the lower layer. Then, the segmentation flag (split_flag) indicating whether the node corresponding to the leaf node of the QT is further segmented with the BT and the segmentation direction information are extracted.

[0086] Once the current block to be decoded is determined by tree structure partitioning, the entropy decoder 510 extracts information about the prediction type indicating whether the current block is intra-predicted or inter-predicted. When the prediction type information indicates intra-prediction, the entropy decoder 510 extracts syntax elements of intra-prediction information (intra-prediction mode) of the current block. When the prediction type information indicates inter-prediction, the entropy decoder 510 extracts syntax elements for inter-prediction information, that is, information indicating a motion vector and a reference image referenced by the motion vector.

[0087] The entropy decoder 510 also extracts information about the quantized transform coefficient of the current block as information related to quantization and information about the residual signal.

[0088] The rearrangement unit 515 may change a sequence of one-dimensional quantized transform coefficients entropy-decoded by the entropy decoder 510 into a 2-dimensional coefficient array (ie, block) in the reverse order of coefficient scanning performed by the video encoding apparatus.

[0089] The inverse quantizer 520 inversely quantizes the quantized transform coefficients and inversely quantizes the quantized transform coefficients using quantization parameters. The inverse quantizer 520 may apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in two dimensions. The inverse quantizer 520 may perform inverse quantization by applying a quantization coefficient (scaling value) matrix from a video encoding device to an array of 2-dimensional quantized transform coefficients.

[0090] The inverse transformer 530 inversely transforms the inversely quantized transform coefficient from the frequency domain to the spatial domain to reconstruct a residual signal, thereby generating a residual block of the current block.

[0091] In addition, when the inverse transformer 530 performs inverse transformation on only a partial area (sub-block) of the transform block, the inverse transform unit 530 extracts a flag (cu_sbt_flag) indicating that only the sub-block of the transform block has been transformed, directional (vertical / horizontal) information (cu_sbt_horizontal_flag) about the sub-block, and / or position information (cu_sbt_pos_flag) about the sub-block, and inversely transforms the transform coefficient of the sub-block from the frequency domain to the spatial domain. Then, the residual signal is reconstructed, and the uninverse transformed area is filled with a "0" value as a residual block, thereby generating a residual block of the final current block.

[0092] In addition, when MTS is applied, the inverse transformer 530 uses the MTS information (mts_idx) notified by the video encoding device using a signal to determine the transformation function or transformation matrix to be applied in the horizontal direction and the vertical direction, respectively, and uses the determined transformation function to inversely transform the transformation coefficients in the transformation block in the horizontal direction and the vertical direction.

[0093] The predictor 540 may include an intra predictor 542 and an inter predictor 544. The intra predictor 542 is activated when the prediction type of the current block is intra prediction, and the inter predictor 544 is activated when the prediction type of the current block is inter prediction.

[0094] The intra predictor 542 determines an intra prediction mode of a current block among a plurality of intra prediction modes based on a syntax element of the intra prediction mode extracted from the entropy decoder 510 , and predicts the current block using reference pixels around the current block according to the intra prediction mode.

[0095] The inter predictor 544 determines a motion vector of a current block and a reference image referred to by the motion vector using a syntax element of the intra prediction mode extracted from the entropy decoder 510 , and predicts the current block based on the motion vector and the reference image.

[0096] The adder 550 reconstructs the current block by adding the residual block output from the inverse transformer 530 and the prediction block output from the inter predictor 544 or the intra predictor 542. When intra-predicting a subsequent block to be decoded, pixels in the reconstructed current block are used as reference pixels.

[0097] The loop filtering unit 560 may include a deblocking filter 562, an SAO filter 564, and an ALF 566. The deblocking filter 562 performs deblocking filtering on the boundaries between the reconstructed blocks to remove block artifacts caused by block-by-block decoding. The SAO filter 564 performs additional filtering on the reconstructed blocks after deblocking filtering to compensate for the difference between the reconstructed pixels and the original pixels caused by lossy coding. The ALF 566 performs filtering on the target sample to be filtered by applying the filter coefficients to the target sample and the adjacent samples of the target sample. The ALF 566 can divide the samples in the image into predetermined groups, and then determine a filter to be applied to the corresponding group to perform filtering on each group differentially. The filter coefficients of the ALF are determined based on information about the filter coefficients decoded from the bitstream.

[0098] The reconstructed blocks filtered by the loop filtering unit 560 are stored in the memory 570. When all blocks in one image are reconstructed, the reconstructed image is used as a reference image for inter-frame prediction of blocks in a subsequent image to be encoded.

[0099] The present invention is directed to encoding and decoding of video as described above. More specifically, the present invention provides a method for efficiently operating a binary data buffer by limiting the ratio of binary data to bits in entropy encoding and decoding associated with bitstream generation and parsing. In addition, the present invention provides such a method, which is configured to include a list of various entropy decoding methods and adaptively utilizes the entropy encoding / decoding method for each basic unit of entropy encoding / decoding.

[0100] The video encoding device uses the above encoding operation to generate a bit stream consisting of one or more continuous network abstraction layer (NAL) units. The NAL unit includes a NAL unit header and a raw byte sequence payload (RBSP), and the RBSP includes a string of data bits (SODB). SODB corresponds to the encoded video data or video parameters.

[0101] NAL units can be classified into video coding layer (VCL) NAL type or non-VCL type. Parameters for video decoding can be included in the non-VCL type NAL unit in the coded state. In addition, data related to the video signal can be included in the VCL type NAL unit in the coded state.

[0102] The NAL unit is generated by the entropy encoder 155 in the video encoding device. Specifically, a context-based adaptive binary arithmetic coding (CABAC) algorithm is used as an arithmetic decoding process to generate a coded video signal included in a NAL unit of the VCL type. The CABAC algorithm is a method of binary arithmetic coding of symbols using previously encoded / decoded statistical information. A symbol is a value represented as a binary number with a value of 0 or 1. A binary data in a binary string formed by binarization of zero or a larger number can be a symbol. The symbol itself can represent a syntax element required for video decoding. The context of the CABAC algorithm refers to statistical information about previously encoded / decoded symbols and is used to improve the prediction performance of subsequent symbols. The compression rate (which is the coding performance of the CABAC algorithm) depends on the context modeling method. As the probability of the most probable symbol (MPS) of the context model increases, the compression rate can be improved.

[0103] Other entropy encoding schemes are used to generate encoded parameters included in the non-VCL type NAL unit or the VCL type NAL unit, but further detailed description thereof will be omitted because the other entropy encoding schemes are not related to the scope of the present invention.

[0104] In the following, various methods are proposed to maintain constraints on the ratio of binary data to bits in order to efficiently manage the binary data buffer.

[0105] Figure 6 is a flow chart of a method for entropy decoding according to an embodiment of the present invention. Figure 6Various methods S602, S604 and S606 for maintaining the constraint on the ratio of binary data to bits are sequentially shown in FIG. 6 , but the present invention is not necessarily limited thereto, and one or more methods may be used to maintain the constraint on the ratio of binary data to bits.

[0106] The entropy decoder 510 of the video decoding apparatus according to the present invention may decode a VCL type NAL unit based on an arithmetic decoding process.

[0107] The entropy decoder 510 checks the type of the NAL unit from the bitstream (S600). The type of the NAL unit can be identified by parsing the NAL unit header, and the following process is continued for the VCL type NAL unit.

[0108] The entropy decoder 510 checks the number of zero words (zero_word) ( S602 ).

[0109] The entropy decoder 510 checks the number of zero_words included in the VCL type NAL unit in order to effectively manage the binary data buffer. The binary data buffer may include a buffer for storing a binary string and a buffer used by the entropy decoder 510 in a decoding process.

[0110] In order to enable the entropy decoder 510 to effectively manage the binary data buffer to cope with buffer overflow / underrun, the entropy encoder 155 of the video encoding device can determine the number of zero_words and add one or more zero_words of a predefined length when generating a VCL type NAL unit. The number of zero_words can be adaptively determined according to at least one of the length of the NAL unit, the length of the binary string, horizontal and vertical size information about the image, bit depth, color space sampling information, temporal identifier (TID), quantization parameter (QP), etc. That is, when the ratio of the number (or length) of binary data to the number (or length) of generated bits exceeds a threshold, the entropy encoder 155 generates a NAL unit by inserting one or more zero_words of a predefined length to maintain the constraint on the ratio of binary data to bits. The threshold can be determined according to at least one of the length (number of bits) of the NAL unit, horizontal and vertical size information of the image, bit depth, color space sampling information (sampling ratio between luminance component and chrominance component), TID, QP, etc. Alternatively, as described below, the threshold can be variably determined according to the level or level of the image.

[0111] In order to improve the efficiency of the encoding / decoding process, the entropy encoder 155 can perform video encoding using at least one of a method of omitting zero_word or a method of adaptively determining the length of zero_word. The entropy decoder 510 checks whether zero_word is omitted in the NAL unit and the number of zero_words, and generates a bitstream for performing entropy decoding by removing zero_words from the NAL unit. In addition, the size of the binary data buffer for applying the arithmetic decoding process to the bitstream can be set with reference to the number of zero_words. The entropy decoder 510 can adaptively maintain and manage the buffer according to the set size of the binary data buffer. The entropy decoder 510 generates a binary string by applying the arithmetic decoding process to the bitstream, and then stores it in the binary data buffer. The arithmetic decoding process performed by the video decoding device may include context-based adaptive binary arithmetic decoding (hereinafter referred to as "adaptive binary arithmetic decoding") processing, and / or as a bypass mode. Based on uniform probability binary arithmetic decoding processing.

[0112] In some embodiments, the constraint on the ratio of binary data to bits may be implemented by selectively skipping the arithmetic decoding process.

[0113] The entropy decoder 510 determines whether to skip the arithmetic decoding process based on skip_flag ( S604 ).

[0114] The entropy decoder 510 may determine whether to perform adaptive binary arithmetic decoding or binary arithmetic decoding on the bitstream according to a preset order. In order to make a determination, a skip_flag indicating whether to skip the arithmetic decoding process may be signaled at a high level. The high level may be an SPS, PPS, CTU, VPDU, shared unit, etc. before the decoding time of the current unit. Alternatively, the high level may be determined based on the skip_flag of one or more units adjacent to the current unit spatially, temporally, or hierarchically. Alternatively, the high level may be determined based on the skip_flag of a unit having a different color space at the same position as the current unit. Alternatively, a syntax element for skipping the application of the arithmetic decoding process may be determined based on previously acquired statistical information. Alternatively, according to a protocol between a video encoding device and a video decoding device, the application of the arithmetic decoding process may be skipped for some syntax elements. For reference, when the skip_flag is signaled, the video decoding device may generate a binary string directly from the bitstream without applying the arithmetic decoding process to the bitstream.

[0115] In some other embodiments, the constraint on the ratio of binary data to bits may be achieved by adaptively switching between adaptive binary arithmetic decoding and uniform probability-based binary arithmetic decoding.

[0116] The entropy decoder 510 determines whether to skip the adaptive binary arithmetic decoding by comparing the length of the previously decoded binary string with a threshold value (S606).

[0117] When the length of the decoded binary string exceeds a preset threshold, the entropy decoder 510 skips the adaptive binary arithmetic decoding and performs binary arithmetic decoding.

[0118] Alternatively, a counter can be used to implement adaptive switching between adaptive binary arithmetic decoding and binary arithmetic decoding based on uniform probability. For example, when the ratio of binary data to bits remains at N / M, reading one bit increases the counter value by N (where N is a natural number), and decoding one binary data decreases the counter value by M (where M is a natural number). For example, when the value of the counter becomes greater than N, adaptive binary arithmetic decoding can be performed. In the opposite case, binary arithmetic decoding as a bypass mode can be performed. The counter can be calculated and initialized based on at least one or more information such as image / slice / tile / brick / CTU size, bit depth, and color space sampling.

[0119] The threshold value (or, N and M) may be transmitted from the video encoding device through one of the SPS / PPS / slice / CTU / VPDU / CU units. Alternatively, the threshold value (or, N and M) may be set for each layer unit according to a protocol between the video encoding device and the video decoding device. Alternatively, the threshold value (or, N and M) may be determined by the video decoding device based on the image / slice / brick / tile / CTU size, bit depth, color space sampling, prediction mode, transform mode, intra-frame prediction mode, inter-frame prediction mode, etc. In addition, the threshold value (or, N and M) may be determined in consideration of the hierarchy and / or level of the image. That is, the threshold value (or, M and N) may be variably set according to the hierarchy and / or level of the image. Alternatively, the ratio between N and M in the current unit may be calculated based on the N and M or their ratio of one or more units around the current unit. Alternatively, N and M for one or more previous units may be stored and managed in a first-in first-out (FIFO) type lookup table, and N and M corresponding to an index transmitted from a video encoding device may be used by a video decoding device.

[0120] In another embodiment of the present invention, when the threshold or counter is not available, the entropy decoder 510 may determine whether to skip adaptive binary arithmetic decoding according to a protocol between the video encoding device and the video decoding device for each layer unit, syntax element, or binary data constituting the syntax element. In this case, the protocol between the video encoding device and the video decoding device may be expressed in the form of a lookup table.

[0121] The entropy decoder 510 performs adaptive binary arithmetic decoding (S608).

[0122] The entropy decoder 510 can decode the binary data to which adaptive binary arithmetic decoding is applied. Figure 7 Describes the details of the adaptive binary arithmetic decoding process.

[0123] The entropy decoder 510 performs inverse binarization ( S610 ).

[0124] By performing inverse binarization, the entropy decoder 510 may generate values ​​representing syntax elements of the video signal from a binary string.

[0125] Figure 7 is a flow chart of an adaptive binary arithmetic decoding process according to an embodiment of the present invention.

[0126] Figure 7 is a detailed flowchart of the adaptive binary arithmetic decoding process (operation S608) performed by the entropy decoder 510 of the video decoding device.

[0127] The entropy decoder 510 determines whether to perform initialization for adaptive binary arithmetic decoding, and performs the initialization if necessary (S700).

[0128] First, when the syntax element to be decoded is the first syntax element of a slice / tile, an initialization operation for adaptive binary arithmetic decoding may be performed. In this case, the following two operations may be performed, namely, an operation of initializing variables pStateIdx0 and pStateIdx1 representing a context model, and an operation of initializing variables ivlCurrRange and ivlOffset representing a decoding state.

[0129] In addition, when the current CTU is the first CTU of a row included in a tile and there is a spatial neighboring block for referencing the value of the variable representing the context model, only the initialization operation of the variable indicating the decoding state is performed. On the other hand, when there is no such spatial neighboring block, an operation of initializing both the context model and the variable indicating the decoding state may be performed.

[0130] Here, the context models pStateIdx0 and pStateIdx1 for each syntax element may be probability models for a maximum probability symbol (MPS) and may have different accuracy and adaptation rates. The context model for each syntax element may be initialized based on the initial values ​​of the probabilities of the MPS and QP for the current unit. The initial value of the probability for the MPS may be obtained from the ctxTable (which is a lookup table for each syntax element) using ctxIdx according to the prediction mode of the current unit. The ctxTable for each syntax element is a lookup table set according to a protocol between a video encoding device and a video decoding device, and provides an initial probability value for the MPS according to ctxIdx and shiftIdx used in a context model update operation. ctxIdx is given as the sum of ctxInc and ctxIdxOffset, where ctxIdxOffset is a value depending on the prediction mode, and ctxInc may be calculated based on various information such as the syntax of neighboring blocks, the depth of the CU, and the size of the transform.

[0131] The variable ivlCurrRange (which indicates the state of adaptive binary arithmetic decoding) represents the current portion for decoding, and ivlOffset is an offset of a preset length obtained from the compressed bitstream and represents a position within the ivlCurrRange. In the initialization operation of the variable indicating the state of decoding, ivlCurrRange can be initialized to a preset value (e.g., 510). Alternatively, ivlCurrRange can be calculated based on a bit depth representing a dynamic range of probabilities. ivlOffset can be initialized to a value of n bits (n is a natural number) obtained from the bitstream. As provided herein, n can be determined based on the length of ivlCurrRange, or n can be signaled from the video encoding device.

[0132] In binary arithmetic decoding, decoding can be performed by repeatedly dividing the portion corresponding to the length of ivlCurrRange into two portions. For two probability values ​​p0 and p1 (where p1=1-p0), the portion of ivlCurrRange can be divided into two portions of length p0×ivlCurrRange and (1-p0)×ivlCurrRange. One of the two portions can be selected according to the binary data value generated by binary arithmetic decoding.

[0133] The entropy decoder 510 generates a binary string by performing binary arithmetic decoding (S702).

[0134] The entropy decoder 510 may perform binary arithmetic decoding processing as part of an adaptive binary arithmetic decoding process. The entropy decoder 510 may update ivlCurrRange and ivlOffset using the context models pStateIdx0 and pStateIdx1 for each syntax element, and generate binary data. First, ivlCurrRange is updated as shown in Equations 1 to 3.

[0135] [Equation 1]

[0136] qRangeIdx = ivlCurrRange >> a

[0137] pState = pStateIdx1 + b * pStateIdx0

[0138] valMps = pState >> c

[0139] [Equation 2]

[0140] ivlLpsRange = (qRangeIdx * ((valMps? (c + 1) - pState : pState)

[0141] >>(c - a)) >> 1) + d

[0142] [Equation 3]

[0143] ivlCurrRange == ivlCurrRange--ivlLpsRange

[0144] Here, a, b, c, and d are all natural numbers, and c is a natural number greater than a. qRangeIdx has a value obtained by dividing ivlCurrRange (the ivlCurrRange is the current part) by (1 << a), and qRangeIdx may represent an index for a specific part. As described above, the two context models pStateIdx0 and pStateIdx1 are probability models for MPS, and each probability model has an accuracy of (c - b) bits and c bits. As shown in Equation 1, using the probability values of the two probability models, the probability value pState of the current state can be calculated, and the MPS value can be determined based on the probability value of the current state. In addition, as shown in Equation 2, after calculating the partial length ivlLpsRange of the least probable symbol (LPS), Equation 3 can be used to update the length of the current part.

[0145] The binary data value may be determined based on the updated ivlCurrRange and ivlOffset. When ivlOffset is greater than or equal to ivlCurrRange, the binary data value is determined as the LPS value (!(MPS value)), and ivlOffset and ivlCurrRange are updated as shown in Equation 4.

[0146] [Equation 4]

[0147] ivlOffset=ivlOffset--ivlCurrRange

[0148] ivlCurrRange=ivlLpsRange

[0149] On the other hand, when ivlOffset is less than ivlCurrRange, the binary data value is determined as the MPS value, and the updating of ivlOffset and ivlCurrRange is skipped. The determined binary data value may form a binary string.

[0150] The entropy decoder 510 stores the binary string in a binary data buffer (S704). As described above, the entropy decoder 510 can decode the value of the syntax element representing the video signal according to the binary data value stored in the binary data buffer using the inverse binarization process S610.

[0151] The entropy decoder 510 updates the context variable ( S706 ).

[0152] To update the context variables, the entropy decoder 510 may calculate the adaptive rates shift0 and shift1 for the two probability models using shiftIdx, as shown in Equation 5.

[0153] [Equation 5]

[0154] shift0=(shiftIdx>>e)+f

[0155] shift1=(shiftIdx&g)+h+shift0

[0156] Here, e, f, g and h are all natural numbers.

[0157] The entropy decoder 510 may update the two probability models pStateIdx0 and pStateIdx1 using the adaptive rates shown in Equation 6 and Equation 7.

[0158] [Equation 6]

[0159] pStateIdx0=min(pStateIdx0-(pStateIdx0>>shift0)+

[0160] ((1<<(cb))*bin>>shift0),pMax0)

[0161] [Equation 7]

[0162] pStateIdx1=min(pStateIdx1-(pStateIdx1>>shift1)+

[0163] ((1<<c)*bin> >shift1),pMax1)

[0164] Here, the min(x, y) function is a function that outputs the smaller value of x and y, and pMax0 and pMax1 are the maximum probability values ​​that the probability model may have. Therefore, compared with the bypass mode, by using the min(x, y) function so that the probability value of each probability model is truncated to the maximum value, the adaptive binary arithmetic coding / decoding process according to the present embodiment effectively limits the amount of binary data and operates the binary data buffer.

[0165] pMax0 and pMax1 may be transmitted from the video encoding device. Alternatively, pMax0 and pMax1 may be determined, calculated, or derived using a lookup table based on ShiftIdx. Alternatively, only one of pMax0 and pMax1 may be signaled, and pMax1 may be calculated based on pMax0, or pMax0 may be calculated based on pMax1. Alternatively, pMax0 and pMax1 may be values ​​predetermined according to a protocol between the video encoding device and the video decoding device. Alternatively, pMax0 and pMax1 of the current unit may be calculated based on pMax0 and pMax1 of the previous unit in the order of encoding and decoding. Alternatively, pMax0 and pMax1 of the current unit may be calculated based on pMax0 and pMax1 of a unit at the same position as the current unit in one or more color spaces.

[0166] The entropy decoder 510 determines and performs renormalization (S708).

[0167] When the length of the updated ivlCurrRange is less than a preset threshold T, the entropy decoder 510 may perform renormalization. The threshold T may be transmitted from the video encoding device. Alternatively, the threshold T may be derived using high-level information. Alternatively, the threshold T may be derived from one or more neighboring units. When performing the renormalization operation, the entropy decoder 510 doubles the length of ivlCurrRange and appends 1 bit obtained from the bitstream to the least significant bit (LSB) of ivlOffset.

[0168] When the adaptive binary arithmetic decoding is skipped, the entropy decoder 510 may perform the binary arithmetic decoding as a bypass mode. In the binary arithmetic decoding process, the probability value for the MPS is set to 0.5, and accordingly, the most significant bit (MSB) of ivlOffset may be determined as a binary data value.

[0169] The above mainly describes entropy decoding. Since entropy coding corresponds to the reverse process of entropy decoding, a detailed description of entropy coding will be skipped. In addition, the above method of maintaining constraints on the ratio of binary data to bits in order to efficiently operate a binary data buffer can even be applied to entropy coding.

[0170] As described above, according to the present embodiment, by providing a method of efficiently operating a binary data buffer in entropy encoding and decoding related to bitstream generation and parsing, the ratio of binary data to bits can be restricted.

[0171] In one aspect of the present invention, the video encoding / decoding apparatus may configure a list including various entropy encoding / decoding methods and adaptively use the entropy encoding / decoding method for each basic unit of entropy encoding / decoding.

[0172] In a method for entropy coding (hereinafter referred to as "entropy coding method"), a syntax element for representing a video signal is binarized and converted into a binary string. Then, adaptive binary arithmetic coding (hereinafter referred to as "binary arithmetic coding") is performed on each binary data of the binary string to generate a bit stream. The syntax element may include information related to segmentation and prediction of the current block, a signal generated by transforming / quantizing a residual signal of the current block, a differential signal, and information related to the quantized signal.

[0173] In order to perform encoding on each binary data of the input binary string, binary arithmetic coding can be performed based on the bit depth of the encoded signal, information about the segmentation of the probability range according to the bit depth, and the state information of the current binary arithmetic coding process within the segmentation probability range, the syntax element, the LPS (MPS) of each binary data of the string, or at least one of the probability information about the LPS (MPS) of each binary data. In the binary arithmetic coding process after binarization, the probability range including the binary data to be encoded can be determined within the current probability range using the probability information related to the LPS (MPS) and the LPS (MPS) of the binary data encoded based on the above information. When the probability range of the binary data to be encoded is determined, the state information about the binary arithmetic coding process can be updated within the segmentation probability range based on the context information related to the corresponding binary data. That is, context information such as the probability information about the LPS (MPS) of the binary data to be encoded can be updated according to the binary arithmetic coding method. Updating of context information of binary data to be encoded can be performed based on one or more syntax elements that are binary arithmetically encoded without additional information according to a protocol between a video encoding device and a video decoding device, and index information about each binary data in a binary string obtained by binarizing the syntax elements.

[0174] In another embodiment of the present invention, a fixed value may be used as the probability of LPS (MPS) and LPS (MPS) applied to binary data whose context information is not updated. For example, the LPS of all binary data whose context information is not updated in the binary string may be fixed to 1, and the probability value of LPS may be fixed to 1 / 2. The fixed probability value of LPS may be set to 1 / 2, 1 / 4, 3 / 8, etc., and various probability values ​​whose denominators can be expressed as powers of 2 may be used. Alternatively, the probability of LPS (MPS) and LPS (MPS) may be encoded using a table with an index, the table may be transmitted to a video decoding device, and binary arithmetic decoding may be performed based on the transmitted information.

[0175] When performing binarization and binary arithmetic coding processes, the entropy encoder 155 of the video encoding apparatus may utilize a basic unit of entropy coding including a coding block (see Figure 8) performs entropy coding using an entropy coding method limited by the high-level information. The entropy encoder 155 transmits a list of entropy coding methods using high-level information, and transmits the index of the list in the basic unit of entropy coding. In addition, the entropy encoder 155 performs entropy coding using the entropy coding method assigned to the corresponding index in the coding block within the basic unit of entropy coding. The basic unit of entropy coding may include one or more coding blocks. The entropy coding method may be one or more K (wherein K is a natural number). When K is 2, the entropy coding method may be a 1-bit flag. In addition, when K is 1, an entropy coding method is provided, so that information about the list index can be omitted, and a fixed method can be used as the entropy coding method according to the agreement between the video encoding device and the video decoding device.

[0176] In one aspect of the present invention, binary arithmetic coding can be skipped in the entropy coding method. When executing the entropy coding method, the entropy encoder 155 can output the corresponding information as a bit stream, and skip the binary arithmetic coding of the binarization result. In addition, the method of skipping binary arithmetic coding can be included in the list of the above-mentioned entropy coding methods. A list including skipping binary arithmetic coding as a first method of entropy coding, a first binary arithmetic coding as a second method, and a second binary arithmetic coding as a third method can be formed. There may be one or more, J binary arithmetic coding methods (where J is a natural number), and there may be K entropy coding methods according to the J binary arithmetic coding methods. K may be greater than or equal to J. In addition, when K is 2, in the entropy coding method, a 1-bit flag may be used to turn on / off the binary arithmetic coding method. When K is 1, there is only one entropy coding method, so the information about the list index can be omitted, and a fixed method according to the protocol between the video encoding device and the video decoding device can be used as the entropy coding method.

[0177] In the present invention, an entropy coding method can be configured as a combination of various binarization methods and binary arithmetic coding methods. In this case, the number K of coding methods included in the list of entropy coding methods can be the maximum number of combinations between binarization methods and binary arithmetic coding methods. In addition, a specific binarization method can be combined with a specific binary arithmetic coding method. After the configuration of the list is determined, the list can be transmitted from the video encoding device to the video decoding device, or the configuration of the list can be determined according to a protocol between the video encoding device and the video decoding device. In addition, the binarization method can be different between the case of skipping binary arithmetic coding and the case of performing binary arithmetic coding.

[0178] Detailed information about the binarization method can be derived at the video decoding device according to a protocol between the video encoding device and the video decoding device without transmitting additional information. In addition, a portion of the index information can be transmitted to the video decoding device, and then all information about the binarization method can be derived.

[0179] In the entropy encoding method according to the present invention, information about the binary arithmetic coding method transmitted at a high level may include at least one of context information about binary data after binarization of each syntax element for encoding, such as the MPS of the binary data and probability information about the MPS, the size of a probability update window for updating a probability range, and the probability of the LPS (MPS) of the binary data, and an indication of whether to update the context information. The binary arithmetic coding method is characterized by context information about the binary data after binarization of each syntax element, such as the MPS of the binary data and probability information about the MPS, the size of a probability update window for updating a probability, and the probability of the LPS (MPS) of the binary data, and an update configuration of the context information. A plurality of binary arithmetic coding methods constitute a list.

[0180] Accordingly, according to the binary arithmetic coding method used by the binary data to be encoded, it can be discriminated whether to update the context such as the probability update window size, the MPS, and the initial probability information about the MPS, and the probability information about the LPS (MPS) of the binary data. For example, when the MPS of the binary data indicated as "0" is 1, the probability of the MPS is 3 / 4, and the update of the context such as the probability information about the LPS (MPS) of the binary data is not performed, the corresponding binary data is 0 and is not MPS, but the probability of the MPS can be maintained at 3 / 4 without the update of the context such as the probability information about the LPS (MPS) of the binary data, and then the encoding can be performed.

[0181] In addition, whether to update the context of the binary data can be determined according to the agreement between the video encoding device and the video decoding device. Accordingly, when an agreement is reached between the video encoding device and the video decoding device to skip the context update of a specific syntax element or specific binary data of a specific syntax element, the context can be updated unconditionally for the remaining binary data.

[0182] For encoding efficiency, probability information about the MPS may be indicated as probability information about the LPS and transmitted, and may be derived by the video decoding apparatus as “1-probability information of the LPS”.

[0183] In the entropy coding method according to the present invention, a method for performing binary arithmetic coding on binary data after binarization without context update can be set as a separate binary arithmetic coding method. In this method, binary arithmetic coding is performed using only the MPS of the binary data after binarization and the probability information about the MPS, and can be performed repeatedly without context update. Therefore, this binary arithmetic coding method can be defined by the MPS and MPS probability information for each binary data of each syntax element. That is, each binary data for each syntax element can have an independent MPS and MPS probability information, or can have the same MPS and MPS probability information. Alternatively, only one MPS and the probability information about the MPS can be the same.

[0184] When both the MPS and the MPS probability information for each binary data of each syntax element are the same, the video encoding device may directly transmit a combination of the MPS and the MPS probability information to the video decoding device, or may transmit derivable information so that the video decoding device can derive the combination of the MPS and the MPS probability information. In addition, when each binary data of each syntax element has an independent combination, the video encoding device may directly transmit each combination to the video decoding device, or may transmit derivable information so that the video decoding device can derive the combination. In addition, when all MPSs are the same and the probability information about the MPSs is not the same, the video encoding device may directly transmit an MPS and probability information for each binary data of each syntax element to the video decoding device, or may transmit derivable information so that the video decoding device can derive the MPS and probability information for each binary data of each syntax element.

[0185] In the entropy coding method according to the present invention, information about the list components of the binary arithmetic coding method can be transmitted from the video encoding device to the video decoding device using a high-level syntax element. In the video encoding / decoding structure, the high-level syntax element can be transmitted at a level with a partitioning concept higher than the entropy coding unit, or at one of the levels such as VPS, SPS, PPS, APS or SEI messages. The higher partitioning concept can mean the unit into which an image is partitioned, and can be a group of blocks such as a sub-image, a tile group, a tile, a slice, a brick, etc., and a group of pixels such as a grid.

[0186] In the entropy coding method according to the present invention, the probability value of the initial MPS, the size of the probability range update window, and the range division depth for dividing the normalized value between 0 and 0.5 into probability ranges can be expressed in the exponential form of k. As provided herein, k can be a positive integer greater than or equal to 2, and the exponents p, q, and r can be non-zero integers. At least one of the exponents p, q, and r can be signaled from the video encoding device to the video decoding device. The video encoding / decoding device can use the exponents p, q, r, and k to derive the starting probability of the initial MPS and the value of the update window. For example, when k is 2, the probability index p of the MPS is 3, the window size index q is 1, and the range depth index r is 7, the probability range between 0 and 0.5 is divided into corresponding to 2 7 (0~127), the initial probability position of MPS is 8, that is, 2 3 position, and with 2 1 When expressed as a normalized value between 0 and 0.5, the initial probability of MPS may be 1 / 16, and the probability update may be performed in units of 1 / 128.

[0187] In addition, the depth of the partition, the size of the update window, and the MPS and probability information about the MPS can be transmitted in the form of a table with an index. Information about the binary arithmetic coding method according to the present invention can be indicated by transmitting an index corresponding to each binary data in the binary string of each syntax element.

[0188] In the entropy encoding method according to the present invention, the entropy encoder 155 may have a list memory to store state information about binary arithmetic encoding. According to the binary arithmetic encoding method applied in the basic unit including entropy encoding of the coding block (see Figure 8 ), the entropy encoder 155 performs binary arithmetic coding after binarization of the coding block. When the binary arithmetic coding method of the basic unit of the entropy coding of the block to be encoded performs binary arithmetic coding after binarization, the probability information as the execution result can be stored. Since the N binary arithmetic coding methods constitute a list, the storage and management of the memory are also performed according to the index of the binary arithmetic coding method list. When the same binary arithmetic coding method is applied later, the stored probability information can be used and updated. The application of the same binary arithmetic coding method includes the same entropy coding basic unit, and may include a case where the unit is not the same entropy coding basic unit. Accordingly, even after the encoding of one basic unit of entropy coding is completed, the stored probability information can be stored and used.

[0189] In addition, initialization of binary arithmetic coding may be performed for each entropy coding unit. In this case, after probability information is stored and managed in one entropy coding unit, when one entropy coding is finished, the stored information may be initialized. In addition, when initialization of binary arithmetic coding is performed in each entropy coding unit, a memory in a basic unit of entropy coding may be used to store and manage probability information.

[0190] Hereinafter, a method for entropy decoding performed by a video decoding apparatus (hereinafter, referred to as an "entropy decoding method") for a bitstream generated by a video encoding apparatus according to the entropy encoding method as described above will be described.

[0191] Fig. 9 is a flow chart of a method for entropy decoding according to another embodiment of the present invention.

[0192] The entropy decoder 510 parses information related to the entropy decoding method included in the high level of the bitstream (S900). The parsed information may be stored in a related memory.

[0193] The entropy decoder 510 derives entropy decoding information about a basic unit of entropy decoding (S902).

[0194] Based on the parsed high-level information, the entropy decoder 510 can derive information about the binary arithmetic coding method, the size of the basic unit of entropy decoding (see Figure 8 ), a binary arithmetic coding method of a basic unit of entropy decoding, a binary arithmetic decoding method of a current decoding block according to an index, etc. In addition, after the above information is derived in a basic unit of entropy decoding, additional information can be derived in a decoding block unit.

[0195] The entropy decoder 510 acquires information about binary arithmetic decoding (S904). Information about a binary arithmetic decoding method of a corresponding index may be acquired from a storage memory related to binary arithmetic decoding.

[0196] The entropy decoder 510 performs binary arithmetic decoding according to whether binary arithmetic decoding is performed ( S906 ), and performs inverse binarization ( S908 ).

[0197] When a current decoding method of a basic unit of entropy decoding is a method of skipping binary arithmetic decoding, the entropy decoder 510 may perform entropy decoding by inversely binarizing a value of a bitstream.

[0198] In the case of entropy decoding including inverse binarization, the entropy decoder 510 may generate an entropy decoding result by inverse binarization of the binary string. In the case of entropy decoding not including inverse binarization, the entropy decoder 510 may output a binary string as the entropy decoding result.

[0199] In the following, reference will be made to Fig.10 A detailed description is given of binary arithmetic decoding of each binary data of a bit stream (operation S906 ) performed when a current decoding method of a basic unit of entropy decoding is a method of performing binary arithmetic decoding.

[0200] Fig.10 is a flowchart of a binary arithmetic decoding process according to another embodiment of the present invention.

[0201] The entropy decoder 510 determines binary data based on the probability range information and the offset information (S1000). Based on the probability range information and the offset information, the entropy decoder 510 can decide the range including the decoded binary data between the probability ranges of the LPS and the MPS, and determine the binary data suitable for the corresponding range.

[0202] The entropy decoder 510 determines whether decoding on binary data is binary arithmetic decoding in which context updating is performed or binary arithmetic decoding in which updating is skipped ( S1002 ), and performs context updating ( S1004 ).

[0203] For ease of calculation, context updating may be performed using one or more of a method of changing the index of a table and mapping the changed index of the table, a method of directly updating the probability to be updated using an equation, a method of performing an update using a shift operation, or a method of multiplying a weight and performing a shift operation.

[0204] In addition, context updating can be performed using one or more of the following methods: a method of applying to all binary data according to a protocol between video encoding / decoding devices or binary arithmetic decoding information transmitted at a high level, a method of selectively applying to some binary data according to a protocol between video encoding / decoding devices, or a method of selectively applying to each binary data according to binary arithmetic decoding information transmitted at a high level.

[0205] In the context update, the probability information about LPS and MPS may be changed using one or more of the following methods: a method of changing the index of a table according to a protocol between video encoding / decoding devices, a method of adding a preset probability value or multiplying a preset value or shifting a preset value according to a protocol between video encoding / decoding devices, or a method of adaptively adding a probability value or multiplying a preset value or shifting a preset value to each binary data according to context information transmitted at a high level.

[0206] The entropy decoder 510 stores the state of binary arithmetic decoding ( S1006 ), and outputs determined binary data ( S1008 ).

[0207] The detailed execution process of the entropy decoding method not described above may be performed using a method corresponding to the above-described entropy encoding method.

[0208] In this embodiment, when the encoding / decoding method of skipping binary arithmetic encoding / decoding is selected, entropy encoding / decoding can be adaptively performed according to information about the current encoding / decoding block. Adaptive execution means that the binary arithmetic encoding / decoding method can be derived based on the encoding / decoding information about the encoding / decoding block and the adjacent blocks, independently of the information specified in the basic unit of entropy encoding / decoding including the encoding / decoding block. When the number of entropy coding lists K is 1 and the binary arithmetic encoding / decoding method of skipping the entropy encoding / decoding method applied in the basic unit of entropy encoding / decoding is used, the entropy encoding / decoding methods of the basic units of entropy encoding can all be the same.

[0209] In addition, when specific conditions are met according to the size of the block currently being decoded, the prediction mode of the block, the MV accuracy of the block, the signal characteristics (luminance / chrominance) of the block, the segmentation state of the block, the type of the transform kernel of the block, the state of the second transform of the block, the quantization coefficient value of the block, the quantization matrix value of the block, the state of the transform of the block, etc., the entropy encoding / decoding method can be determined as a method for skipping binary arithmetic encoding / decoding. In this case, the specific skipped method may be a method not included in the transmission list of the entropy encoding / decoding method. The method may be a method according to a protocol between video encoding / decoding devices, and the video encoding / decoding device can identify the entropy encoding / decoding method without the need for transmitted information.

[0210] For example, when the current decoding block has a maximum size that can be encoded / decoded without segmentation, the entropy encoding / decoding method of the block may be a method with skipping binary arithmetic encoding / decoding. In addition, when the current decoding block is segmented into blocks of a minimum size, the entropy encoding / decoding method of the block may be a method with skipping binary arithmetic encoding / decoding. The foregoing is an example of a specific condition, and various implementations may be possible depending on the situation in which such a specific condition can be derived.

[0211] As described above, according to the present invention, in entropy coding and decoding related to the generation and parsing of a bit stream, a method is provided, wherein a configuration includes a list of various entropy coding / decoding methods and the entropy coding / decoding method is adaptively utilized for each basic unit of entropy coding / decoding. Accordingly, entropy coding / decoding can be performed according to the characteristics of an application and a signal.

[0212] Although the various operations are shown to be performed sequentially in each flowchart according to the present embodiment, the embodiment is not limited thereto. In other words, changes in the operations in the execution or parallel execution of one or more operations may be applicable, so the flowchart is not limited to the time series order.

[0213] Although exemplary embodiments have been described for illustrative purposes, those skilled in the art will appreciate that various modifications and variations are possible without departing from the spirit and scope of the embodiments. For the sake of brevity and clarity, exemplary embodiments have been described. Accordingly, those of ordinary skill will appreciate that the scope of the embodiments is not limited to the embodiments explicitly described above, but is included in the claims and their equivalents.

Claims

1. An apparatus for entropy decoding performed by a video decoding apparatus, the apparatus comprising at least one processor, the at least one processor being configured to: receiving a bitstream generated by encoding an image; performing an arithmetic decoding process to generate at least one binary string by decoding the bit stream, each binary string including at least one binary data; as well as Generate syntactic elements by debinarizing the binary string, wherein the amount of binary data generated by decoding the bit stream satisfies a constraint that the amount does not exceed a threshold value, The threshold is a value calculated based on the width and height of the image, the sampling ratio between the luma component and the chroma component, and the bit depth, The threshold value can be variably set according to the level or grade of the video.

2. The device for entropy decoding performed by a video decoding device according to claim 1, wherein: In order to satisfy the constraint that the amount of binary data does not exceed the threshold, performing arithmetic decoding processing includes: The counter is incremented by N each time a bit is read, and the counter is decremented by M according to the appearance of a binary data, wherein M and N are natural numbers and are determined based on the threshold value; Depending on whether the counter is greater than N, context-based adaptive binary arithmetic decoding or uniform probability-based binary arithmetic decoding is selectively performed.

3. The device for entropy decoding performed by a video decoding device according to claim 1, wherein: The arithmetic decoding process includes: generating a specific portion from a current portion for decoding; For each syntax element, obtain a current probability value for a maximum probability symbol MPS using a first context model and a second context model, and generate an MPS value; Calculating the length of the portion of the least-probable symbol LPS using the specific portion and the MPS value, and updating the current portion using the length of the portion of the LPS; Generate a binary data using an offset of a preset length obtained from a bit stream and an updated current portion, and readjust the offset and the current portion when a value of an LPS is determined as a binary data, wherein the value of the LPS is generated from the value of the MPS; updating the first context model and the second context model; and When the readjusted current portion is smaller than a preset threshold, the current portion and the offset are renormalized.

4. The device for entropy decoding performed by a video decoding device according to claim 3, wherein: Updating the first context model and the second context model includes: For each syntax element, calculating a first adaptation rate and a second adaptation rate; and updating the first context model using a first adaptation rate and updating the second context model using a second adaptation rate, The first context model is trimmed to a maximum probability value that the first context model can have, and the second context model is trimmed to a maximum probability value that the second context model can have.

5. An apparatus for entropy encoding a syntax element, wherein the syntax element is generated based on predictive encoding of each block constituting an image, the apparatus comprising at least one processor, wherein the at least one processor is configured to: Binarize each syntax element and generate at least one binary string, each binary string including at least one binary data; performing an arithmetic encoding process to generate encoded data from the binary string; as well as Generates a bitstream consisting of one or more Network Abstraction Layer (NAL) units from the encoded data. wherein the amount of binary data relative to the length of one or more NAL units is constrained not to exceed a threshold, The threshold is a value calculated based on the width and height of the image, the sampling ratio between the luma component and the chroma component, and the bit depth, The threshold value can be variably set according to the level or grade of the video.

6. The apparatus for entropy encoding a syntax element according to claim 5, wherein: When the amount of binary data exceeds a threshold with respect to the length of the one or more NAL units, each of the one or more NAL units is configured by inserting one or more zero words of a predefined length.

7. The apparatus for entropy encoding a syntax element according to claim 5, wherein: In order to satisfy the constraint that the amount of binary data does not exceed a threshold, performing arithmetic coding processing includes: incrementing the counter by N per bit and decrementing the counter by M per binary data, wherein M and N are natural numbers and are determined based on the threshold; Depending on whether the counter is greater than N, context-based adaptive binary arithmetic coding or uniform probability-based binary arithmetic coding is selectively performed.

8. An apparatus for sending a bitstream comprising encoded video data, the apparatus comprising at least one processor, the at least one processor being configured to: generating a bitstream by encoding syntax elements generated based on predictive encoding of each block constituting the image; Sending a bitstream to a video decoding device, in, The processor is configured to generate a bitstream by the following process, the process comprising: Binarize each syntax element to generate at least one binary string, each binary string including at least one binary data; performing an arithmetic encoding process to generate encoded data from the binary string; and generating a bitstream composed of one or more network abstraction layer NAL units from the encoded data, wherein an amount of binary data is constrained not to exceed a threshold with respect to a length of the one or more NAL units, The threshold is a value calculated based on the width and height of the image, the sampling ratio between the luma component and the chroma component, and the bit depth, The threshold value can be variably set according to the level or grade of the video.

Images