Image encoding / decoding method and device

Abstract

The present invention relates to an image encoding / decoding method and device. The image decoding method according to an embodiment of the present invention may comprise the steps of: binarizing a syntax element; determining a context model for the binarized syntax element; encoding the binarized syntax element on the basis of a probability value induced by the context model; determining whether to update the context model; and updating the probability value of the context model on the basis of whether to update the probability value.

Description

Video encoding / decoding method and device

[0001] The present invention relates to an image encoding / decoding method and apparatus, and more specifically, to an image encoding / decoding method and apparatus for improving the throughput of CABAC (Context Adaptive Binary Arithmetic Coding).

[0002] Recently, the demand for multimedia data, such as video, has been increasing rapidly. In particular, the demand for high-resolution, high-quality video, such as HD (High Definition) and UHD (Ultra High Definition), is growing across various application fields. High-resolution, high-quality video data involves a much larger volume compared to conventional video data. Consequently, the transmission and storage costs for storing and / or transmitting such high-resolution, high-quality video data increase compared to conventional video data.

[0003] To solve these problems, high-efficiency video encoding / decoding technology for videos with higher resolution and quality is required.

[0004] To encode images, various techniques are used, including intra-prediction techniques that predict pixel values ​​within the current picture using pixel information within the current picture, intra-prediction techniques that predict pixel values ​​from previous or subsequent pictures, transformation and quantization techniques to compress the energy of residual signals—the difference between the predicted signal and the original signal—and entropy coding techniques that assign short codes to values ​​with high frequency and long codes to values ​​with low frequency. Furthermore, to improve image encoding efficiency, various tools are being developed to implement each of these techniques. Additionally, to decode the encoded image, the image can be restored and reproduced through image decoding techniques that utilize technologies and tools corresponding to the image encoding techniques.

[0005] By utilizing these video encoding and video decoding technologies, video data can be effectively compressed, transmitted, stored, and played back.

[0006] The present disclosure aims to provide an image encoding / decoding method and apparatus that improves the inefficiency of CABAC (Context Adaptive Binary Arithmetic Coding) and enhances the compression efficiency of image data.

[0007] In addition, the present disclosure aims to provide an image encoding / decoding method and apparatus that improve the compression efficiency of image data by improving the algorithm of the encoding / decoding process.

[0008] The technical problems to be solved by the present disclosure are not limited to those mentioned above. In addition, other technical problems not mentioned in the present disclosure will be clearly understood by those skilled in the art from the present disclosure.

[0009] A video decoding method according to one embodiment of the present invention may include the steps of: acquiring a bitstream containing encoded syntactic elements; determining a context model for the encoded syntactic elements; decoding the encoded syntactic elements based on a probability value derived by the context model; determining whether to update the probability value; and updating the probability value of the context model based on whether to update the probability value.

[0010] In the above image decoding method, whether the probability value is updated can be determined based on the above syntax element.

[0011] In the above image decoding method, whether the probability value is updated can be determined based on the number of bins of the syntax elements being decoded.

[0012] In the above image decoding method, whether the probability value is updated can be determined based on the continuity of the bins of the decoded syntax elements.

[0013] In the above image decoding method, whether the probability value is updated can be determined according to a preset prediction unit.

[0014] In the above image decoding method, whether the probability value is updated may be determined as not updating the probability value when a predetermined number or more of pre-set prediction unit syntax elements are decoded.

[0015] A video encoding method according to one embodiment of the present invention may include the steps of binarizing syntactic elements, determining a context model for the binarized syntactic elements, encoding the binarized syntactic elements based on a probability value derived by the context model, determining whether to update the probability value, and updating the probability value of the context model based on whether to update the probability value.

[0016] A non-transient computer-readable recording medium storing a bitstream generated by an image encoding method according to an embodiment of the present invention can store a bitstream generated by an image encoding method comprising: a step of binarizing a syntactic element; a step of determining a context model for the binarized syntactic element; a step of encoding the binarized syntactic element based on a probability value derived by the context model; a step of determining whether to update the probability value; and a step of updating the probability value of the context model based on whether to update the probability value.

[0017] A method for transmitting a bitstream generated by a video encoding method according to an embodiment of the present invention includes the step of transmitting the bitstream, and the video encoding method can transmit the bitstream generated by the video encoding method, the step of binarizing a syntactic element, the step of determining a context model for the binarized syntactic element, the step of encoding the binarized syntactic element based on a probability value derived by the context model, the step of determining whether to update the probability value, and the step of updating the probability value of the context model based on whether to update the probability value.

[0018] According to the present disclosure, an image encoding / decoding method and apparatus can be provided that improve the inefficiency of CABAC (Context Adaptive Binary Arithmetic Coding) and improve the compression efficiency of image data.

[0019] In addition, according to the present disclosure, an image encoding / decoding method and apparatus may be provided that improve the compression efficiency of image data by improving the information reference process during the encoding / decoding process.

[0020] In addition, according to the present disclosure, an image encoding / decoding method and apparatus can be provided that improve the compression efficiency of image data by improving the algorithm of the encoding / decoding process.

[0021] Additionally, according to the present disclosure, a recording medium storing a bitstream generated by the image encoding method or device of the present invention may be provided.

[0022] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art from the description below.

[0023] FIG. 1 is a block diagram showing an image encoding device according to one embodiment of the present invention.

[0024] FIG. 2 is a block diagram showing an image decoding device according to one embodiment of the present invention.

[0025] FIG. 3 is a schematic diagram showing a video coding system to which the present invention can be applied.

[0026] FIG. 4 is a diagram illustrating an exemplary content streaming system to which an embodiment according to the present invention can be applied.

[0027] Figure 5 is a diagram showing a CABAC encoding structure for encoding a single syntax element.

[0028] Figure 6 is a diagram showing a CABAC decoding structure for decoding a single syntax element.

[0029] FIG. 7 is a diagram showing a low-complexity CABAC encoding structure for encoding a single syntax element according to one embodiment of the present disclosure.

[0030] FIG. 8 is a diagram showing a low-complexity CABAC decoding structure for decoding a single syntax element according to one embodiment of the present disclosure.

[0031] FIG. 9 is a flowchart illustrating an image decoding method for a low-complexity CABAC method according to one embodiment of the present disclosure.

[0032] FIG. 10 is a flowchart illustrating an image encoding method for a low-complexity CABAC method according to one embodiment of the present disclosure.

[0033] The present invention is susceptible to various modifications and may have various embodiments; specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the invention to specific embodiments, and it should be understood that the invention includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the invention. Similar reference numerals have been used for similar components in the description of each drawing.

[0034] Terms such as "first," "second," etc., may be used to describe various components, but said components should not be limited by said terms. These terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" includes a combination of a plurality of related described items or any of a plurality of related described items.

[0035] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0036] The terms used in this application are used merely to describe specific embodiments and are not intended to limit the invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" are intended to specify the presence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0037] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. Hereinafter, the same reference numerals are used for identical components in the drawings, and redundant descriptions of identical components are omitted.

[0038] FIG. 1 is a block diagram showing an image encoding device according to one embodiment of the present invention.

[0039] Referring to FIG. 1, the image encoding device (100) may include an image segmentation unit (101), an intra prediction unit (102), an inter prediction unit (103), a subtraction unit (104), a conversion unit (105), a quantization unit (106), an entropy encoding unit (107), an inverse quantization unit (108), an inverse conversion unit (109), an addition unit (110), a filter unit (111), and a memory (112).

[0040] Each component shown in FIG. 1 is depicted independently to represent different characteristic functions of the image encoding device and does not imply that each component consists of separate hardware or a single software unit. That is, each component is listed and included as a separate component for the convenience of explanation, but at least two of the components may be combined to form a single component, or a single component may be divided into multiple components to perform functions, and such integrated and separated embodiments of each component are included within the scope of the present invention as long as they do not deviate from the essence of the present invention.

[0041] Furthermore, some components may not be essential components performing an essential function in the present invention, but merely optional components for enhancing performance. The present invention may be implemented by including only the components essential for realizing the essence of the present invention, excluding components used solely for performance enhancement, and a structure including only the essential components, excluding optional components used solely for performance enhancement, is also included within the scope of the rights of the present invention.

[0042] The image segmentation unit (101) can divide the input image into at least one block. At this time, the input image may have various shapes and sizes, such as a sequence, picture, slice, tile, segment, tile group, coding tree unit, etc. According to another embodiment, the image segmentation unit (101) may divide an input picture into a plurality of sub-pictures defined as a group of rectangular slices, divide each sub-picture into the tile / slice, and divide the tile / slice into coding tree units.

[0043] Additionally, the image segmentation unit (101) can recursively segment the segmented coding tree unit. The terminal node segmented from the coding tree unit may be referred to as a coding unit (CU). A block may refer to a coding unit (CU), or a prediction unit (PU) or a transformation unit (TU) segmented from the coding unit (CU). The segmentation may be performed based on at least one of a quadtree, a binary tree, and a ternary tree. A quadtree is a method of dividing an upper block into lower blocks, each having a width and height that are half that of the upper block. A binary tree is a method of dividing an upper block into lower blocks, each having either a width or a height that is half that of the upper block. A ternary tree is a method of dividing an upper block into three lower blocks. For example, the three lower blocks may be obtained by dividing the width or height of the upper block in a ratio of 1:2:1. Through the aforementioned binary tree-based partitioning, blocks can have shapes that are not only square but also non-square. Blocks can first be partitioned into a quad tree. Blocks corresponding to the leaf nodes of the quad tree may not be partitioned, or may be partitioned into a binary tree or a terminal tree. The leaf nodes of the binary tree or terminal tree may be units of encoding, prediction, and / or transformation.

[0044] The image segmentation unit (101) can recursively segment the CTU into a quad tree (QT) as well as a multi-type tree (MTT). Here, the MTT can be composed of a binary tree (BT) and a triple tree (TT). For example, the MTT structure can be divided into a vertical binary tree segmentation mode (SPLIT_BT_VER), a horizontal binary tree segmentation mode (SPLIT_BT_HOR), a vertical binary tree segmentation mode (SPLIT_TT_VER), and a horizontal binary tree segmentation mode (SPLIT_TT_HOR).

[0045] Additionally, the image segmentation unit (101) can divide the CTU by applying a dual tree in which the CTU segmentation structures of the luminance and color difference components are used differently, or by applying a single tree in which the luminance and color difference CTBs (Coding Tree Blocks) within the CTU share a coding tree structure.

[0046] The prediction unit (102, 103) may include an intra prediction unit (102) that performs intra prediction and an inter prediction unit (103) that performs inter prediction. The prediction unit (102, 103) may determine whether to use intra prediction or perform inter prediction for a prediction unit. Additionally, the prediction unit (102, 103) may determine specific information (e.g., intra prediction mode, inter prediction mode, motion vector, reference picture, etc.) according to the determined prediction method. At this time, the processing unit in which the prediction is performed and the processing unit in which the prediction method and specific details are determined may be different. For example, the prediction unit (102, 103) may determine the prediction method and prediction mode, etc. for each prediction unit and perform prediction according to the transformation unit.

[0047] According to another embodiment, the prediction unit may encode an input image using a third mode (e.g., IBC mode, Palette mode, etc.) that is a mode other than the intra mode and the inter mode. However, if the third mode has functional characteristics similar to the intra mode or the inter mode, the third mode may be classified as the intra mode or the inter mode. In this disclosure, the third mode will be described only when a specific description of the third mode is required.

[0048] The intra prediction unit (102) can generate a prediction block of the current block based on the intra prediction mode of the current block and reference pixel information around the current block, which is pixel information within the current picture. If the surrounding blocks of the current block are predicted by inter prediction, the reference pixels included in the inter-predicted surrounding blocks can be replaced with reference pixels in other surrounding blocks that are intra-predicted. That is, if a reference pixel is not available, the intra prediction unit (102) can perform intra prediction of the current block by replacing the unavailable reference pixel with at least one of the available reference pixels.

[0049] The intra prediction unit (102) may use multiple reference pixel lines for intra prediction of the current block. When multiple reference pixel lines are available, information indicating the reference pixel line used for intra prediction among the multiple reference pixel lines may be signaled.

[0050] The intra prediction mode used for intra prediction may include a directional prediction mode that uses reference pixel information according to the prediction direction and a non-directional mode that does not use directional information. Here, the intra prediction mode may be derived using a list of most probable modes (MPM), and in particular, may be derived using a first MPM and / or second MPM list. Additionally, the mode for predicting luminance information and the mode for predicting chrominance information may be different, and the intra prediction mode information of the luminance component block or the predicted luminance signal information may be utilized to predict chrominance information.

[0051] Alternatively, the intra prediction unit (102) may perform intra prediction for the current block by applying at least one of the following modes: DIMD (decoder-side intra mode derivation), BVG-DIMD (block-vector guided DIMD), OBIC (Occurrence-based intra coding), EIP (extrapolation filter based intra prediction mode), BVG-DIMD (block-vector guided EIP), MM-EIP (multi-model EIP), TIMD (template based intra mode derivation), TIMD merge mode, SGPM (spatial geometric partitioning mode) mode, intra template matching prediction (IntraTMP), and intra block copy. When the intra prediction mode of the block is a predetermined mode, the intra prediction unit (102) may perform intra prediction for the current block using a block vector indicating a block other than the current block.

[0052] In particular, when the current block is a block of chroma components, the intra prediction unit (102) can perform intra prediction for the current block by applying at least one of DM (direct mode), DBV (direct block-vector), DIMD chroma mode, CCLM (cross-component linear model), CCCM (convolutional cross-component model), BVG-CCCM (block-vector guided CCCM), and GL-CCCM (gradient and location based CCCM).

[0053] The intra prediction unit (102) may include a reference sample filter, an interpolation filter, and a DC filter. The reference sample filter is a filter that performs filtering on the reference pixel of the current block and may be adaptively applied depending on the prediction mode, size, shape, and / or whether the reference pixel is included in the reference pixel line immediately adjacent to the current block. If the prediction mode of the current block is a mode that does not perform reference sample filtering, the reference sample filter may not be applied.

[0054] The interpolation filter is a filter that interpolates and filters the prediction samples of the current block, and can be applied adaptively depending on the prediction mode, size, shape, and / or whether the reference pixel is included in the reference pixel line immediately adjacent to the current block.

[0055] If the prediction mode of the current block is DC mode, a prediction block can be generated by applying a DC filter.

[0056] According to one embodiment, the intra prediction unit (102) can perform intra prediction using a pre-trained neural network (NN) model. For example, the intra prediction unit (102) can induce an intra prediction mode of a block to which a DIMD mode is applied and use a pre-trained neural network model to perform intra prediction.

[0057] The inter prediction unit (103) generates a prediction block using a previously restored reference image stored in memory (112), an inter prediction mode, and motion information. Here, inter prediction may mean motion prediction or motion compensation.

[0058] The inter prediction unit (103) can set the inter mode of the prediction unit included in the encoding unit to one of Skip Mode, Merge Mode, or Advanced Motion Vector Prediction (AMVP) Mode in order to perform motion prediction and / or motion compensation. In addition, the inter prediction unit (103) can perform motion prediction and / or motion compensation for the prediction unit according to the set mode.

[0059] Additionally, the inter prediction unit (103) can perform motion prediction and / or motion compensation for the prediction unit by applying the AFFINE mode of sub-PU-based prediction, the SbTMVP (Subblock-based Temporal Motion Vector Prediction) mode, and the MMVD (Merge with MVD) mode and GPM (Geometric Partitioning Mode) mode of PU-based prediction based on the inter prediction mode. In addition, the inter prediction unit (103) can perform motion prediction and / or motion compensation for the prediction unit by applying HMVP (History based MVP), PAMVP (Pairwise Average MVP), CIIP (Combined Intra / Inter Prediction), AMVR (Adaptive Motion Vector Resolution), DMVR (Decoder side Motion Vector Refinement), BDOF (Bi-Directional Optical-Flow), PROF (Prediction Refinement With Optical Flow), BCW (Bi-predictive with CU Weights), LIC (Local Illumination Compensation), TM (Template Matching), OBMC (Overlapped Block Motion Compensation), etc., to improve the performance of each mode.

[0060] Here, AFFINE mode can be used in both AMVP and MERGE modes. It is a technique with high encoding efficiency. AFFINE mode can be a prediction mode using a 4-parameter affine motion model using two control point motion vectors (CPMV) and a 6-parameter affine motion model using three control point motion vectors. Here, CPMV can be a vector representing any one of the top-left, top-right, or bottom-left affine motion models of the current block.

[0061] Motion information may include, for example, motion vectors, reference picture indices, List 1 prediction flags, List 0 prediction flags, half-sample interpolation filter indices, bidirectional prediction weight indices, etc.

[0062] According to one embodiment, the inter prediction unit (103) can perform inter prediction using a pre-trained neural network model. For example, the inter prediction unit (103) can synthesize a reference frame using a pre-trained neural network model and perform inter prediction based on the synthesized reference frame.

[0063] A residual block containing residual value information, which is the difference between the prediction unit generated in the prediction unit (102, 103) and the original block of the prediction unit, can be generated. The generated residual block can be input to the conversion unit (130) and converted.

[0064] The subtraction unit (104) subtracts the prediction block generated by the intra prediction unit (102) or the inter prediction unit (103) from the block currently to be encoded to generate a residual block of the current block. The residual value (residual block) between the generated prediction block and the original block can be input to the conversion unit (105).

[0065] In addition, the prediction mode information and motion vector information used for prediction can be encoded in the entropy encoding unit (107) along with the residual value and transmitted to the decoder. When a specific encoding mode is used, it is also possible to encode the original block as is and transmit it to the decoder without generating a prediction block through the prediction unit (102, 103).

[0066] The transformation unit (105) can perform a transformation on a residual block containing residual data to generate and output a transformation coefficient. Here, the transformation coefficient may be a coefficient value generated by performing a transformation on the residual block. When a transform skip mode is applied, the transformation unit (105) may skip the transformation on the residual block.

[0067] The conversion unit (105) can determine a conversion type and a conversion kernel based on at least one of encoding parameters such as the size of the conversion block, color component, and prediction mode, and can perform a conversion on the conversion block using the determined conversion type and conversion kernel.

[0068] According to one embodiment, the transformation unit (105) can perform a transformation on the 4x4 luminance residual block generated from the intra prediction result using a transformation type and transformation kernel according to the Discrete Sine Transform (DST), and for the remaining residual block, perform a transformation using a transformation type and transformation kernel according to the Discrete Cosine Transform (DCT).

[0069] According to another embodiment, the transformation unit (105) may apply MTS (Multiple Transform Selection) technology, which performs transformations by selectively using various transformation types and transformation kernels. That is, the transformation unit (105) may perform transformations on a sub-block basis through SBT (Sub-block Transform) technology. Specifically, SBT may be applied only to inter-predicted blocks, and the current block may be divided into ½ or ¼ sizes in the vertical or horizontal direction, and transformation may be performed on only one of the blocks. For example, the transformation unit (105) may perform transformation on the leftmost or rightmost block among the current blocks divided vertically, and perform transformation on the topmost or bottommost block among the current blocks divided horizontally.

[0070] According to another embodiment, the transformation unit (105) may apply a non-separable primary transform (NSPT) technique that performs transformations by selectively using multiple transformation kernels based on an intra-prediction mode or the size and / or shape of the block.

[0071] According to another embodiment, the conversion unit (105) may apply a Low Frequency Non-Separable Transform (LFNST), which is a technique for applying a secondary transform to a residual signal converted into the frequency domain through a DCT or DST. LFNST can additionally perform a transformation on a 4x4 or 8x8 low-frequency region in the upper left to concentrate the residual coefficients to the upper left.

[0072] The quantization unit (106) can quantize the conversion coefficient or residual signal converted into the frequency domain by the conversion unit (105) according to a quantization parameter (QP). The quantization parameter may vary depending on the block or the importance of the image. The value calculated by the quantization unit (106) may be provided to the inverse quantization unit (108) and the entropy encoding unit (107).

[0073] The above-mentioned conversion unit (105) and / or quantization unit (106) may be optionally included in the image encoding device (100). That is, the image encoding device (100) may perform at least one of conversion or quantization on the residual data of the residual block, or may encode the residual block by skipping both conversion and quantization. Even if neither conversion nor quantization is performed in the image encoding device (100), or if neither conversion nor quantization is performed, the block that enters as input to the entropy encoding unit (107) is typically referred to as a conversion block.

[0074] The entropy encoding unit (107) can generate and output a bitstream by performing entropy encoding according to a probability distribution on values ​​output by the quantization unit (106), coding parameter values ​​output during the encoding process, information for decoding an image, etc. Here, the information for decoding an image may include syntax elements, etc.

[0075] Coding parameters may include information (flags, indexes, etc.) that is encoded in the encoding device (100) and signaled to the decoding device (200), such as syntax elements, as well as information derived during the encoding process or decoding process, and may refer to information required when encoding or decoding images.

[0076] The entropy encoding unit (107) can encode various information such as coefficient information of a conversion block, block type information, prediction mode information, division unit information, prediction unit information, transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information. The coefficients of the conversion block can be encoded in units of sub-blocks within the conversion block.

[0077] For encoding the coefficients of a transform block, various syntax elements may be encoded, such as Last_sig, a syntax element indicating the location of the first non-zero coefficient in backscan order; Coded_sub_blk_flag, a flag indicating whether there is at least one non-zero coefficient in the subblock; Sig_coeff_flag, a flag indicating whether it is a non-zero coefficient; Abs_greater1_flag, a flag indicating whether the absolute value of the coefficient is greater than 1; Abs_greater2_flag, a flag indicating whether the absolute value of the coefficient is greater than 2; and Sign_flag, a flag indicating the sign of the coefficient. The remaining values ​​of the coefficients that are not encoded by the above syntax elements alone may be encoded through the syntax element remaining_coeff.

[0078] When entropy encoding is applied, the size of the bit sequence for the symbols to be encoded can be reduced by allocating a small number of bits to symbols with a high probability of occurrence and a large number of bits to symbols with a low probability of occurrence when representing symbols. Input data is entropied. For example, entropy encoding can utilize various encoding methods such as Exponential Golomb and CABAC (Context-Adaptive Binary Arithmetic Coding).

[0079] The inverse quantization unit (108) and the inverse transformation unit (109) can inverse quantize the values ​​quantized in the quantization unit (106) and inverse transform the values ​​transformed in the transformation unit (105). The residual value generated in the inverse quantization unit (108) and the inverse transformation unit (109) can be combined with the prediction unit predicted through the motion estimation unit, motion compensation unit, and intra prediction unit (102) included in the prediction unit (102, 103) to generate a reconstructed block. The addition unit (110) generates a reconstructed block by adding the prediction block generated in the prediction unit (102, 103) and the residual block generated through the inverse transformation unit (109).

[0080] The filter section (111) can apply deblocking filter, Sample Adaptive Offset (SAO), Adaptive Loop Filter (ALF), Bilateral filter (BIF), LMCS (Luma Mapping with Chroma Scaling), etc., to the reconstructed sample, reconstructed block, or reconstructed image as a whole or part of the filtering technique.

[0081] The deblocking filter can remove block distortion caused by boundaries between blocks in the restored picture. To determine whether to perform deblocking, the decision to apply the deblocking filter to the current block can be made based on the pixels contained in a certain number of columns or rows within the block. When applying the deblocking filter to a block, a Strong Filter or a Weak Filter can be applied depending on the required deblocking filtering strength. Additionally, when applying the deblocking filter, horizontal and vertical filtering can be processed in parallel.

[0082] Sample adaptive offset may be a method of correcting the offset from the original image on a sample-by-sample basis for an image that has undergone deblocking. The filter unit (111) may use a method of dividing the samples included in the image into a certain number of regions, determining the region to perform the offset on, and applying the offset to that region, or a method of applying the offset by considering the edge information of each sample. Here, the sample adaptive offset may be at least one of a general sample adaptive offset, a bilateral filter, and a cross-component sample adaptive offset (CCSAO).

[0083] Adaptive Loop Filtering (ALF) can be performed based on a comparison between the filtered restored image and the original image. After dividing the pixels included in the image into predetermined groups, a single filter to be applied to each group can be determined, allowing for differential filtering for each group. Information regarding whether to apply ALF can be transmitted per coding unit (CU), and the shape and filter coefficients of the ALF filter to be applied may vary depending on each block. Additionally, an ALF filter of the same form (fixed form) may be applied regardless of the characteristics of the block to be applied.

[0084] An adaptive loop filter can perform filtering based on a comparison of the reconstructed image and the original image. After dividing the samples included in the image into predetermined groups, a filter to be applied to each group can be determined, thereby performing filtering differently for each group. Information regarding whether to apply an adaptive loop filter can be signaled per coding unit (CU), and the shape and filter coefficients of the adaptive loop filter to be applied may vary depending on each block.

[0085] According to one embodiment, the filter unit (111) can filter all or part of a restored sample, restored block, or restored image using a pre-trained neural network model. Specifically, the filter unit (111) can apply an adaptive loop filter to all or part of a restored sample, restored block, or restored image using a pre-trained neural network model.

[0086] The memory (112) can store a restored block or picture calculated through the filter unit (111). The memory (112) may include a reference picture buffer. Additionally, the stored restored block or picture in the memory (112) may be provided to the prediction unit (102, 103) when performing inter-prediction.

[0087] Next, an image decoding device according to one embodiment of the present invention will be described with reference to the drawings.

[0088] FIG. 2 is a block diagram showing an image decoding device (200) according to one embodiment of the present invention.

[0089] Referring to FIG. 2, the image decoding device (200) may include an entropy decoding unit (201), an inverse quantization unit (202), an inverse transformation unit (203), a prediction unit (204, 205), an adder unit (206), a filter unit (207), and a memory (208).

[0090] The video decoding device (200) can receive a bitstream output by the video encoding device (100). The video decoding device (200) can receive a bitstream stored in a computer-readable recording medium or receive a bitstream stream streamed through a wired / wireless transmission medium. The video decoding device (200) can decode the bitstream to generate a restored video or a decoded video, and can output the restored video or the decoded video.

[0091] The entropy decoding unit (201) can generate symbols by performing entropy decoding according to the probability distribution of the bitstream. The generated symbols may include symbols in the form of quantized levels. Here, the entropy decoding method may be the inverse process of the entropy encoding method described above.

[0092] The entropy decoding unit (201) can convert a one-dimensional vector-shaped coefficient into a two-dimensional block-shaped coefficient through a conversion coefficient scanning method to decode a conversion coefficient level (quantized level).

[0093] The entropy decoding unit (201) can perform entropy decoding in the opposite procedure to that which the entropy encoding unit (107) of the image encoding device (100) performed. For example, various methods such as Exponential Golomb and CABAC (Context-Adaptive Binary Arithmetic Coding) can be applied in correspondence with the method performed in the image encoder.

[0094] The entropy decoding unit (201) can obtain various information such as coefficient information of a conversion block as described above, block type information, prediction mode information, division unit information, prediction unit information, transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information by decoding.

[0095] The inverse quantization unit (202) generates a conversion block by performing inverse quantization on the quantized conversion block. It operates substantially the same as the inverse quantization unit (108) of FIG. 1.

[0096] The inverse transformation unit (203) performs an inverse transformation on the transformation block to generate a residual block. At this time, the transformation method may be determined based on information regarding the prediction method (inter or intra prediction), the size and / or shape of the block, the intra prediction mode, etc. It operates substantially the same as the inverse transformation unit (109) of FIG. 1.

[0097] According to one embodiment, the inverse transformation unit (203) can perform an inverse transformation using a transformation type and a transformation kernel according to the Discrete Sine Transform (DST) on the transformation coefficient level of the 4x4 luminance component generated from the intra prediction result, and perform an inverse transformation using a transformation type and a transformation kernel according to the Discrete Cosine Transform (DCT) on the remaining transformation coefficient level.

[0098] According to another embodiment, the inverse transformation unit (203) may apply MTS (Multiple Transform Selection) technology, which performs transformations by selectively using multiple transformation kernels.

[0099] According to another embodiment, the inverse transform unit (203) may apply LFNST (Low Frequency Non-Separable Transform), which is a technique for applying a secondary inverse transform to the inverse transform coefficient level through a DCT or DST-based transform type and a transform kernel.

[0100] According to another embodiment, the inverse transform unit (203) may apply a non-separable primary transform (NSPT) technique that performs the inverse transform by selectively using a transform kernel based on an intra-prediction mode or the size and / or shape of the block.

[0101] The prediction unit (204, 205) can generate a prediction block based on the prediction block generation information provided by the entropy decoding unit (201) and the previously decoded block or picture information provided by the memory (208).

[0102] The prediction unit (204, 205) may include an intra prediction unit (204) and an inter prediction unit (205). The prediction unit (204, 205) receives various information, such as prediction unit information input from the entropy decoding unit (201), prediction mode information of the intra prediction method, and motion prediction related information of the inter prediction method, distinguishes the prediction unit from the current encoding unit, and determines the prediction mode of the prediction unit.

[0103] The intra prediction unit (204) can generate a prediction block of the current block based on the intra prediction mode of the current block and reference pixel information around the current block, which is pixel information within the current picture.

[0104] The intra prediction unit (204) can generate a prediction block based on reference pixel information around the current block, which is pixel information within the current picture. The intra prediction unit (204) can use multiple reference pixel lines for intra prediction. If multiple reference pixel lines are available, the intra prediction unit (204) can obtain information indicating a reference pixel line used for intra prediction among the multiple reference pixel lines.

[0105] The intra prediction mode used for intra prediction may be a directional prediction mode or a non-directional mode. Additionally, the mode for predicting luminance information and the mode for predicting chrominance information may be different, and the intra prediction mode information of the luminance component block or the predicted luminance signal information may be utilized to predict chrominance information.

[0106] According to one embodiment, the intra prediction unit (204) can perform intra prediction using a pre-trained neural network model. The intra prediction unit (204) can perform intra prediction using the same neural network model as the intra prediction unit (102) of FIG. 1.

[0107] The intra prediction unit (204) operates substantially the same as the intra prediction unit (102) of FIG. 1.

[0108] The inter prediction unit (205) can perform an inter prediction for the current prediction unit based on information included in at least one of the previous or subsequent pictures of the current picture containing the current prediction unit, using information necessary for inter prediction of the current prediction unit provided by the video encoding device (100). Alternatively, the inter prediction may be performed based on information of a partially restored area within the current picture containing the current prediction unit.

[0109] The inter prediction unit (205) can set the inter mode of the prediction unit included in the encoding unit to one of Skip Mode, Merge Mode, or Advanced Motion Vector Prediction (AMVP) Mode in order to perform motion prediction and / or motion compensation. Additionally, the inter prediction unit (205) can perform motion compensation for the prediction unit according to the set mode.

[0110] Additionally, the inter prediction unit (205) can perform motion compensation for the prediction unit by applying the AFFINE mode of sub-PU-based prediction, the SbTMVP (Subblock-based Temporal Motion Vector Prediction) mode, and the MMVD (Merge with MVD) mode and GPM (Geometric Partitioning Mode) mode of PU-based prediction based on the inter prediction mode. Additionally, the inter prediction unit (205) can perform motion compensation for the prediction unit by applying HMVP (History based MVP), PAMVP (Pairwise Average MVP), CIIP (Combined Intra / Inter Prediction), AMVR (Adaptive Motion Vector Resolution), BDOF (Bi-Directional Optical-Flow), BCW (Bi-predictive with CU Weights), LIC (Local Illumination Compensation), TM (Template Matching), OBMC (Overlapped Block Motion Compensation), etc., to improve the performance of each mode.

[0111] Motion information may include, for example, motion vectors, reference picture indices, List 1 prediction flags, List 0 prediction flags, half-sample interpolation filter indices, bidirectional prediction weight indices, etc.

[0112] According to one embodiment, the inter prediction unit (205) can perform inter prediction using a pre-trained neural network model. The inter prediction unit (205) can perform inter prediction using the same neural network model as the inter prediction unit (103) of FIG. 1.

[0113] The inter prediction unit (205) can operate substantially the same as the inter prediction unit (103) of FIG. 1.

[0114] The adder (206) generates a restoration block by adding the prediction block generated in the intra prediction unit (204) or the inter prediction unit (205) and the residual block generated through the inverse transformation unit (203). It operates substantially the same as the adder (110) of FIG. 1.

[0115] The filter section (207) can reduce various types of noise occurring in the restored blocks. The filter section (207) may include a deblocking filter, a sample adaptive offset, an adaptive loop filter, a bidirectional filter, and an LMCS.

[0116] The filter unit (207) may receive information regarding whether each filter is applied, information regarding the filter strength, etc. from the image encoding device (100). The filter unit (207) of the image decoding device (200) may receive filter-related information provided by the image encoding device (100) and perform filtering on the corresponding block in the image decoding device (200).

[0117] According to one embodiment, the filter unit (207) can filter all or part of a restored sample, restored block, or restored image using a pre-trained neural network model. Specifically, the filter unit (207) can apply an adaptive loop filter to all or part of a restored sample, restored block, or restored image using a pre-trained neural network model.

[0118] The filter section (207) can operate substantially the same as the filter section (111) of FIG. 1.

[0119] The memory (208) can store a restoration block generated by the adder (206). For example, the memory (208) may include a reference picture buffer. The memory (208) may operate substantially the same as the memory (112) of FIG. 1.

[0120]

[0121] FIG. 3 is a schematic diagram showing a video coding system to which the present invention can be applied.

[0122] A video coding system according to one embodiment may include an encoding device (10) and a decoding device (20). The encoding device (10) may transmit encoded video and / or image information or data to the decoding device (20) via a digital storage medium or network in the form of a file or streaming.

[0123] An encoding device (10) according to one embodiment may include an image generation unit (11), an encoding unit (12), and a transmission unit (13). A decoding device (20) according to one embodiment may include a receiving unit (21), a decoding unit (22), and an image playback unit (23). The encoding unit (12) may be called a video / image encoding unit, and the decoding unit (22) may be called a video / image decoding unit. The transmission unit (13) may be included in the encoding unit (12). The receiving unit (21) may be included in the decoding unit (22). The image playback unit (23) may include a display unit, and the display unit may be composed of a separate device or an external component.

[0124] The image generation unit (11) can acquire video / image through a process of capturing, synthesizing, or generating video / image. The image generation unit (11) may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, a video / image archive containing previously captured video / image, etc. The video / image generation device may include, for example, a computer, a tablet, and a smartphone, etc., and can generate video / image (electronically). For example, a virtual video / image may be generated through a computer, etc., in which case the video / image capture process may be replaced by a process of generating related data.

[0125] The encoding unit (12) can encode the input video / image. The encoding unit (12) can perform a series of procedures such as prediction, conversion, and quantization for compression and encoding efficiency. The encoding unit (12) can output the encoded data (encoded video / image information) in the form of a bitstream. The detailed configuration of the encoding unit (12) can be configured in the same way as the encoding device (100) of FIG. 1 described above.

[0126] The transmission unit (13) can transmit encoded video / image information or data output in the form of a bitstream to the receiving unit (21) of the decoding device (20) via a digital storage medium or network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmission unit (13) may include elements for creating a media file through a predetermined file format and elements for transmission via a broadcasting / communication network. The receiving unit (21) can extract / receive the bitstream from the storage medium or network and transmit it to the decoding unit (22).

[0127] The decoding unit (22) can decode a video / image by performing a series of procedures such as inverse quantization, inverse transformation, and prediction corresponding to the operation of the encoding unit (12). The detailed configuration of the decoding unit (22) can be configured in the same way as the decoding device (200) of FIG. 2 described above.

[0128] The image playback unit (23) can render the decoded video / image. The rendered video / image can be displayed through the display unit.

[0129]

[0130] FIG. 4 is a diagram illustrating an exemplary content streaming system to which an embodiment according to the present invention can be applied.

[0131] As illustrated in FIG. 4, a content streaming system to which an embodiment of the present invention is applied may largely include a multimedia input device, a media storage, an encoding server, a streaming server, a web server, and a user device.

[0132] The above encoding server plays the role of compressing content input from multimedia input devices, such as smartphones, cameras, and CCTVs, into digital data to generate a bitstream and transmitting it to the above streaming server. Alternatively, the encoding server plays the role of compressing content previously stored in a media storage into digital data to generate a bitstream and transmitting it to the above streaming server.

[0133] As another example, when multimedia input devices such as smartphones, cameras, and CCTVs directly generate bitstreams, the encoding server may be omitted.

[0134] The bitstream above may be generated by a video encoding method and / or video encoding device to which an embodiment of the present invention is applied, and the streaming server may temporarily or non-temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[0135] The streaming server transmits multimedia data to a user device based on a user request through a web server, and the web server can act as a medium to inform the user of available services. When a user device requests a desired service from the web server, the web server forwards it to the streaming server, and the streaming server can transmit multimedia data to the user device. At this time, the content streaming system may include a separate control server, and in this case, the control server can perform the role of controlling commands and responses between each device within the content streaming system.

[0136] The streaming server can receive content from a media storage and / or an encoding server. For example, when receiving content from the encoding server, the content can be received in real time. In this case, to provide a seamless streaming service, the streaming server can store the bitstream for a certain period of time.

[0137] Examples of the above user devices may include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, HMDs (head-mounted displays)), digital TVs, desktop computers, digital signage, etc.

[0138] Each server within the above-mentioned content streaming system can be operated as a distributed server, and in this case, data received from each server can be processed in a distributed manner.

[0139]

[0140] Video encoding and decoding methods are implemented through a hybrid compression framework based on prediction methods, transformation methods, and entropy coding methods. In particular, probability-based arithmetic coding methods have been used to eliminate redundancy in the symbols to be coded. Generally, the arithmetic coding method used in codec technology is binary arithmetic coding that uses only 0 and 1. Specifically, the binary arithmetic coding used in codec technology may be CABAC (Context Adaptive Binary Arithmetic Coding), which is adaptive binary arithmetic coding that adaptively sets the probability model according to the surrounding situation to improve compression performance. The CABAC encoding structure and CABAC decoding structure, respectively, may be as described below.

[0141]

[0142] Figure 5 is a diagram showing a CABAC encoding structure for encoding a single syntax element.

[0143] Referring to FIG. 5, the CABAC encoder may include a binarizer and a binary arithmetic coder. The binary arithmetic coder may include a regular coding engine and a bypass coding engine. Additionally, the CABAC encoder may further include a context modeler for updating the context model of the regular coding engine.

[0144] The CABAC encoding procedure may include a binary conversion procedure and a binary arithmetic encoding procedure. When a signal that is a syntax element rather than a binary value is input, the input signal may be converted into a binary value through binary conversion. If the input signal is already a binary value, it may be bypassed without undergoing binary conversion. Here, each binary digit 0 or 1 constituting the binary value is called a bin. And, the bin(s) for a single syntax element may represent the value of that syntax element.

[0145] In the binary arithmetic coding procedure, the binarized bins can be input into a regular coding engine or a bypass coding engine. In the CABAC coding procedure, the CABAC encoder determines whether to perform encoding through the regular coding engine or the bypass coding engine, and can switch the coding path.

[0146] A regular coding engine assigns a context model that reflects probability values ​​to binarized bins and can encode the binarized bins based on a probability model corresponding to the assigned context model. Furthermore, the regular coding engine, having performed coding for each bin, can update the probability model for that bin. Bins coded by the regular coding engine are called context-coded bins.

[0147] The context model can be assigned and updated per context-coded (normally coded) bean, and the context model can be indicated based on the context index (ctxidx) or the context index increment (ctxInc). ctxidx can be derived based on ctxInc. Specifically, for example, the context index pointing to the context model for each normally coded bean can be derived as the sum of the context index increment (ctxInc) and the context index offset (ctxIdxOffset). Here, ctxInc can be derived differently for each bean. ctxIdxOffset can be represented as the lowest value of ctxIdx. ctxIdxOffset can generally be determined by the slice type, and the context model for a single syntax element within the slice can be distinguished / derived based on ctxinc.

[0148] The bypass coding engine skips the procedure of estimating the probability for the input bin and the procedure of updating the probability model applied to the bin after coding. Instead of assigning context to the input bin, the bypass coding engine improves coding speed by coding the input bin by applying a uniform probability distribution (e.g., 50:50). Bins coded in this way are called bypass bins.

[0149]

[0150] Figure 6 is a diagram showing a CABAC decoding structure for decoding a single syntax element.

[0151] Referring to FIG. 6, the CABAC decoder may include a binary arithmetic coder and an inverse-binarizer. The binary arithmetic coder may include a regular coding engine and a bypass coding engine. Additionally, the CABAC decoder may further include a context modeler for updating the context model of the regular coding engine.

[0152] The CABAC decoding procedure may include a binary arithmetic decoding procedure and a debinary decoding procedure. In the CABAC decoding procedure, it is determined whether to perform decoding through a regular coding engine or a bypass coding engine, and the coding path can be switched.

[0153] The input signal being decoded by CABAC may be a binarized bin of a bitstream. In the binary arithmetic decoding procedure, the binarized bins may be input to a regular coding engine or a bypass coding engine. The regular coding engine may assign a context model that reflects the probability value for the corresponding bin and decode the corresponding bin based on a probability model corresponding to the assigned context model. After performing decoding for each bin, the regular coding engine may update the probability model for that bin. The context model may be assigned and updated per context-coded (regularly coded) bin, and the context model may be indicated based on ctxidx or ctxInc. ctxidx may be derived based on ctxInc.

[0154] The bypass coding engine skips the procedure of estimating the probability for the input bin and the procedure of updating the probability model applied to the bin after coding. Instead of allocating context, it improves coding speed by decoding the input bin by applying a uniform probability distribution (e.g., 50:50).

[0155]

[0156] Since the adoption of CABAC in the AVC / H.264 standard, entropy coding and decoding methods utilizing CABAC have been used in video encoding and decoding standards. In particular, CABAC demonstrates a higher compression ratio compared to Hoffman coding, a conventional entropy coding method. Consequently, CABAC is used exclusively as an entropy coding method in the HEVC / H.265 and VVC / H266 standards.

[0157] However, CABAC has the disadvantage of lower throughput compared to Hoffman coding. In particular, when encoding and / or decoding high-resolution images, the low throughput of the encoding and decoding operations can be a problem.

[0158] To improve the throughput of encoding and decoding operations, a bypass coding engine may be used instead of a regular coding engine. However, using a bypass coding engine instead of a regular coding engine may result in a decrease in compression ratio. Therefore, the present disclosure can provide a low-complexity CABAC method and apparatus to improve throughput while maintaining the advantages of CABAC.

[0159] The low-complexity CABAC encoding structure and the low-complexity CABAC decoding structure, respectively, may be as described below.

[0160]

[0161] FIG. 7 is a diagram showing a low-complexity CABAC encoding structure for encoding a single syntax element according to one embodiment of the present disclosure.

[0162] Referring to FIG. 7, the low-complexity CABAC encoder may include a binarizer and a binary arithmetic coder. The binary arithmetic coder may include a regular coding engine, a simplified regular coding engine, and a bypass coding engine. Additionally, the CABAC encoder may further include a context modeler for updating the context model of the regular coding engine.

[0163] The CABAC encoding procedure may include a binary conversion procedure and a binary arithmetic encoding procedure. In the binary arithmetic encoding procedure, the binary converted bins may be input to a regular coding engine, a simplified regular coding engine, or a bypass coding engine of a binary arithmetic coder.

[0164] The regular coding engine assigns a context model that reflects the probability value for the corresponding bin, and can encode the corresponding bin based on the probability model corresponding to the assigned context model. Additionally, the regular coding engine that has performed coding for each bin can update the probability model for that bin.

[0165] A simplified regular coding engine can assign a context model that reflects the probability value for the corresponding bin and encode the bin based on a probability model corresponding to the assigned context model. Here, the simplified regular coding engine can encode the bin by simplifying the context model selection procedure or the probability model update procedure.

[0166] The bypass coding engine skips the procedure of estimating the probability for the input bin and the procedure of updating the probability model applied to the bin after coding. It improves coding speed by encoding the input bin by applying a uniform probability distribution (e.g., 50:50) instead of assigning context.

[0167] In the CABAC encoding procedure, the low-complexity CABAC encoder determines which coding engine to use to perform encoding among the regular coding engine, the simplified regular coding engine, and the bypass coding engine, and can switch the coding path.

[0168]

[0169] FIG. 8 is a diagram showing a low-complexity CABAC decoding structure for decoding a single syntax element according to one embodiment of the present disclosure.

[0170] Referring to FIG. 8, the CABAC decoder may include a binary arithmetic decoder and an inverse binary decoder. The binary arithmetic decoder may include a regular coding engine, a simplified regular coding engine, and a bypass coding engine. Additionally, the CABAC decoder may further include a context modeler for updating the context model of the regular coding engine.

[0171] The CABAC decoding procedure may include a binary arithmetic decoding procedure and a debinary decoding procedure. The low-complexity CABAC decoder determines which coding engine to use to perform decoding among a regular coding engine, a simplified regular coding engine, and a bypass coding engine, and can switch the coding path.

[0172] In the binary arithmetic decoding procedure, the binarized bins can be input into a regular coding engine, a simplified regular coding engine, or a bypass coding engine.

[0173] The regular coding engine assigns a context model that reflects the probability value for the corresponding bean, and can decode the bean based on the probability model corresponding to the assigned context model. Additionally, the regular coding engine that has performed coding for each bean can update the probability model for that bean.

[0174] A simplified normalized coding engine can assign a context model that reflects the probability value for the corresponding bin and encode the bin based on a probability model corresponding to the assigned context model. Here, the simplified normalized coding engine can perform coding for the bin by simplifying the context model selection procedure or the probability model update procedure.

[0175] The bypass coding engine skips the procedure of estimating the probability for the input bin and the procedure of updating the probability model applied to the bin after coding. Instead of allocating context, it improves coding speed by coding the input bin by applying a uniform probability distribution (e.g., 50:50). Bins coded in this way are called bypass bins.

[0176] When a signal that is a syntax element rather than a binary value is input, the input signal can be converted into a binary value through binarization. If the input signal is already a binary value, it can be bypassed without undergoing binarization.

[0177]

[0178] A low-complexity CABAC encoder can perform encoding by simplifying operations requiring high throughput within the standard coding procedure. Additionally, a low-complexity CABAC decoder can perform decoding by simplifying operations requiring high throughput within the standard coding procedure. Here, the method for simplifying operations requiring high throughput within the standard coding procedure may be as follows.

[0179]

[0180] One of the procedures that directly affects the throughput of CABAC may be the procedure for updating the probabilistic model. In particular, a significant delay may occur during the procedure for updating the probabilistic model. Accordingly, according to one embodiment of the present disclosure, the probabilistic model in the CABAC procedure can be updated adaptively.

[0181] According to one embodiment of the present disclosure, a binary arithmetic decoder may include a bypass coding engine and a regular coding engine. Here, the binary arithmetic decoder may determine whether to update a probability model based on additional information transmitted along with a bin. If it is determined based on the additional information that the probability model is not to be updated, the binary arithmetic decoder may decode the syntax using the bypass coding engine instead of the regular coding engine.

[0182] Alternatively, according to one embodiment of the present disclosure, the binary arithmetic decoder of a low-complexity CABAC decoder may include a bypass coding engine, a regular coding engine, and a simplified regular coding engine. Here, unlike the regular coding engine, the simplified regular coding engine may not update the probabilistic model in the CABAC decoding procedure. Accordingly, unlike the regular coding engine, the simplified regular coding engine may not require a feedback procedure from a context modeler.

[0183] Meanwhile, according to another embodiment of the present disclosure, the binary arithmetic decoder may include a bypass coding engine and a regular coding engine. Here, the regular coding engine may determine whether to update the probability model based on additional information transmitted along with the bin. If it is determined not to update the probability model based on the additional information, the regular coding engine may not perform a probability update by blocking information fed back from the context modeler or by maintaining the probability model as it is.

[0184]

[0185] According to one embodiment of the present disclosure, the probability model used in the CABAC procedure can be adaptively updated according to the syntax.

[0186] When decoding syntax that requires high throughput in the CABAC procedure, the decoder may not update the probabilistic model. That is, the decoder performs regular coding using the probabilistic model determined from the initialization result, but may omit updating the probabilistic model. Consequently, the complexity of CABAC decoding can be reduced, and the throughput of syntax decoding can be increased.

[0187] For example, when decoding a syntax that uses many types of context models in a decoding procedure, the decoder may not update the probabilistic model. Here, the syntax that uses many types of context models in the decoding procedure may be a syntax consisting of a predetermined number of bins or more. Alternatively, the syntax that uses many types of context models in the decoding procedure may be an experimentally determined syntax. The encoder and the decoder may share information regarding the experimentally determined syntax. Therefore, the encoder and the decoder can decide whether to update the probabilistic model based on the information regarding the syntax.

[0188] Alternatively, an encoder that encodes syntax using many types of context models may transmit information indicating whether to update the probabilistic model to a decoder for each predefined unit. Then, the decoder may determine whether to update the probabilistic model based on the information received from the encoder. Here, the predefined units may be a sequence, a group of picture (GOP), an intra-frame period, a picture, a slice, a tile, a CTU, a CU, a TU, and a context initialization unit, etc.

[0189] According to one embodiment of the present disclosure, the probability model used in the CABAC procedure can be adaptively updated according to the bin.

[0190] In the CABAC procedure, the throughput of decoding for a bin may vary depending on the bin's transfer characteristics. Therefore, when decoding a bin expected to have a high throughput, the decoder may not update the probabilistic model. That is, the decoder performs regular coding using the probabilistic model determined from the initialization result, but may omit updating the probabilistic model. Consequently, the complexity of CABAC decoding can be reduced, and the throughput of syntax decoding can be increased.

[0191] For example, when decoding bins that use many types of context models during the decoding process, the decoder may not update the probabilistic model. Alternatively, the probabilistic model for a bin may vary significantly during the decoding process of consecutive bins. Therefore, when decoding consecutively transmitted bins, the decoder may not update the probabilistic model. Or, if it is determined that there is a large difference between the probability for a bin predicted by context modeling and the probability for the actual bin, the decoder may not update the probabilistic model.

[0192] Alternatively, the number of bins encoded by the regular coding engine may be pre-set to a specific number. And, if the decoder decodes more bins than the pre-set number, the decoder may not update the probabilistic model. Here, the encoder may transmit information regarding the pre-set number of bins to the decoder. And, the decoder may decide whether to update the probabilistic model based on the information received from the encoder. Here, the information regarding the pre-set number of bins may be transmitted according to a predetermined unit. And, the pre-defined unit may be a sequence, GOP, intra-frame period, picture, slice, tile, CTU, CU, TU, and context initialization unit, etc.

[0193] Alternatively, according to one embodiment of the present disclosure, whether to update the probabilistic model in a CABAC procedure for syntax may be used adaptively based on various information.

[0194] For example, in the CABAC procedure for the syntax of the current block, whether to update the probabilistic model can be adaptively determined based on the value of the syntax of the adjacent block.

[0195] Alternatively, in the CABAC procedure for the syntax of the current block, whether to update the probabilistic model may be adaptively determined based on at least one of the size, location, and prediction mode of the current block.

[0196] Alternatively, if the current block is a chrominance block and is predicted by a cross-component model, in the CABAC procedure for the syntax of the current block, whether to update the probabilistic model can be adaptively determined based on the information of the luminance block corresponding to the current block.

[0197] Here, the decoding operation that reduces the complexity of CABAC by omitting the probabilistic model update can be performed by the simplified regular coding engine of the binary arithmetic decoder. Alternatively, the decoding operation that reduces the complexity of CABAC by omitting the probabilistic model update can be optionally performed by the regular coding engine of the binary arithmetic decoder.

[0198] In addition, by omitting the previously described probabilistic model update, operations identical or equivalent to the decoding operation that reduces the complexity of CABAC can be performed by the encoder.

[0199]

[0200] In addition, to improve the throughput of CABAC, according to one embodiment of the present disclosure, the decoder may adaptively configure and use a probabilistic model and / or a context model in an initialization procedure during the CABAC procedure. According to one embodiment of the present disclosure, the initialization procedure in the CABAC procedure may be performed as follows.

[0201] Specifically, the initial probability values ​​used in the CABAC procedure may be probability values ​​pre-set from the training results. In this case, the decoder may perform the CABAC procedure without updating the initial probability values. For example, the initial probability values ​​may be set to equal probabilities such as 0.5(MPS):0.5(LPS). Or, the initial probability values ​​may be set to a(MPS):1-a(LPS). Here, a may be a value between 0 and 1.

[0202] When decoding syntax or bins using one or more context models, the decoder may use a probability value preset from the training results as the initial probability value. The decoder may update the probability model until a predetermined number of units of syntax or bins have been decoded. The decoder may not update the probability model when decoding subsequent units of syntax or bins. Alternatively, the decoder may optionally update the probability model when decoding subsequent units of syntax or bins. Here, the predefined units may be sequences, GOPs, intra-frame periods, pictures, slices, tiles, CTUs, CUs, TUs, and context initialization units, etc.

[0203] For example, the decoder may perform a probabilistic model update during the procedure of decoding syntax or bins for 10 CTUs. And, the decoder may not update the probabilistic model when decoding subsequent syntax or bins.

[0204]

[0205] An image decoding method and an image encoding method for a low-complexity CABAC method according to one embodiment of the present disclosure may be as described below.

[0206]

[0207] FIG. 9 is a flowchart illustrating an image decoding method for a low-complexity CABAC method according to one embodiment of the present disclosure. The image decoding method of FIG. 9 can be performed by an image decoding device.

[0208] Referring to FIG. 9, the image decoder can obtain a bitstream containing encoded syntax elements (S910).

[0209] The video decoder can determine a context model for encoded syntax elements (S920).

[0210] The video decoder can decode encoded syntactic elements based on probability values ​​derived by a context model (S930).

[0211] The video decoder can determine whether to update the probability value (S940).

[0212] And, the image decoder can update the probability value of the context model based on whether the probability value is updated (S950).

[0213] Here, whether to update the probabilistic model may be determined based on the syntax elements. Alternatively, whether to update the probabilistic model may be determined based on the number of bins of the decoded syntax elements. Alternatively, whether to update the probabilistic model may be determined based on the continuity of the bins of the decoded syntax elements.

[0214] Here, whether the probability model is updated can be determined based on a preset prediction unit. For example, if a predetermined number or more of the syntax elements of the preset prediction unit are decoded, the probability model may not be updated.

[0215]

[0216] FIG. 10 is a flowchart illustrating an image encoding method for a low-complexity CABAC method according to one embodiment of the present disclosure. The image encoding method of FIG. 10 can be performed by an image encoding device.

[0217] Referring to FIG. 10, the image encoding device can binarize syntax elements (S1010).

[0218] The video encoding device can determine a context model for the binarized syntax elements (S1020).

[0219] The video encoding device can encode binary syntactic elements based on probability values ​​derived by a context model (S1030).

[0220] The video encoding device can determine whether to update the probability value (S1040).

[0221] And, the video encoding device can update the probability value of the context model based on whether the probability value is updated (S1050).

[0222] Here, whether to update the probabilistic model may be determined based on the syntax elements. Alternatively, whether to update the probabilistic model may be determined based on the number of bins of the decoded syntax elements. Alternatively, whether to update the probabilistic model may be determined based on the continuity of the bins of the decoded syntax elements.

[0223] Here, whether the probability model is updated can be determined based on a preset prediction unit. For example, if a predetermined number or more of the syntax elements of the preset prediction unit are decoded, the probability model may not be updated.

[0224]

[0225] Additionally, a bitstream may be generated by a video encoding method including the steps described in FIG. 10. The bitstream may be stored on a non-transient computer-readable recording medium and may also be transmitted (or streamed).

[0226]

[0227] The exemplary methods of the present disclosure are described as a series of operations for clarity of description, but this is not intended to limit the order in which the steps are performed, and if necessary, each step may be performed simultaneously or in a different order. To implement the method according to the present disclosure, additional steps may be included in addition to the steps exemplified, steps excluding some steps and including the remaining steps, or steps excluding some steps and including additional steps.

[0228] The various embodiments of the present disclosure are not intended to list all possible combinations but to describe representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combination of two or more.

[0229] Various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of implementation by hardware, it may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general processors, controllers, microcontrollers, microprocessors, etc.

[0230] Alternatively, various embodiments of the present disclosure may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. And, a bitstream generated by the encoding method according to the embodiment may be stored on a non-transient computer-readable recording medium.

[0231] The above-mentioned computer-readable recording medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the above-mentioned computer-readable recording medium may be those specifically designed and configured for the present disclosure, or they may be those known and available to those skilled in the art of computer software.

[0232] In the foregoing, the present disclosure is described based on specific details, such as specific components, limited embodiments, and drawings. However, the embodiments of the present disclosure are provided merely to aid in the overall understanding of the present disclosure and are not intended to limit the present disclosure to the above embodiments. Accordingly, a person skilled in the art can make various modifications and variations from this description.

[0233] Accordingly, the scope of the present invention should not be limited to the embodiments described above, and all modifications equivalent to or equivalent to the claims set forth below, as well as the claims described below, shall be considered to fall within the scope of the concept of the present invention.

[0234] The present invention can be used in a device for encoding an image, a device for decoding an image, and a recording medium for storing a bitstream.

Claims

1. In a video decoding method, A step of obtaining a bitstream containing encoded syntax elements; A step of determining a context model for the above-mentioned encoded syntactic elements; A step of decoding the encoded syntactic elements based on probability values ​​derived by the above context model; and A step of determining whether to update the above probability value; and A video decoding method characterized by including a step of updating the probability value of a context model based on whether the above probability value is updated.

2. In Paragraph 1, Whether the above probability value is updated is, A video decoding method characterized by being determined based on the above-mentioned syntax elements.

3. In Paragraph 1, Whether the above probability value is updated is, An image decoding method characterized by being determined based on the number of bins of the syntax elements to be decoded.

4. In Paragraph 1, Whether the above probability value is updated is, An image decoding method characterized by being determined based on the continuity of bins of the syntactic elements being decoded.

5. In Paragraph 1, Whether the above probability value is updated is, A video decoding method characterized by being determined according to a preset prediction unit.

6. In Paragraph 5, Whether the above probability value is updated is, An image decoding method in which, when a predetermined number or more of syntax elements of a preset prediction unit are decoded, the probability value is determined not to be updated.

7. In a video encoding method, Step of binarizing syntactic elements; A step of determining a context model for the above-mentioned binary syntactic elements; A step of encoding the binary syntactic elements based on probability values ​​derived by the above context model; A step of determining whether to update the above probability value; and A video encoding method characterized by including a step of updating the probability value of a context model based on whether the above probability value is updated.

8. A non-transient computer-readable recording medium storing a bitstream generated by a video encoding method, The above image encoding method is, Step of binarizing syntactic elements; A step of determining a context model for the above-mentioned binary syntactic elements; A step of encoding the binary syntactic elements based on probability values ​​derived by the above context model; A step of determining whether to update the above probability value; and A non-transient computer-readable recording medium characterized by including a step of updating the probability value of a context model based on whether the above probability value is updated.

9. A method for transmitting a bitstream generated by a video encoding method, The above transmission method includes the step of transmitting the bitstream, and The above image encoding method is, Step of binarizing syntactic elements; A step of determining a context model for the above-mentioned binary syntactic elements; A step of encoding the binary syntactic elements based on probability values ​​derived by the above context model; A step of determining whether to update the above probability value; and A transmission method characterized by including a step of updating the probability value of a context model based on whether the above probability value is updated.

Images