Method and apparatus for coding transform coefficient in video / image coding system
By using multiple rice parameters to find tables and export binary strings in the video decoding method, the problem of inefficient video encoding in the prior art is solved, and higher encoding efficiency and lower transmission and storage costs are achieved.
Patent Information
- Application Number
- JP2025028026
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2019-09-20
- Filing Date
- 2025-02-25
- Publication Date
- 2025-05-02
- Estimated Expiration
- 2040-08-31
AI Technical Summary
The prior art is difficult to effectively improve video encoding efficiency, especially when processing high-resolution and high-quality video data, the transmission and storage costs are high, and at the same time, the efficiency is inefficient in image video encoding with different characteristics such as virtual reality, virtual reality and holograms.
By using multiple rice parameter lookup tables in the video decoding method, the rice parameters are derived based on the order values of the transform coefficients, and the binary string is derived through these parameters, and the order values of the transform coefficients are finally obtained. This method improves the efficiency of transform coefficient-order coding.
Improve the overall efficiency of video encoding, especially in high-resolution and high-quality video data processing, reduce transmission and storage costs, and show higher efficiency in image video encoding with different characteristics such as virtual reality, virtual reality and holograms.
Smart Images

Figure 2025071300000001_ABST
Abstract
Description
[Technical field]
[0001] The present technology relates to a method and apparatus for coding transform coefficients in encoding / decoding video / images. [Background technology]
[0002] In recent years, the demand for high-resolution, high-quality images / videos such as 4K or 8K or higher UHD (Ultra High Definition) images / videos is increasing in various fields. As the image / video data has higher resolution and quality, the amount of information or bits to be transmitted increases relatively compared to existing image / video data, so that when image data is transmitted using a medium such as an existing wired or wireless broadband line or image / video data is stored using an existing storage medium, the transmission cost and storage cost increase.
[0003] In addition, in recent years, interest in and demand for immersive media such as virtual reality (VR), artificial reality (AR) content, and holograms has been increasing, and the broadcast of images / videos having different image characteristics from real images, such as game images, has been increasing.
[0004] Therefore, there is a demand for a highly efficient image / video compression technology to effectively compress and transmit, store, and play back high-resolution, high-quality image / video information having the above-mentioned various characteristics. Summary of the Invention [Problem to be solved by the invention]
[0005] The technical problem of this document is to provide a method and apparatus for increasing video / image coding efficiency.
[0006] Another technical problem of this document is to provide a method and apparatus for improving the efficiency of residual coding.
[0007] Another technical object of this document is to provide a method and apparatus capable of improving the coding performance of level coding for transform coefficients in residual coding. [Means for solving the problem]
[0008] According to one embodiment of this document, a video decoding method performed by a decoding device includes a step of obtaining information indicating a level value of a transform coefficient in a current block from a bitstream; a step of selecting one of a plurality of Rice parameter look-up tables for the information indicating the level value of the transform coefficient; a step of deriving a Rice parameter for the information indicating the level value of the transform coefficient based on the selected Rice parameter look-up table; a step of deriving a bin string for the information indicating the level value of the transform coefficient based on the Rice parameter; and a step of deriving the level value of the transform coefficient based on the bin string.
[0009] According to another embodiment of the present document, a video decoding method performed by a decoding device includes the steps of obtaining information indicating level values of transform coefficients in a current block from a bitstream, determining an index value of a Rice parameter lookup table for the information indicating the level values of the transform coefficients, deriving a Rice parameter for the information indicating the level values of the transform coefficients from the Rice parameter lookup table based on the index value, deriving a bin string for the information indicating the level values of the transform coefficients based on the Rice parameter, and deriving the level values of the transform coefficients based on the bin string. Effect of the Invention
[0010] According to an embodiment of the present document, the overall video / image compression efficiency may be improved.
[0011] According to one embodiment of this document, the efficiency of residual coding can be improved.
[0012] According to one embodiment of this document, the coding performance of level coding for transform coefficients can be improved by residual coding.
[0013] According to one embodiment of this document, when the level values of the transform coefficients are a mixture of low and high level values, higher performance can be provided in lossless or high bit rate environments (low QP) having relatively high level values. [Brief description of the drawings]
[0014] [Figure 1] 1 illustrates generally an example of a video / image coding system in which embodiments of the present document can be applied; [Diagram 2] FIG. 2 is a diagram illustrating the configuration of a video / image encoding device to which the embodiments of this document can be applied. [Diagram 3] 1 is a diagram illustrating the configuration of a video / image decoding device to which an embodiment of this document can be applied. [Figure 4] Illustrates an exemplary Context-Adaptive Binary Arithmetic Coding (CABAC) for encoding syntax elements. [Diagram 5] Transform coefficients in a 4x4 block are shown as an example. [Figure 6] 1 illustrates a schematic diagram of an example entropy encoding method and associated components according to an embodiment of the present document. [Figure 7] 1 illustrates a schematic diagram of an example entropy encoding method and associated components according to an embodiment of the present document. [Figure 8] 1 illustrates a schematic diagram of an example of an entropy encoding method according to another embodiment of the present document; [Figure 9] 1 illustrates a schematic diagram of an example of an entropy decoding method and associated components according to an embodiment of the present document; [Figure 10] 1 illustrates a schematic diagram of an example of an entropy decoding method and associated components according to an embodiment of the present document; [Figure 11] 1 illustrates a schematic diagram of an example of an entropy decoding method according to another embodiment of the present document; [Figure 12] 1 illustrates a video / image encoding method according to an embodiment of the present document. [Figure 13] 1 illustrates a video / image decoding method according to an embodiment of the present document. [Figure 14] 1 illustrates an example of a content streaming system in which the embodiments disclosed herein can be applied. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0015] Since the disclosure of this document can be modified in various ways and can have various embodiments, a specific embodiment will be illustrated in the drawings and described in detail. The terms used in this document are merely used to describe a specific embodiment and are not intended to limit the technical ideas of this document. A singular expression includes the expression "at least one" unless the context clearly indicates otherwise. In this document, the terms "include" or "have" are intended to specify the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, and should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.
[0016] Meanwhile, each component in the drawings described in this document is illustrated independently for the convenience of describing different characteristic functions, and does not mean that each component is realized by separate hardware or software. For example, two or more components among the components may be combined to form one component, or one component may be divided into multiple components. An embodiment in which each component is integrated and / or separated is also included in the disclosure scope of this document as long as it does not deviate from the essence of the method disclosed in this document.
[0017] Hereinafter, the embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. In the following, the same reference numerals are used for the same components in the drawings, and the duplicated description of the same components will be omitted.
[0018] FIG. 1 illustrates a schematic diagram of an example of a video / image coding system in which embodiments of the present document can be applied.
[0019] As shown in Figure 1, the video / image coding system includes a first device (source device) and a second device (receiving device). The source device can transmit encoded video / image information or data to the receiving device in a file or streaming form via a digital storage medium or a network.
[0020] The source device may include a video source, an encoding device, and a sending unit. The receiving device may include a receiving unit, a decoding device, and a renderer. The encoding device may be referred to as a video / video encoding device, and the decoding device may be referred to as a video / video decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may also include a display unit, which may be a separate device or an external component.
[0021] A video source may acquire video / images through a video / image capture, synthesis, or generation process, etc. A video source may include a video / image capture device and / or a video / image generation device. A video / image capture device may include, for example, one or more cameras, a video / image archive containing previously captured video / images, etc. A video / image generation device may include, for example, a computer, a tablet, a smartphone, etc., and may (electronically) generate video / images. For example, a virtual video / image may be generated via a computer, etc., in which case the video / image capture process may be replaced with a process in which related data is generated.
[0022] An encoding device may encode an input video / image. The encoding device may perform a series of steps such as prediction, transformation, quantization, etc. for compression and coding efficiency. The encoded data (encoded video / image information) may be output in the form of a bitstream.
[0023] The transmitting unit can transmit the encoded video / image information or data output in the form of a bitstream to a receiving unit of a receiving device via a digital storage medium or a network in the form of a file or streaming. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitting unit can include elements for generating a media file through a predetermined file format and can include elements for transmission via a broadcasting / communication network. The receiving unit can receive / extract the bitstream and transmit it to a decoding device.
[0024] The decoding device can decode the video / image by performing a series of steps such as inverse quantization, inverse transformation, and prediction that correspond to the operations of the encoding device.
[0025] The renderer may render the decoded video / image, and the rendered video / image may be displayed via a display unit.
[0026] This document relates to video / image coding. For example, the methods / embodiments disclosed in this document can be applied to methods disclosed in the versatile video coding (VVC) standard. Also, the methods / embodiments disclosed in this document can be applied to methods disclosed in the essential video coding (EVC) standard, the AOMedia Video 1 (AV1) standard, the 2nd generation of audio video coding standard (AVS2) or next generation video / image coding standards (ex. H.267, H.268, etc.).
[0027] Various embodiments relating to video / image coding are presented in this document and, unless otherwise stated, the embodiments may be combined with each other.
[0028] In this document, a video may mean a collection of a series of images over time. A picture generally means a unit showing one image at a specific time, and a slice / tile is a unit constituting a part of a picture in coding. A slice / tile includes one or more coding tree units (CTUs). A picture is composed of one or more slices / tiles. A picture is composed of one or more tile groups. A tile group includes one or more tiles. A brick may represent a rectangular region of CTU rows within a tile in a picture. A tile may be partitioned into multiple bricks, each of which consisting of one or more CTU rows within the tile. A tile that is not partitioned into multiple bricks may also be referred to as a brick.A brick scan is a specific sequential ordering of CTUs partitioning a picture in which the CTUs are ordered consecutively in CTU raster scan in a brick, bricks within a tile are ordered consecutively in a raster scan of the bricks of the tile, and tiles in a picture are ordered consecutively in a raster scan of the tiles of the picture. A tile is a rectangular region of CTUs within a particular tile column and a particular tile row in a picture. The tile column is a rectangular region of CTUs having a height equal to the height of the picture and a width specified by syntax elements in the picture parameter set.The tile row is a rectangular region of CTUs having a height specified by syntax elements in the picture parameter set and a width equal to the width of the picture. A tile scan is a specific sequential ordering of CTUs partitioning a picture in which the CTUs are ordered consecutively in CTU raster scan in a tile whereas tiles in a picture are ordered consecutively in a raster scan of the tiles of the picture. A slice includes an integer number of bricks of a picture that may be exclusively contained in a single NAL unit. A slice may consist of either a number of complete tiles or only a consecutive sequence of complete bricks of one tile. In this document, the terms tile group and slice may be used interchangeably.For example, in this document, a tile group / tile group header may be referred to as a slice / slice header.
[0029] A pixel or a pel may refer to the smallest unit constituting one picture (or image). A term corresponding to a pixel may also be used as a "sample." A sample may generally refer to a pixel or a pixel value, may refer to only a pixel / pixel value of a luma component, or may refer to only a pixel / pixel value of a chroma component. Alternatively, a sample may refer to a pixel value in the spatial domain, or may refer to a transform coefficient in the frequency domain when such a pixel value is transformed into the frequency domain.
[0030] A unit may indicate a basic unit of image processing. A unit may include at least one of a specific region of a picture and information related to the region. One unit may include one luma block and two chroma (e.g., cb, cr) blocks. A unit may be used in combination with terms such as block or area depending on the case. In a general case, an M×N block may include a sample (or sample array) consisting of M columns and N rows, or a set (or array) of transform coefficients.
[0031] In this document, the term " / " and "," should be interpreted to indicate "and / or". For instance, the expression "A / B" may mean "A and / or B." Further, "A, B" may mean "A and / or B." Further, "A / B / C" may mean "at least one of A, B, and / or C." Also, "A / B / C" may mean "at least one of A, B, and / or C."
[0032] Further, in the document, the term "or" should be interpreted to indicate "and / or." For instance, the expression "A or B" may comprise 1) only A, 2) only B, and / or 3) both A and B. In other words, the term "or" in this document should be interpreted to indicate "additionally or alternatively."
[0033] Furthermore, parentheses used in this document may mean "for example." Specifically, when "prediction (intra prediction)" is displayed, "intra prediction" is proposed as an example of "prediction." In other words, "prediction" in this document is not limited to "intra prediction," and "intra prediction" is proposed as an example of "prediction." Furthermore, when "prediction (i.e., intra prediction)" is displayed, "intra prediction" is proposed as an example of "prediction."
[0034] In this document, technical features described individually in one drawing may be embodied individually or simultaneously.
[0035] 2 is a diagram for explaining a schematic configuration of a video / image encoding device to which an embodiment of the present document can be applied. Hereinafter, the video encoding device includes an image encoding device.
[0036] As shown in FIG. 2, the encoding apparatus 200 may be configured to include an image partitioner 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter predictor 221 and an intra predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may further include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstructed block generator. The image division unit 210, the prediction unit 220, the residual processing unit 230, the entropy encoding unit 240, the addition unit 250, and the filtering unit 260 may be configured by one or more hardware components (e.g., an encoder chip set or a processor) according to an embodiment. In addition, the memory 270 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware components may further include the memory 270 as an internal / external component.
[0037] The image division unit 210 may divide an input image (or picture, frame) input to the encoding device 200 into one or more processing units. As an example, the processing units may be called coding units (CUs). In this case, the coding units may be recursively divided from a coding tree unit (CTU) or a largest coding unit (LCU) according to a quad-tree binary-tree ternary-tree (QTBTTT) structure. For example, one coding unit may be divided into a plurality of coding units of a deeper depth based on a quad-tree structure, a binary tree structure, and / or a ternary structure. In this case, for example, the quad-tree structure may be applied first, and then the binary tree structure and / or the ternary structure may be applied. Alternatively, the binary tree structure may be applied first. The coding procedure according to the present disclosure may be performed based on a final coding unit that is not further divided. In this case, the maximum coding unit may be used as the final coding unit based on coding efficiency according to image characteristics, or the coding unit may be recursively divided into coding units of lower depths as necessary, and a coding unit of an optimal size may be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration, which will be described later. As another example, the processing unit may further include a prediction unit (PU: Prediction Unit) or a transform unit (TU: Transform Unit). In this case, the prediction unit and the transform unit may each be divided or partitioned from the final coding unit described above.The prediction unit may be a unit of sample prediction, and the transform unit may be a unit for deriving transform coefficients and / or a unit for deriving a residual signal from the transform coefficients.
[0038] The unit may be used interchangeably with terms such as block or area. In general, an M×N block may refer to a collection of samples or transform coefficients consisting of M columns and N rows. A sample may generally refer to a pixel or pixel value, may refer to only a pixel / pixel value of a luma component, or may refer to only a pixel / pixel value of a chroma component. A sample may be used as a term that corresponds to a pixel or pel of one picture (or image).
[0039] The encoding apparatus 200 subtracts a prediction signal (predicted block, prediction sample array) output from the inter prediction unit 221 or the intra prediction unit 222 from an input video signal (original block, original sample array) to generate a residual signal (residual block, residual sample array), and the generated residual signal is transmitted to the conversion unit 232. In this case, as shown in the figure, a unit in the encoder 200 that subtracts a prediction signal (predicted block, prediction sample array) from an input video signal (original block, original sample array) is called a subtraction unit 231. The prediction unit 220 may perform prediction on a processing target block (hereinafter, referred to as a current block) and generate a predicted block including a prediction sample for the current block. The prediction unit 220 determines whether intra prediction or inter prediction is applied in units of the current block or CU. The prediction unit 220 may generate various information related to prediction, such as prediction mode information, as will be described later in the description of each prediction mode, and transmit the information to the entropy encoding unit 240. The prediction information can be encoded in the entropy encoding unit 240 and output in the form of a bitstream.
[0040] The intra prediction unit 222 may predict the current block by referring to samples in the current picture. The referenced samples may be located in the neighborhood of the current block or may be located away from the current block depending on the prediction mode. In the intra prediction, the prediction mode may include a plurality of non-directional modes and a plurality of directional modes. The non-directional modes may include, for example, a DC mode and a planar mode. The directional modes may include, for example, 33 directional prediction modes or 65 directional prediction modes depending on the fineness of the prediction direction. However, this is merely an example, and more or less directional prediction modes may be used depending on the setting. The intra prediction unit 222 may also determine the prediction mode to be applied to the current block using the prediction modes applied to the neighboring blocks.
[0041] The inter prediction unit 221 may derive a predicted block for a current block based on a reference block (reference sample array) identified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between a neighboring block and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, the neighboring block may include a spatial neighboring block present in the current picture and a temporal neighboring block present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be called a collocated reference block, a collocated CU (col CU), etc., and the reference picture including the temporal neighboring block may be called a collocated picture (colPic). For example, the inter prediction unit 221 may generate information indicating which candidate is used to derive a motion vector and / or a reference picture index of the current block by forming a motion information candidate list based on neighboring blocks. Inter prediction may be performed based on various prediction modes, and for example, in the case of a skip mode and a merge mode, the inter prediction unit 221 may use motion information of a neighboring block as motion information of the current block. In the case of the skip mode, unlike the merge mode, a residual signal may not be transmitted.In the case of motion vector prediction (MVP) mode, the motion vector of the neighboring block can be used as a motion vector predictor, and the motion vector of the current block can be indicated by signaling the motion vector difference.
[0042] The prediction unit 220 generates a prediction signal based on various prediction methods described later. For example, the prediction unit 200 can apply intra prediction or inter prediction for prediction of one block, as well as intra prediction and inter prediction simultaneously. This is called combined inter and intra prediction (CIIP). The prediction unit can also be based on an intra block copy (IBC) prediction mode or a palette mode for prediction of a block. The IBC prediction mode or palette mode can be used for content image / movie coding such as games, for example, as in screen content coding (SCC). IBC basically performs prediction within a current picture, but is similar to inter prediction in that a reference block is derived within the current picture. That is, IBC can use at least one of the inter prediction techniques described in this document. The palette mode can be considered as an example of intra coding or intra prediction. When the palette mode is applied, a sample value within a picture can be signaled based on information regarding a palette table and a palette index.
[0043] The prediction signal generated via the prediction unit (including the inter prediction unit 221 and / or the intra prediction unit 222) can be used to generate a reconstructed signal or can be used to generate a residual signal.
[0044] The transform unit 232 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a graph-based transform (GBT), or a conditionally non-linear transform (CNT). Here, the GBT refers to a transform obtained from a graph when the relationship information between pixels is expressed as a graph. The CNT refers to a transform obtained based on a predicted signal generated using all previously reconstructed pixels. In addition, the transform process may be applied to pixel blocks having the same square size, or may be applied to blocks of variable sizes that are not square.
[0045] The quantization unit 233 quantizes the transform coefficients and transmits the quantized transform coefficients to the entropy encoding unit 240, and the entropy encoding unit 240 encodes the quantized signal (information about the quantized transform coefficients) and outputs it as a bitstream. The information about the quantized transform coefficients may be called residual information. The quantization unit 233 may rearrange the quantized transform coefficients in a block form into a one-dimensional vector form based on a coefficient scan order, and generate information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form.
[0046] The entropy encoding unit 240 may perform various encoding methods, such as exponential Golomb, context-adaptive variable length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). The entropy encoding unit 240 may also encode information required for video / image restoration (e.g., values of syntax elements, etc.) together with or separately from the quantized transform coefficients. The encoded information (e.g., encoded video / image information) may be transmitted or stored in a network abstraction layer (NAL) unit unit in the form of a bitstream. The video / image information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). The video / image information may also include general constraint information. In this document, information and / or syntax elements transmitted / signaled from an encoding device to a decoding device are included in video / image information. The video / image information is encoded through the above-mentioned encoding procedure and included in the bitstream. The bitstream may be transmitted via a network or stored in a digital storage medium. Here, the network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitter (not shown) for transmitting and / or a storage unit (not shown) for storing the signal output from the entropy encoding unit 240 may be configured as an internal / external element of the encoding device 200, or the transmitter may be included in the entropy encoding unit 240.
[0047] The quantized transform coefficients output from the quantizer 233 may be used to generate a prediction signal. For example, a residual signal (residual block or residual sample) is restored by applying inverse quantization and inverse transform to the quantized transform coefficients via the inverse quantizer 234 and the inverse transformer 235. A reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) is generated by the adder 250 adding the reconstructed residual signal to the prediction signal output from the inter prediction unit 221 or the intra prediction unit 222. When there is no residual for the current block, such as when the skip mode is applied, the predicted block may be used as the reconstructed block. The adder 250 may be referred to as a reconstruction unit or a reconstructed block generator. The generated reconstructed signal is used for intra prediction of the next current block in the current picture, and may also be used for inter prediction of the next picture after filtering, as described below.
[0048] Meanwhile, luma mapping with chroma scaling (LMCS) may be applied in the picture encoding and / or restoration process.
[0049] The filtering unit 260 may apply filtering to the reconstructed signal to improve subjective / objective image quality. For example, the filtering unit 260 may apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture, and store the modified reconstructed picture in the memory 270, specifically, in the DPB of the memory 270. The various filtering methods include, for example, deblocking filtering, sample adaptive offset, an adaptive loop filter, a bilateral filter, etc. The filtering unit 260 generates various information related to filtering, as will be described later in the description of each filtering method, and transmits the information to the entropy encoding unit 240. The information related to filtering is encoded in the entropy encoding unit 240 and output in the form of a bitstream.
[0050] The modified reconstructed picture transmitted to the memory 270 can be used as a reference picture in the inter prediction unit 221. In this way, when inter prediction is applied, the encoding apparatus can avoid prediction mismatch between the encoding apparatus 100 and the decoding apparatus, and can also improve coding efficiency.
[0051] The DPB of the memory 270 may store the modified reconstructed picture to be used as a reference picture in the inter prediction unit 221. The memory 270 may store motion information of a block from which motion information in the current picture is derived (or encoded) and / or motion information of a block in an already reconstructed picture. The stored motion information may be transmitted to the inter prediction unit 221 to be used as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory 270 may store reconstructed samples of reconstructed blocks in the current picture and transmit them to the intra prediction unit 222.
[0052] FIG. 3 is a diagram illustrating a schematic configuration of a video / image decoding device to which the embodiments of the present document can be applied.
[0053] As shown in FIG. 3, the decoding device 300 may be configured to include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter predictor 331 and an intra predictor 332. The residual processor 320 may include a dequantizer 321 and an inverse transformer 321. The entropy decoding unit 310, the residual processor 320, the predictor 330, the adder 340, and the filter 350 may be configured as one hardware component (e.g., a decoder chipset or processor) according to an embodiment. Also, the memory 360 may include a decoded picture buffer (DPB) and may be configured as a digital storage medium. The hardware components may further include a memory 360 as an internal / external component.
[0054] If a bitstream including video / image information is input, the decoding apparatus 300 can restore an image corresponding to a process in which the video / image information is processed by the encoding apparatus of FIG. 3. For example, the decoding apparatus 300 can derive a unit / block based on block division related information obtained from the bitstream. The decoding apparatus 300 can perform decoding using a processing unit applied in the encoding apparatus. Thus, the processing unit for decoding can be, for example, a coding unit, and the coding unit can be divided from a coding tree unit or a maximal coding unit according to a quad tree structure, a binary tree structure, and / or a ternary tree structure. One or more transform units can be derived from the coding unit. And, the restored image signal decoded and output by the decoding apparatus 300 can be reproduced by a reproduction device.
[0055] The decoding apparatus 300 receives a signal output from the encoding apparatus of FIG. 2 in the form of a bitstream, and the received signal is decoded by the entropy decoding unit 310. For example, the entropy decoding unit 310 may derive information (e.g., video / image information) required for image restoration (or picture restoration) by parsing the bitstream. The video / image information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). The video / image information may also include general constraint information. The decoding apparatus may decode a picture further based on information on the parameter set and / or the general constraint information. Signaling / received information and / or syntax elements described later in this document may be obtained from the bitstream by being decoded through the decoding procedure. For example, the entropy decoding unit 310 decodes information in a bitstream based on a coding method such as exponential Golomb coding, CAVLC (context-adaptive variable length coding), or CABAC (context-adaptive arithmetic coding), and outputs values of syntax elements required for image restoration and quantized values of transform coefficients related to residuals. More specifically, the CABAC entropy decoding method receives bins corresponding to each syntax element in the bitstream, determines a context model using information on the syntax element to be decoded and decoding information of neighboring and to-be-decoded blocks or information on symbols / bins decoded in previous steps, predicts the occurrence probability of bins according to the determined context model, and performs arithmetic decoding of the bins to generate symbols corresponding to the values of each syntax element.At this time, the CABAC entropy decoding method may update the context model using information of the decoded symbol / bin for the context model of the next symbol / bin after determining the context model. Prediction information among the information decoded in the entropy decoding unit 310 is provided to a prediction unit (inter prediction unit 332 and intra prediction unit 331), and residual values entropy decoded in the entropy decoding unit 310, i.e., quantized transform coefficients and related parameter information, may be input to the residual processing unit 320.
[0056] The residual processing unit 320 may derive a residual signal (residual block, residual sample, residual sample array). In addition, information related to filtering among the information decoded in the entropy decoding unit 310 is provided to the filtering unit 350. Meanwhile, a receiving unit (not shown) for receiving a signal output from the encoding device may be further configured as an internal / external element of the decoding device 300, and the receiving unit may be a component of the entropy decoding unit 310. Meanwhile, the decoding device according to this document may be called a video / image / picture decoding device, and the decoding device may be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder includes the entropy decoding unit 310, and the sample decoder includes at least one of the inverse quantization unit 321, the inverse transform unit 322, the addition unit 340, the filtering unit 350, the memory 360, the inter prediction unit 332, and the intra prediction unit 331.
[0057] The inverse quantization unit 321 may inverse quantize the quantized transform coefficients to output transform coefficients. The inverse quantization unit 321 may rearrange the quantized transform coefficients in a two-dimensional block form. In this case, the rearrangement may be performed based on a coefficient scan order performed in an encoding apparatus. The inverse quantization unit 321 may perform inverse quantization on the quantized transform coefficients using a quantization parameter (e.g., quantization step size information) to obtain transform coefficients.
[0058] The inverse transform unit 322 inversely transforms the transform coefficients to obtain a residual signal (residual block, residual sample array).
[0059] The prediction unit 330 performs prediction on a current block and generates a predicted block including prediction samples for the current block. The prediction unit 330 may determine whether intra prediction or inter prediction is applied to the current block based on information on the prediction output from the entropy decoding unit 310, and may determine a specific intra / inter prediction mode.
[0060] The prediction unit 330 may generate a prediction signal based on various prediction methods described later. For example, the prediction unit may apply intra prediction or inter prediction for prediction of one block, and may simultaneously apply intra prediction and inter prediction. This may be called combined inter and intra prediction (CIIP). In addition, the prediction unit may be based on an intra block copy (IBC) prediction mode or a palette mode for prediction of a block. The IBC prediction mode or palette mode may be used for content image / movie coding such as games, for example, as in screen content coding (SCC). IBC basically performs prediction within a current picture, but may be performed similarly to inter prediction in that a reference block is derived within the current picture. That is, IBC may use at least one of the inter prediction techniques described in this document. The palette mode may be considered as an example of intra coding or intra prediction. When the palette mode is applied, information on a palette table and a palette index is included in the video / image information and signaled.
[0061] The intra prediction unit 331 may predict the current block by referring to samples in the current picture. The referenced samples may be located in the neighborhood of the current block or may be located away from the current block depending on the prediction mode. In intra prediction, prediction modes include a plurality of non-directional modes and a plurality of directional modes. The intra prediction unit 331 may also determine the prediction mode applied to the current block using the prediction modes applied to the neighboring blocks.
[0062] The inter prediction unit 332 may derive a predicted block for the current block based on a reference block (reference sample array) identified by a motion vector on a reference picture. At that time, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on correlation of motion information between neighboring blocks and the current block. The motion information includes a motion vector and a reference picture index. The motion information may further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, the neighboring blocks include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the inter prediction unit 332 may configure a motion information candidate list based on the neighboring blocks and derive a motion vector and / or a reference picture index for the current block based on the received candidate selection information. Inter prediction may be performed based on various prediction modes, and the information on the prediction includes information indicating an inter prediction mode for the current block.
[0063] The adder 340 may generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the acquired residual signal to a prediction signal (predicted block, predicted sample array) output from a prediction unit (including the inter prediction unit 332 and / or the intra prediction unit 331). When there is no residual for a current block, such as when a skip mode is applied, the predicted block may be used as a reconstructed block.
[0064] The adder 340 may be referred to as a reconstruction unit or a reconstruction block generator. The generated reconstruction signal may be used for intra prediction of a next block to be processed in a current picture, may be output after filtering as described below, or may be used for inter prediction of a next picture.
[0065] Meanwhile, luma mapping with chroma scaling (LMCS) can be applied during the picture decoding process.
[0066] The filtering unit 350 may apply filtering to the reconstructed signal to improve subjective / objective image quality. For example, the filtering unit 350 may apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture, and may transmit the modified reconstructed picture to the memory 60, specifically, to the DPB of the memory 360. The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, an adaptive loop filter, a bilateral filter, etc.
[0067] The (modified) reconstructed picture stored in the DPB of the memory 360 can be used as a reference picture in the inter prediction unit 332. The memory 360 can store motion information of a block from which motion information in the current picture is derived (or decoded) and / or motion information of a block in an already reconstructed picture. The stored motion information is transmitted to the inter prediction unit 221 to be used as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory 360 can store reconstructed samples of reconstructed blocks in the current picture and transmit them to the intra prediction unit 331.
[0068] In this document, the embodiments described for the filtering unit 260, the inter prediction unit 221, and the intra prediction unit 222 of the encoding device 200 can also be applied identically or correspondingly to the filtering unit 350, the inter prediction unit 332, and the intra prediction unit 331 of the decoding device 300, respectively.
[0069] The video / image coding method according to the present document may be performed based on the following partitioning structure. Specifically, procedures such as prediction, residual processing ((inverse) transform, (inverse) quantization, etc.), syntax element coding, filtering, etc., described below, may be performed based on the CTU, CU (and / or TU, PU) derived based on the partitioning structure. The block partitioning procedure is performed in the image partitioning unit 210 of the encoding device described above, and partition-related information may be (encoded) by the entropy encoding unit 240 and transmitted to the decoding device in the form of a bitstream. The entropy decoding unit 310 of the decoding device may derive a block partitioning structure of the current picture based on the partitioning-related information obtained from the bitstream, and perform a series of procedures for image decoding based on the block partitioning structure (e.g., prediction, residual processing, block / picture reconstruction, in-loop filtering, etc.). The CU size and the TU size may be the same, or there may be multiple TUs in the CU area. Meanwhile, the CU size may generally indicate a luma component (sample) CB (coding block) size. The TU size may generally indicate a luma component (sample) TB (transform block) size. The chroma component (sample) CB or TB size may be derived based on the luma component (sample) CB or TB size according to a component ratio according to a color format (chroma format, for example, 4:4:4, 4:2:2, 4:2:0, etc.) of a picture / image. The TU size may be derived based on maxTbSize. For example, if the CU size is larger than the maxTbSize, a plurality of TUs (TBs) of the maxTbSize may be derived from the CU, and transform / inverse transform may be performed in units of the TUs (TBs). Also, for example, if intra prediction is applied, an intra prediction mode / type may be derived in units of the CU (or CB), and a procedure of deriving a neighboring reference sample and generating a predicted sample may be performed in units of a TU (or TB).In this case, one or more TUs (or TBs) may exist within one CU (or CB) region, and in this case, the multiple TUs (or TBs) may share the same intra prediction mode / type.
[0070] In addition, in the video / image coding according to this document, the image processing units may have a hierarchical structure. A picture may be divided into one or more tiles, bricks, slices, and / or tile groups. A slice may include one or more bricks. A brick may include one or more CTU rows in a tile. A slice may include an integer number of bricks in a picture. A tile group may include one or more tiles. A tile may include one or more CTUs. The CTUs may be divided into one or more CUs. A tile is a rectangular region of CTUs within a particular tile column and a particular tile row in a picture. A tile group may include an integer number of tiles according to tile raster scan in a picture. A slice header may carry information / parameters that may be applied to a corresponding slice (blocks in a slice). If the encoding / decoding device has a multi-core processor, the encoding / decoding procedures for the tiles, slices, bricks, and / or tile groups may be parallelized. In this document, slices or tile groups may be mixed. That is, a tile group header may be referred to as a slice header. Here, a slice may have one of slice types including intra (I) slice, predictive (P) slice, and bi-predictive (B) slice. For blocks in an I slice, inter prediction may not be used for prediction, and only intra prediction may be used. Of course, in this case, original sample values may be coded and signaled without prediction. For blocks in a P slice, intra prediction or inter prediction may be used, and if inter prediction is used, only uni prediction may be used.Meanwhile, for blocks in a B slice, intra prediction or inter prediction can be used, and if inter prediction is used, up to bi-prediction can be used.
[0071] The encoder determines the tile / tile group, brick, slice, maximum and minimum coding unit sizes based on the characteristics of the video image (e.g., resolution) or taking into account coding efficiency or parallel processing, and information regarding this or information that can guide this can be included in the bitstream.
[0072] The decoder can obtain information indicating whether the tile / tile group, brick, slice, or CTU within the current picture is divided into multiple coding units, etc. Efficiency can be improved by allowing such information to be obtained (transmitted) only under certain conditions.
[0073] The slice header (slice header syntax) may include information / parameters commonly applicable to the slices. The APS (APS syntax) or PPS (PPS syntax) may include information / parameters commonly applicable to one or more pictures. The SPS (SPS syntax) may include information / parameters commonly applicable to one or more sequences. The VPS (VPS syntax) may include information / parameters commonly applicable to multiple layers. The DPS (DPS syntax) may include information / parameters commonly applicable to all videos. The DPS may include information / parameters related to concatenation of coded video sequences (CVS).
[0074] In this document, the higher level syntax may include at least one of the APS syntax, PPS syntax, SPS syntax, VPS syntax, DPS syntax, and slice header syntax.
[0075] Also, for example, information regarding the division and configuration of tiles / tile groups / bricks / slice can be configured at the encoding end via the higher level syntax and transmitted to the decoding device in the form of a bitstream.
[0076] In this document, at least one of quantization / dequantization and / or transform / inverse transform may be omitted. If the quantization / dequantization is omitted, the quantized transform coefficients may be referred to as transform coefficients. If the transform / inverse transform is omitted, the transform coefficients may be referred to as coefficients or residual coefficients, or may still be referred to as transform coefficients for uniformity of expression.
[0077] In this document, the quantized transform coefficients and the transform coefficients may be referred to as transform coefficients and scaled transform coefficients, respectively. In this case, the residual information may include information on the transform coefficient(s), and the information on the transform coefficient(s) may be signaled via a residual coding syntax. Transform coefficients may be derived based on the residual information (or information on the transform coefficient(s)), and scaled transform coefficients may be derived via an inverse transform (scaling) on the transform coefficients. Residual samples may be derived based on an inverse transform (transform) on the scaled transform coefficients. This may be similarly applied / expressed in other parts of this document.
[0078] As described above, the encoding apparatus may perform various encoding methods such as exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), CABAC (Context-Adaptive Binary Arithmetic Coding), etc. Also, the decoding apparatus may decode information in a bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output values of syntax elements required for image restoration and quantized values of transform coefficients related to residuals. For example, the above coding methods may be performed as described below.
[0079] FIG. 4 exemplarily illustrates Context-Adaptive Binary Arithmetic Coding (CABAC) for encoding syntax elements.
[0080] The encoding process of CABAC may include a process of converting an input signal into a binary value through binarization when the input signal is a syntax element that is not a binary value. If the input signal is already a binary value (i.e., the value of the input signal is a binary value), the input signal may be bypassed without being binarized. Here, each binary digit 0 or 1 constituting a binary value may be referred to as a bin. For example, if the binary string after binarization is 110, each of 1, 1, and 0 may be referred to as a bin. The bin(s) for one syntax element may indicate the value of the syntax element.
[0081] The binarized bins of the syntax elements may be input to a regular coding engine or a bypass coding engine. The regular coding engine may assign a context model reflecting a probability value to the corresponding bin, and may code the corresponding bin based on the assigned context model. The regular coding engine may update the context model for the corresponding bin after performing coding for each bin. As described above, the coded bins may be referred to as context-coded bins.
[0082] Meanwhile, when the binarized bin of the syntax element is input to a bypass encoding engine, it can be coded as follows. For example, the bypass encoding engine of the encoding device omits a procedure of estimating a probability for an input bin and a procedure of updating a probability model applied to the bin after encoding. When bypass encoding is applied, the encoding device can encode the input bin by applying a uniform probability distribution instead of allocating a context model, thereby improving the encoding speed. As described above, the encoded bin can be called a bypass bin.
[0083] Entropy decoding performs the same process as the entropy encoding described above in reverse order. For example, when a syntax element is decoded based on a context model, a decoding device may receive a bin corresponding to the syntax element through a bitstream. Then, a context model may be determined using the syntax element and decoding information of a block to be decoded or a neighboring block, or information of a symbol / bin decoded in a previous step, and a value of the syntax element may be derived by predicting the occurrence probability of the received bin according to the determined context model and performing arithmetic decoding of the bin. Then, the context model of the bin to be decoded next may be updated as the determined context model.
[0084] Also, for example, when a syntax element is bypass decoded, a decoding device may receive a bin corresponding to the syntax element through a bitstream and may decode the input bin by applying a uniform probability distribution. In this case, a procedure for deriving a context model of the syntax element and a procedure for updating the context model applied to the bin after decoding may be omitted.
[0085] The residual samples may be derived as quantized transform coefficients through a transform and quantization process. The quantized transform coefficients may also be called transform coefficients. In this case, the transform coefficients in the block may be signaled in the form of residual information. The residual information may include a residual coding syntax. That is, an encoding device may construct a residual coding syntax based on the residual information, encode the residual coding syntax, and output the resulting bitstream, and a decoding device may decode the residual coding syntax obtained from the bitstream to derive residual (quantized) transform coefficients. The residual coding syntax may include syntax elements indicating whether a transform has been applied to the corresponding block, where the last valid transform coefficient in the block is located, whether a valid transform coefficient exists in a subblock, and what the magnitude / sign of the valid transform coefficient is, as described below.
[0086] For example, the (quantized) transform coefficients may be encoded and / or decoded based on syntax elements such as last_sig_coeff_x_prefix, last_sig_coeff_y_prefix, last_sig_coeff_x_suffix, last_sig_coeff_y_suffix, coded_sub_block_flag, sig_coeff_flag, par_level_flag, abs_level_gtX_flag, abs_remainder, coeff_sign_flag, dec_abs_level, etc. This may be referred to as residual (data) coding or (transform) coefficient coding. Syntax elements related to encoding / decoding of residual data may be shown in Table 1 or Table 2 below.
[0087] [Table 1-1]
[0088]
Table 1-2
[0089]
Table 1-3
[0090]
Table 1-4
[0091]
Table 1-5
[0092]
Table 1-6
[0093]
Table 1-7
[0094]
Table 2-1
[0095]
Table 2-2
[0096]
Table 2-3
[0097]
Table 2-4
[0098]
Table 2-5
[0099]
Table 2-6
[0100]
Table 2-7
[0101]
Table 2-8
[0102] In Tables 1 and 2, transform_skip_flag indicates whether transform is skipped for an associated block. The transform_skip_flag is a syntax element of a transform skip flag. The associated block is a coding block (CB) or a transform block (TB). With respect to the transform (and quantization) and residual coding procedures, CB and TB may be used interchangeably. For example, a residual sample may be derived for the CB, and a (quantized) transform coefficient may be derived through transform and quantization for the residual sample. Through the residual coding procedure, information (e.g., syntax elements) efficiently indicating the position, magnitude, sign, etc. of the (quantized) transform coefficient may be generated and signaled. The quantized transform coefficient may be simply referred to as a transform coefficient. In general, if the CB is not larger than the maximum TB, the size of the CB is the same as the size of the TB, and in this case, the target block to be transformed (and quantized) and residual coded may be referred to as a CB or a TB. On the other hand, if CB is greater than the maximum TB, the target block to be transformed (and quantized) and residually coded may be referred to as a TB. Hereinafter, it will be described that syntax elements related to residual coding are signaled in units of a transform block (TB), but this is merely an example, and as described above, the TB may be used interchangeably with the CB.
[0103] The syntax for residual coding according to the transform omission flag is shown in Table 3 or Table 4.
[0104] [Table 3-1]
[0105] [Table 3-2]
[0106]
Table 3-3
[0107]
Table 3-4
[0108]
Table 4-1
[0109]
Table 4-2
[0110]
Table 4-3
[0111]
Table 4-4
[0112]
Table 4-5
[0113] According to this embodiment, the residual coding may be branched according to the value of the transform skip flag transform_skip_flag. That is, depending on the value of the transform skip flag (depending on whether the transform is skipped), different syntax elements may be used for the residual coding. Residual coding used when transform skipping is not applied (i.e., when a transform is applied) may be called Regular Residual Coding (RRC), and residual coding used when transform skipping is not applied (i.e., when a transform is not applied) may be called Transform Skip Residual Coding (TSRC). The regular residual coding may also be called general residual coding. The regular residual coding may also be called a regular residual coding syntax structure, and the transform skipped residual coding may also be called a transform skipped residual coding syntax structure. Tables 1 and 2 may show syntax elements for residual coding when the value of transform_skip_flag is 0, i.e., when a transform is applied, and Tables 3 and 4 may show syntax elements for residual coding when the value of transform_skip_flag is 1, i.e., when a transform is not applied.
[0114] Specifically, for example, a transform skip flag indicating whether or not a transform of a transform block is skipped may be parsed, and it may be determined whether the transform skip flag is 1. If the value of the transform skip flag is 0, syntax elements last_sig_coeff_x_prefix, last_sig_coeff_y_prefix, last_sig_coeff_x_suffix, last_sig_coeff_y_suffix, sb_coded_flag, sig_coeff_flag, abs_level_gtx_flag, par_level_flag, abs_remainder, dec_abs_level, and / or coeff_sign_flag for residual coefficients of the transform block may be parsed, as shown in Table 1 or Table 2, and the residual coefficients may be derived based on the syntax elements. In this case, the syntax elements may be parsed sequentially, or the parsing order may be changed. Furthermore, the abs_level_gtx_flag may indicate abs_level_gt1_flag and / or abs_level_gt3_flag. For example, abs_level_gtx_flag[n][0] is an example of the first transform coefficient level flag (abs_level_gt1_flag), and the abs_level_gtx_flag[n][1] is an example of the second transform coefficient level flag (abs_level_gt3_flag).
[0115] In one embodiment, the encoding device may encode (x, y) position information of the last non-zero transform coefficient in a transform block based on syntax elements last_sig_coeff_x_prefix, last_sig_coeff_y_prefix, last_sig_coeff_x_suffix, and last_sig_coeff_y_suffix. More specifically, the last_sig_coeff_x_prefix indicates the prefix of the column position of the last significant coefficient in the scanning order in the transform block, the last_sig_coeff_y_prefix indicates the prefix of the row position of the last significant coefficient in the scanning order in the transform block, the last_sig_coeff_x_suffix indicates the suffix of the column position of the last significant coefficient in the scanning order in the transform block, and the last_sig_coeff_y_suffix indicates the suffix of the row position of the last significant coefficient in the scanning order in the transform block. Here, the significant coefficient may indicate the non-zero coefficient. Also, the scan order may be a diagonal scan order from top right to bottom right. Alternatively, the scan order may be a horizontal scan order or a vertical scan order. The scan order may be determined based on whether intra / inter prediction is applied to a target block (CB or CB including TB) and / or a specific intra / inter prediction mode.
[0116] Next, the encoding apparatus divides the transform block into 4x4 sub-blocks, and for each 4x4 sub-block, may use a 1-bit syntax element coded_sub_block_flag to indicate whether there is a non-zero coefficient in the current sub-block.
[0117] If the value of coded_sub_block_flag is 0, there is no more information to transmit, and therefore the encoding apparatus may end the encoding process for the current sub-block. On the other hand, if the value of coded_sub_block_flag is 1, the encoding apparatus may continue to perform the encoding process for sig_coeff_flag. Since the last sub-block containing a non-zero coefficient does not require encoding for coded_sub_block_flag, and since the sub-block containing DC information of the transform block is highly likely to contain a non-zero coefficient, coded_sub_block_flag may not be encoded and may be assumed to have a value of 1.
[0118] If the value of coded_sub_block_flag is 1 and it is determined that a non-zero coefficient exists in the current sub-block, the encoding apparatus may encode sig_coeff_flag having a binary value according to the reverse scanned order. The encoding apparatus may encode a 1-bit syntax element sig_coeff_flag for each transform coefficient according to the scan order. If the value of the transform coefficient at the current scan position is not zero, the value of sig_coeff_flag may be 1. Here, in the case of a sub-block including the last non-zero coefficient, since sig_coeff_flag does not need to be encoded for the last non-zero coefficient, the encoding process for the sub-block may be omitted. Level information encoding may be performed only if sig_coeff_flag is 1, and four syntax elements may be used in the level information encoding process. More specifically, each sig_coeff_flag[xC][yC] may indicate whether the level (value) of a corresponding transform coefficient at each transform coefficient position (xC, yC) in the current TB is non-zero. In one embodiment, the sig_coeff_flag may correspond to an example of a syntax element of a significance coefficient flag indicating whether a quantized transform coefficient is a significance coefficient other than zero.
[0119] The remaining level value after encoding for sig_coeff_flag can be derived as follows: That is, a syntax element remAbsLevel indicating a level value to be encoded can be derived as follows: remAbsLevel=[sig_coeff_flag+remAbsLevel+remAbsLevel]=[sig_coeff_flag+remAbsLevel].
[0120]
number
[0121] Here, coeff[n] denotes the actual transform coefficient value.
[0122] In addition, abs_level_gtx_flag[n][0] may indicate whether remAbsLevel[n] at a corresponding scanning position (n) is greater than 1. For example, if the value of abs_level_gtx_flag[n][0] is 0, the absolute value of the transform coefficient at the corresponding position is 1. In addition, if the value of abs_level_gtx_flag[n][0] is 1, the remAbsLevel[n] indicating the level value to be coded thereafter may be updated according to the following formula.
[0123]
number
[0124] In addition, the least significant coefficient (LSB) value of remAbsLevel[n] described in the above-mentioned Equation 2 can be encoded via par_level_flag as shown in the following Equation 3.
[0125]
number
[0126] Here, par_level_flag[n] can indicate the parity of the transform coefficient level (value) at scanning position n.
[0127] The transform coefficient level value remAbsLevel[n] to be encoded after encoding par_level_flag[n] can be updated according to the following formula.
[0128]
number
[0129] abs_level_gtx_flag[n][1] may indicate whether the remAbsLevel at the corresponding scanning position (n) is greater than 3. Encoding for abs_remainder[n] may be performed only if abs_level_gtx_flag[n][1] is 1. The relationship between the actual transform coefficient value coeff and each syntax element is as follows:
[0130]
number
[0131] Moreover, the following Table 5 shows examples related to the above-mentioned Equation 5.
[0132] [Table 5]
[0133] Here, |coeff[n]| indicates a transform coefficient level (value) and may be expressed as AbsLevel for the transform coefficient. Also, the sign of each coefficient may be encoded using coeff_sign_flag, which is a 1-bit symbol.
[0134] As another example, when the value of the transform skip flag is 1, syntax elements sb_coded_flag, sig_coeff_flag, coeff_sign_flag, abs_level_gtx_flag, par_level_flag, and / or abs_remainder for the residual coefficients of the transform block may be parsed, and the residual coefficients may be derived based on the syntax elements, as shown in Table 3 or Table 4. In this case, the syntax elements may be parsed sequentially, or the parsing order may be changed. Also, the abs_level_gtx_flag may indicate abs_level_gt1_flag, abs_level_gt3_flag, abs_level_gt5_flag, abs_level_gt7_flag, and / or abs_level_gt9_flag. For example, abs_level_gtx_flag[n][j] is a flag indicating whether the absolute value or level (value) of the transform coefficient at scanning position n is greater than (j<<1)+1, where (j<<1)+1 may be replaced with a predetermined threshold such as a first threshold or a second threshold, depending on the case.
[0135] Meanwhile, CABAC provides high performance but has a drawback of poor throughput performance. This is due to the regular encoding engine of CABAC, and regular encoding (i.e., encoding through the regular encoding engine of CABAC) shows high data dependency because it uses the probability state and range updated through the encoding of the previous bin, and it may take a long time to read the probability interval and determine the current state. The throughput problem of CABAC can be solved by limiting the number of context-coded bins. For example, as shown in Table 5 above, the sum of bins used to express sig_coeff_flag[n], abs_level_gtx_flag[n][0], par_level_flag[n], and abs_level_gtx_flag[n][1] may be limited to 1.75 per pixel in the transform block depending on the size of the transform block. In this case, when the encoding device uses all of the limited number of context-coded bins to encode the context elements, it can perform bypass coding by binarizing the remaining coefficients through a binarization method described below without using context coding. That is, when the number of coded context coding bins in a TU is TU width * TU height * 1.75, sig_coeff_flag[n], abs_level_gtx_flag[n][0], par_level_flag[n], and abs_level_gtx_flag[n][1] coded to further context coding bins are not coded, and the |coeff[n]| value can be immediately coded to dec_abs_level[n] as shown in Table 6 below.
[0136] [Table 6]
[0137] In this case, the sign of each coefficient can be coded using a 1-bit symbol, coeff_sign_flag[n].
[0138] FIG. 5 exemplarily illustrates transform coefficients in a 4×4 block.
[0139] The 4×4 block in FIG. 5 shows an example of quantized coefficients. The block shown in FIG. 5 is a 4×4 transform block, or a 4×4 sub-block of an 8×8, 16×16, 32×32, 64×64 transform block. The 4×4 block in FIG. 5 may represent a luma block or a chroma block. However, this is only an example, and large block-size (up to 64×64 size) transforms are possible in this embodiment, which are mainly useful for high-resolution videos (e.g., 1080p and 4K sequences). High-frequency transform coefficients may be zeroed for transform blocks having a size (width or height or both width and height) of 64, and only low-frequency coefficients may be kept. For example, for an M×N transform block, where M is the block width and N is the block height, only the left 32 columns of transform coefficients may be kept when M is 64. Alternatively, only the top 32 rows of transform coefficients may be kept when N is 64.
[0140] When the transform skipping mode is used for a large size block, the entire block can be used without zeroing for any value. Since a configurable maximum transform size is supported in SPS, an encoding device can adaptively select a transform size of up to 16, 32, or 64 lengths depending on the needs of a specific implementation. Specifically, the binarization of the last non-zero coefficient position coding is coded based on the reduced TU size, and the context model selection for the last non-zero coefficient position coding can be determined by the original TU size.
[0141] As an example, Fig. 5 shows the coding result for the coefficients scanned in the reverse diagonal direction. In Fig. 5, n (0 to 15) designates the scan position of the coefficients in the reverse diagonal direction. When n is 15, it indicates the coefficient in the lower right corner that is scanned first in the 4x4 block, and when n is 0, it indicates the coefficient in the upper left corner that is scanned last.
[0142] Meanwhile, as described above, when an input signal is a syntax element that is not a binary value, the encoding apparatus can convert the input signal into a binary value by binarizing the value of the input signal. Also, the decoding apparatus can decode the syntax element to derive a binarized value (i.e., a binarized bin) of the syntax element, and can de-binarize the binarized value to derive the value of the syntax element. The binarization process can be performed as a truncated rice (TR) binarization process, a k-th order Exp-Golomb (EGk) binarization process, a Limited k-th order Exp-Golomb (Limited EGk), or a fixed-length (FL) binarization process, which will be described later. In addition, the inverse binarization process may refer to a process performed based on the TR binarization process, the EGk binarization process, the Limetid EGK binarization process, or the FL binarization process to derive the value of the syntax element.
[0143] For example, the TR binarization process can be performed as follows.
[0144] The inputs of the TR binarization process are the TR binarization request and the syntax elements cMax and cRiceParam, and the output of the TR binarization process is the TR binarization for the value symbolVal corresponding to the bin string.
[0145] As an example, if a suffix bin string exists for a syntax element, the TR bin string for the syntax element is the concatenation of a prefix bin string and a suffix bin string, and if the suffix bin string does not exist, the TR bin string for the syntax element is the prefix bin string. For example, the prefix bin string can be derived as follows:
[0146] The prefix value of the symbolVal can be derived as follows:
[0147]
number
[0148] Here, prefixVal may represent a prefix value of symbolVal. The prefix of the TR bin string (ie, the prefix bin string) may be derived as follows.
[0149] For example, if the prefixVal is less than cMax>>cRiceParam, the prefix bin string is a bit string of length prefixVal+1 indexed by binIdx. That is, if the prefixVal is less than cMax>>cRiceParam, the prefix bin string is a bit string of prefixVal+1 bits pointed to by binIdx. For binIdx less than prefixVal, bin is 1. For binIdx equal to prefixVal, bin is 0.
[0150] For example, the bin string derived by unary binarization of the prefixVal is as shown in Table 7 below.
[0151] [Table 7]
[0152] On the other hand, if the prefixVal is not less than cMax>>cRiceParam, then the prefix bin string is a bit string whose length is cMax>>cRiceParam and whose bins are all 1's.
[0153] Also, a suffix bin string of the TR bin string may exist if cMax is greater than symbolVal and cRiceParam is greater than 0. For example, the prefix bin string may be derived as follows:
[0154] The suffix value of the symbolVal for the syntax element can be derived as follows:
[0155]
number
[0156] Here, suffixVal can indicate the suffix value of the symbolVal.
[0157] The suffix of the TR bin string (i.e., the suffix bin string) can be derived based on the FL binary evolution process for suffixVal where the cMax value is (1<<cRiceParam)-1.
[0158] When the value of the input parameter cRiceParam is 0, the TR binarization is exactly a truncated unary binarization and it is always invoked with a cMax value equal to the largest possible value of the syntax element being decoded. (For the input parameter cRiceParam=0,the TR binarization is exactly a truncated unary binarization and it is always invoked with a cMax value equal to the largest possible value of the syntax element being decoded.)
[0159] On the other hand, the EGk binarization process can be executed as follows.
[0160] The input of the EGk binarization process is a request for EGk binarization. Also, the output of the EGk binarization process is the EGk binarization for the value symbolVal corresponding to the bin string.
[0161] The bit string of the EGk binarization process for symbolVal can be derived as follows.
[0162] [Table 8]
[0163] Referring to Table 8, a binary value X can be added to the end of the bin string via each call to put(X), where X is either 0 or 1.
[0164] And, the Limited EGk binarization process can be performed as follows.
[0165] The input of the Limited EGk binarization process is the request for Limited EGk binarization and the Rice parameter riceParam, and the output of the Limited EGk binarization process is the Limited EGk binarization for the value symbolVal associated with the corresponding bin string.
[0166] The bin string of the Limited EGk binarization process for symbolVal can be derived as follows:
[0167] [Table 9]
[0168] Referring to Table 9, a binary value X can be added to the end of the bin string via each call of put(X), where X is either 0 or 1.
[0169] The variables log2TransformRange and maxPrefixExtensionLength can be derived as follows:
[0170]
number
[0171] And the FL binarization process can be performed as follows.
[0172] The input of the FL binarization process is the FL binarization request and cMax for the syntax element, and the output of the FL binarization process is the FL binarization for the value symbolVal corresponding to the bin string.
[0173] The FL binarization may be constructed using a bin string having a fixed length of the number of bits of the symbol value symbolVal, where the fixed length may be derived as follows:
[0174]
number
[0175] That is, a bin string for a symbol value symbolVal can be derived through FL binarization, and the bin length (ie, the number of bits) of the bin string is a fixed length.
[0176] The bin indexing for FL binarization is a scheme using increasing values from the most significant bit to the least significant bit, for example, the bin index associated with the most significant bit is binIdx=0.
[0177] Meanwhile, the binarization process for the syntax element abs_remainder[n] of the residual information can be performed as follows.
[0178] The inputs of the binarization process for the abs_remainder[n] are a binarization request for the syntax element abs_remainder[n], a color component index cIdx, a luma position (x0, y0) indicating the top left sample of the current luma transform block based on the top left luma sample of the picture, a current coefficient scan position (xC, yC), a binary logarithm of the transform block width longTbWidth, and a binary logarithm of the transform block height log2TbHeight. The output of the binarization process for the abs_remainder is the binarization of the abs_remainder (i.e., a binarized bin string of the abs_remainder). An available bin string for the abs_remainder can be derived through the binarization process.
[0179] The Rice parameter cRiceParam for the abs_remainder[n] may be derived through a Rice parameter derivation process that is executed by inputting the hue component index cIdx, the luma position (x0, y0), the current coefficient scan position (xC, yC), log2TbWidth, which is the binary logarithm of the width of the transform block, and log2TbHeight, which is the binary logarithm of the height of the transform block. A detailed description of the Rice parameter derivation process will be given later.
[0180] cMax for the currently coded abs_remainder[n] may be derived based on the Rice parameter cRiceParam. For example, cMax may be derived as follows:
[0181]
number
[0182] Meanwhile, the binarization for syntax element abs_remainder[n], i.e., the bin string for the abs_remainder[n] is the concatenation of the prefix bin string and the suffix bin string if the suffix bin string exists, otherwise the bin string for the abs_remainder[n] is the prefix bin string.
[0183] For example, the prefix bin string for abs_remainder[n] can be derived as follows:
[0184] The prefix value prefixVal of the abs_remainder[n] can be derived as follows:
[0185]
number
[0186] The prefix bin string of abs_remainder[n] can be derived via a TR binarization process on the prefixVal using cMax and the cRiceParam as inputs.
[0187] If the prefix bin string is the same as a bit string with all bits set to 1 and a bit length of 6, then there can be a suffix bin string of abs_remainder[n], which can be derived as follows:
[0188] The suffix value suffixVal of the abs_remainder[n] can be derived as follows:
[0189]
number
[0190] The suffix bin string of abs_remainder[n] can be derived via a Limited EGk binarization process for the binarization of the suffixVal using cRiceParam+1 and cRiceParam as inputs.
[0191] The Rice parameter for abs_remainder[n] can be derived by the following process.
[0192] The inputs of the Rice parameter derivation process are a base level baseLevel, a color component index cIdx, a luma position (x0, y0), a current coefficient scan position (xC, yC), a binary logarithm of the width of the transform block log2TbWidth, and a binary logarithm of the height of the transform block log2TbHeight. The luma position (x0, y0) may indicate the top left sample of the current luma transform block based on the top left luma sample of the picture. The output of the Rice parameter derivation process is the Rice parameter cRiceParam.
[0193] For example, based on the array AbsLevel[x][y] for a transform block with a given component index cIdx and top-left luma position (x0, y0), the variable locSumAbs can be derived by the pseudo code disclosed in the table below.
[0194] [Table 10]
[0195] In Table 10, when baseLevel is 0, the variable s is set to Max(0, QState-1), and the Rice parameter cRiceParam and the variable ZeroPos[n] can be derived based on the variables locSumAbs, trafoSkip, and s as shown in Table 11 below. When baseLevel is greater than 0, the Rice parameter cRiceParam can be derived based on the variables locSumAbs and trafoSkip as shown in Table 11.
[0196] [Table 11]
[0197] Meanwhile, the binarization process for the syntax element dec_abs_level of the residual information can be performed as follows.
[0198] The inputs of the binarization process for the dec_abs_level are a binarization request for the syntax element dec_abs_level[n], a color component index cIdx, a luma position (x0, y0), a current coefficient scan position (xC, yC), a binary logarithm of the transform block width log2TbWidth, and a binary logarithm of the transform block height log2TbHeight. The luma position (x0, y0) may indicate the top left sample of the current luma transform block based on the top left luma sample of the picture.
[0199] The output of the binarization process for the dec_abs_level is the binarization of the dec_abs_level (ie, a binarized bin string of the dec_abs_level). Through the binarization process, an available bin string for the dec_abs_level can be derived.
[0200] The Rice parameter cRiceParam for the dec_abs_level[n] can be derived through a Rice parameter derivation process that is executed by inputting the hue component index cIdx, the luma position (x0, y0), the current coefficient scan position (xC, yC), log2TbWidth, which is the binary logarithm of the width of the transform block, and log2TbHeight, which is the binary logarithm of the height of the transform block. A detailed description of the Rice parameter derivation process will be given later.
[0201] Also, for example, cMax for the dec_abs_level[n] can be derived based on the Rice parameter cRiceParam. The cMax can be derived as shown in Equation 10.
[0202] Meanwhile, the binarization for the dec_abs_level[n], i.e., the bin string for the dec_abs_level[n], is the concatenation of the prefix bin string and the suffix bin string if a suffix bin string exists, and if the suffix bin string does not exist, the bin string for the dec_abs_level[n] is the prefix bin string.
[0203] For example, the prefix bin string can be derived as follows:
[0204] The prefix value prefixVal of the dec_abs_level[n] can be derived as follows:
[0205]
number
[0206] The dec_abs_level[n] prefix bin string can be derived via a TR binarization process on the prefixVal using cMax and cRiceParam as inputs.
[0207] If the prefix bin string is the same as a bit string with all bits set to 1 and a bit length of 6, then there can be a suffix bin string of dec_abs_level[n], which can be derived as follows:
[0208] The suffix value suffixVal of the dec_abs_level[n] can be derived as follows:
[0209]
number
[0210] The suffix bin string of dec_abs_level[n] may be derived via a Limited EGk binarization process for the binarization of suffixVal with the exp-Golomb order k set to cRiceParam+1.
[0211] The Rice parameters for dec_abs_level[n] can be derived by the pseudo code in Table 10.
[0212] Meanwhile, the above-mentioned Regular Residual Coding (RRC) and Transform Skip Residual Coding (TSRC) may have the following differences.
[0213] For example, the Rice parameter cRiceParam of the syntax element abs_remainder[] in regular residual coding can be derived as described above, but the Rice parameter cRiceParam of the syntax element abs_remainder[] in transform omitted residual coding can be derived to 1. That is, for example, when transform skip is applied to a current block (e.g., current TB), the Rice parameter cRiceParam for abs_remainder[] of transform omitted residual coding for the current block can be derived to 1.
[0214] Also, referring to Tables 1 to 4, in regular residual coding, abs_level_gtx_flag[n][0] and / or abs_level_gtx_flag[n][1] can be signaled, while in transform-less residual coding, abs_level_gtx_flag[n][0], abs_level_gtx_flag[n][1], abs_level_gtx_flag[n][2], abs_level_gtx_flag[n][3], and abs_level_gtx_flag[n][4] can be signaled. Here, the abs_level_gtx_flag[n][0] can be indicated as abs_level_gt1_flag or the first coefficient level flag, the abs_level_gtx_flag[n][1] can be indicated as abs_level_gt3_flag or the second coefficient level flag, the abs_level_gtx_flag[n][2] can be indicated as abs_level_gt5_flag or the third coefficient level flag, the abs_level_gtx_flag[n][3] can be indicated as abs_level_gt7_flag or the fourth coefficient level flag, and the abs_level_gtx_flag[n][4] can be indicated as abs_level_gt9_flag or the fifth coefficient level flag. Specifically, the first coefficient level flag is a flag indicating whether the coefficient level is greater than a first threshold (e.g., 1), the second coefficient level flag is a flag indicating whether the coefficient level is greater than a second threshold (e.g., 3), the third coefficient level flag is a flag indicating whether the coefficient level is greater than a third threshold (e.g., 5), the fourth coefficient level flag is a flag indicating whether the coefficient level is greater than a fourth threshold (e.g., 7), and the fifth coefficient level flag is a flag indicating whether the coefficient level is greater than a fifth threshold (e.g., 9).
[0215] As described above, compared to regular residual coding, transform-omitted residual coding may further include abs_level_gtx_flag[n][0], abs_level_gtx_flag[n][1], as well as abs_level_gtx_flag[n][2], abs_level_gtx_flag[n][3], and abs_level_gtx_flag[n][4].
[0216] Also, for example, in regular residual coding, the syntax element coeff_sign_flag can be bypass coded, but in transform omitted residual coding, the syntax element coeff_sign_flag can be bypass coded or context coded.
[0217] The following description is made to describe a specific example of this document. The names of specific devices and names of specific signals / information are presented for illustrative purposes only, and the technical features of this specification are not limited to the names used in the following description.
[0218] The following describes a method for efficiently deriving Rice parameters for binarization of information indicating the level value (or absolute value) of a transform coefficient in level coding (e.g., syntax element dec_abs_level indicating the level value of a transform coefficient, syntax element abs_remainder indicating the residual level value of a transform coefficient, etc.).
[0219] The Rice parameter is a variable used to binarize the level value of the transform coefficient, and when residual data coding for a transform block (regular residual coding) is applied to the current block, a Rice parameter look-up table as shown in Table 12 below is used, and when residual data coding for a transform-omitted block (transform-omitted residual coding) is applied to the current block, a Rice parameter look-up table as shown in Table 13 below is used. Here, the Rice parameter look-up table may be called a table related to Rice parameters or a table used to determine Rice parameters, or may be called a table related to Rice parameter candidates.
[0220] [Table 12]
[0221] [Table 13]
[0222] In Tables 12 and 13, locSumAbs is a value derived based on the sum of the level values (x) of the surrounding transform coefficients (x) of the current transform coefficient, and can be derived by the pseudo code in Table 10, for example.
[0223] In the binarization step of the level values, the smaller the value of the Rice parameter cRiceParam, the shorter the codewords assigned to the smaller level values to generate codewords that are favorable for binarizing the smaller level values, and the larger the value of the Rice parameter, the shorter the codewords assigned to the larger level values to generate codewords that are favorable for binarizing the larger level values.
[0224] However, depending on the characteristics of the image and / or the presence / absence / type of syntax that is coded prior to the level values of the transform coefficients, the level values that are generated on average or frequently generated by level coding may differ. Also, in a high bit-rate environment where lossless (or near lossless) coding or coding is performed with a low quantization parameter, level values having different characteristics may be mixed. In such a case, it is more efficient to use multiple Rice parameter lookup tables rather than one Rice parameter lookup table to derive the Rice parameters.
[0225] Therefore, according to one embodiment, two or more Rice parameter lookup tables may be used for each of a case where residual data coding for a transform block (regular residual coding) is applied to a current block and a case where residual data coding for a transform-omitted block (transform-omitted residual coding) is applied to a current block, or two or more Rice parameter lookup tables may be used regardless of whether regular residual coding or transform-omitted residual coding is applied to a current block.
[0226] In this case, for example, the encoding apparatus may derive the Rice parameters by selecting at least one table from among a number of Rice parameter lookup tables based on the configuration (presence or absence, type, etc.) of syntax elements coded prior to syntax elements (e.g., dec_abs_level, abs_remainder, etc.) indicating level values of transform coefficients. Alternatively, the encoding apparatus may derive the Rice parameters by selecting at least one table from among a number of Rice parameter lookup tables based on whether the number of context coded bins coded / decoded in residual data coding exceeds the maximum available number of context coded bins set by a context coding bin constraint algorithm.
[0227] Here, the plurality of Rice parameter lookup tables may have different Rice parameter minimum values and Rice parameter maximum values, and update positions of the Rice parameter values may be different from each other. Also, the syntax element coded preceding the syntax element indicating the level value of the transform coefficient may include at least one of sig_coeff_flag, abs_level_gtx_flag, par_level_flag, or coeff_sign_flag. And, the maximum available number of context coding bins set by the context coding bin constraint algorithm may correspond to remBinsPass1 in Table 1 or Table 2 in the case of residual coding for a transform block, and may correspond to MaxCcbs in Table 3 in the case of residual coding for a transform omitted block.
[0228] As an example, if there are two Rice parameter lookup tables, the first Rice parameter lookup table "A" has m as the minimum value of the Rice parameter. A and n is the maximum value of the Rice parameter. AThe second Rice parameter lookup table "B" may have m as the minimum value of the Rice parameter. B and n is the maximum value of the Rice parameter. B In order to improve the performance of level coding, m B m A can be set to a value larger than n B is n A For example, the first rice parameter lookup table may be configured as shown in Table 14 below, and the second rice parameter lookup table may be configured as shown in Table 15 below.
[0229] [Table 14]
[0230] [Table 15]
[0231] In Table 14, the minimum and maximum values of the rice parameter are 0 and 2, respectively, and in Table 15, the minimum and maximum values of the rice parameter are 1 and 3, respectively. That is, the rice parameter lookup table of Table 15 has larger minimum and maximum values than the rice parameter lookup table of Table 14. Also, in Table 14, the rice parameter value is updated when the locSumAbs value is 12 and 24, and in Table 15, the rice parameter value is updated when the locSumAbs value is 7 and 18. That is, the rice parameter lookup table of Table 14 and the rice parameter lookup table of Table 15 have different update positions of the rice parameter value.
[0232] Therefore, the Rice parameter lookup table of Table 15 may have advantages over the Rice parameter lookup table of Table 14 when relatively large level values occur frequently or when the average level values or coefficients of the lower block being encoded are large.
[0233] The rice parameter lookup tables of Tables 14 and 15 are merely examples of various rice parameter lookup tables that can be used by this embodiment, and when applying this embodiment, the rice parameter lookup tables are not limited to these, and tables having different rice parameter minimum values, maximum values, and update positions can be used.
[0234] Meanwhile, the encoding apparatus may signal a syntax element (or flag) for transmitting information on a Rice parameter lookup table used for a current transform coefficient among a plurality of Rice parameter lookup tables. The syntax element may be signaled in units of coefficient groups (CGs) or in units of transform blocks or transform truncated (coding) blocks. For example, when two Rice parameter lookup tables are used, a value of the syntax element of 0 may indicate a first Rice parameter lookup table, and a value of the syntax element of 1 may indicate a second Rice parameter lookup table. The syntax element may be binarized in one of various ways, such as fixed length binarization, truncated unary binarization, etc.
[0235] When a syntax element (or flag) indicating information regarding a Rice parameter lookup table is obtained from a bitstream, the decoding device can select the Rice parameter lookup table indicated by the syntax element from among a number of Rice parameter lookup tables, and derive Rice parameters for the current transform coefficient based on the selected Rice parameter lookup table.
[0236] Alternatively, the decoding apparatus may infer or derive the Rice parameter lookup table used for level coding of the current transform coefficient (or coefficient group or transform block or coding block) from among a number of Rice parameter lookup tables by using already given information such as the configuration (presence or absence of syntax elements, type, etc.) of syntax elements coded prior to the syntax elements (e.g., dec_abs_level, abs_remainder, etc.) indicating the level value of the current transform coefficient, whether the number of context coding bins coded / decoded in residual data coding exceeds the maximum available context coding bin number set in the context coding bin constraint algorithm, whether lossless or near lossless coding is performed, quantization coefficient information, etc. In this case, a syntax element or flag indicating information about the Rice parameter lookup table used for the current transform coefficient is not signaled.
[0237] As an example, whether the number of context coding bins coded / decoded in residual data coding exceeds the maximum available number of context coding bins (remBinsPass1 or MaxCcbs) set in the context coding bin constraint algorithm may be used to select the Rice parameter lookup table. In this case, if the number of context coding bins coded / decoded in residual data coding exceeds the maximum available number of context coding bins, a simplified syntax structure different from the syntax structure used in the existing residual coding may be used.
[0238] For example, in residual data coding for a transform block, (i) if the number of coded / decoded context coding bins does not exceed remBinsPass1, sig_coeff_flag, abs_level_gtx_flag[0], par_level_flag, and abs_level_gtx_flag[1] are coded prior to abs_remainder indicating the residual level value of the transform coefficient. However, (ii) if the number of coded / decoded context coding bins exceeds remBinsPass1, there is no syntax element coded / decoded prior to coding of the syntax element (dec_abs_level) for the level value of the transform coefficient. Therefore, in the case of (ii), since there is no syntax element coded / decoded prior to the level value coding, the average level value is larger than in the case of (i). Therefore, in the case of (i), a Rice parameter lookup table that generates code words favorable to small level values can be assigned as shown in Table 14, and in the case of (ii), a Rice parameter lookup table that generates code words favorable to large level values can be assigned as shown in Table 15. In this way, when using the remBinsPass1 value, which is already given information, to determine the Rice parameter lookup table, there is an advantage that it is not necessary to code / decode additional flags or syntax elements to know the table information.
[0239] As another example, when the number of context coding bins coded / decoded in residual data coding for the transform skip mode exceeds MaxCcbs, the configuration of the existing syntax may be simplified. Some syntax elements among sig_coeff_flag, coeff_sign_flag, abs_level_gtx_flag[0], par_level_flag, abs_level_gtx_flag[1], abs_level_gtx_flag[2], abs_level_gtx_flag[3], and abs_level_gtx_flag[4] coded / decoded in the existing transform skip residual data coding are omitted and not coded / decoded for the effect of reducing coding performance or complexity. In this case, too, since the average level value changes depending on the existing syntax configuration and the simplified syntax configuration, determining the Rice parameter look-up table using MaxCcbs information or / and coefficient position information can provide better coding performance. In this way, by using the already given information MaxCcbs and coefficient position information to determine the Rice parameter lookup table, there is an advantage that no additional flags or syntax coding / decoding is required to know the table information.
[0240] As another example, the Rice parameter lookup table may be determined without additional flag or syntax coding / decoding using whether lossless or near lossless coding is performed. Lossless or near lossless coding may be performed in units of a picture, slice, CU block, or TU block, and generally has a larger level value of residual data than lossy coding. Therefore, the decoding device may use whether the current picture, slice, CU block, or TU block is lossless or near lossless coded to determine the Rice parameter lookup table, and in this case, coding / decoding of additional flag or syntax is not required. As an example, when lossless or near lossless coding is not performed, the decoding device may derive the Rice parameter using a Rice parameter lookup table having a relatively small minimum / maximum value of the Rice parameter as shown in Table 14, and when lossless or near lossless coding is performed, the decoding device may derive the Rice parameter using a Rice parameter lookup table having a relatively large minimum / maximum value of the Rice parameter as shown in Table 15.
[0241] As another example, when block-based variable quantization is used, a Rice parameter lookup table may be selected using quantization coefficient information. In general, large level values occur frequently when a low quantization coefficient is used, and small level values occur frequently when a high quantization coefficient is used. The same is true for average level values. Thus, as an example, when the value of the quantization coefficient is equal to or greater than a threshold, the decoding apparatus may derive a Rice parameter using a Rice parameter lookup table having a relatively small minimum / maximum value of the Rice parameter as shown in Table 14, and when the value of the quantization coefficient is smaller than the threshold, the decoding apparatus may derive a Rice parameter using a Rice parameter lookup table having a relatively large minimum / maximum value of the Rice parameter as shown in Table 15.
[0242] Meanwhile, as another embodiment, a method can be used in which one Rice parameter lookup table is used for level coding to obtain the same effect as using multiple Rice parameter lookup tables. As an example, the encoding device and the decoding device can determine the index of the Rice parameter lookup table based on whether a specific condition is satisfied. Here, whether the specific condition is satisfied can be determined based on the configuration of the syntax (presence or absence, type, etc. of syntax elements) coded prior to the syntax element indicating the level value of the transform coefficient, whether the number of context coding bins coded / decoded in the residual data coding exceeds the maximum available context coding bin number set in the context coding bin constraint algorithm, whether lossless or adjacent lossless coding is applied, whether the sum (locSumAbs) of the level values (s) of the neighboring coefficients (s) of the current transform coefficient is greater than a threshold value or the size of the table (maximum value of the Rice parameter lookup table locSumAbs), etc. The above-mentioned specific condition can also be applied to an embodiment using multiple Rice parameter lookup tables. That is, in order to derive Rice parameters for the transform coefficients, a Rice parameter lookup table may be selected from among a number of Rice parameter lookup tables depending on whether the above-mentioned specific conditions are satisfied.
[0243] In an embodiment using a single Rice parameter lookup table to derive Rice parameters for transform coefficients, the value of the Rice parameter can be determined, for example, according to the following formula when the specific conditions are met:
[0244]
number
[0245]
number
[0246] Here, RiceParamTable means a rice parameter lookup table, and index indicates an index value of the rice parameter lookup table. The index value may be determined based on locSumAbs derived by the pseudo code in Table 10, for example, or based on an index selection method set in the standard.
[0247] Referring to Equation 15 and Equation 16, if the specific condition is not met, the encoding apparatus and the decoding apparatus may derive the value by reading a Rice parameter corresponding to an index value in a Rice parameter lookup table set by default. If the specific condition is met, a shift and / or an offset may be used. Here, the shift and / or offset may have 0, a positive value, or a negative value. If the shift is a positive number, a Rice parameter value larger than a Rice parameter selected by an index in the Rice parameter lookup table may be derived. If the shift is a negative number, a Rice parameter value smaller than a Rice parameter selected by an index in the Rice parameter lookup table may be derived.
[0248] For example, in this embodiment, a Rice parameter lookup table such as the following Table 16 can be used.
[0249] [Table 16]
[0250] Referring to Table 16, if the specific condition is not met and the index is 3, a Rice parameter value of 0 is derived. If the specific condition is met and the index is 3 and the shift is 1 (offset is 0), a Rice parameter value of 1 is derived even though the index is 3.
[0251] On the other hand, if the offset is a positive number, a Rice parameter value greater than the maximum Rice parameter value of the Rice parameter lookup table can be derived.
[0252] For example, when the Rice parameter lookup table of Table 16 is used, if the index is 31 and the specific condition is met, a Rice parameter value of 3 is derived. If the specific condition is met and the offset is 2 (shift is 0), a Rice parameter value of 5 is derived even though the index is 31. That is, when an offset is used, a Rice parameter value higher than the maximum Rice parameter value defined in the Rice parameter lookup table can be derived. Therefore, although only Rice parameters of orders 0 to 3 can be derived according to the Rice parameter lookup table of Table 16, the usable range of Rice parameters can be increased to orders 0 to 5 when an offset is used.
[0253] In this way, it is possible to efficiently binarize up to high level values through the maximum Rice parameter extension. Therefore, this embodiment can have coding advantages not only in general coding environments, but also in high bit rate environments (using low quantization parameters), near lossless, lossless environments, etc. In addition, according to this embodiment, there is no need to transmit an additional Rice parameter lookup table and / or additional syntax.
[0254] 6 and 7 show a schematic diagram of an example entropy encoding method and associated components according to one embodiment of the present document.
[0255] The Rice parameter derivation method disclosed in Fig. 6 may be performed by the encoding apparatus 200 disclosed in Fig. 2 and Fig. 7. Specifically, for example, S600 to S620 in Fig. 6 may be performed by the Rice parameter derivation unit 241 of the entropy encoding unit 240. S630 in Fig. 6 may be performed by the binarization unit 242 of the entropy encoding unit 240, and S640 in Fig. 6 may be performed by the entropy encoding processing unit 243 of the entropy encoding unit 240.
[0256] The entropy encoding method disclosed in FIG. 6 may include embodiments detailed in this document.
[0257] 6 and 7, the entropy encoding unit 240 performs a residual coding procedure on the (quantized) transform coefficients. Here, the transform coefficients may be mixed with the residual coefficients. The entropy encoding unit 240 may perform residual coding on the (quantized) transform coefficients in the current block (current CB or current TB) according to a scan order. The entropy encoding unit 240 may generate and encode various syntax elements related to residual information, for example, as shown in Tables 1 to 4. As an example, the Rice parameter derivation unit 241 of the entropy encoding unit 240 may generate (or derive) information indicating a level value of a current transform coefficient (quantized transform coefficient or current (quantized) residual coefficient) in the current block (S600). Here, the information indicating the level value of the current transform coefficient may include at least one of abs_remainder[n] or dec_abs_level[n]. The value of abs_remainder[n] may be derived based on values of sig_coeff_flag[xC][yC], abs_level_gtx_flag[n][0], par_level_flag[n], abs_level_gtx_flag[n][1], etc. The value of dec_abs_level[n] may be derived as a level value of the transform coefficient. When a predetermined number of context coding bins in a corresponding block (CU or TU) according to a scanning order reaches a certain threshold, the encoding apparatus may code the level value of the subsequent transform coefficients based on dec_abs_level[n].
[0258] The Rice parameter derivation unit 241 of the entropy encoding unit 240 may configure a plurality of Rice parameter lookup tables and may select one of the tables for information indicating the level value of the transform coefficient (S610). For example, the Rice parameter derivation unit 241 of the entropy encoding unit 240 may select a Rice parameter lookup table to be used for level coding of the current transform coefficient (or coefficient group, transform block, or coding block) from a plurality of Rice parameter lookup tables by using already given information such as a configuration of syntax elements (presence or absence, type, etc. of syntax elements) to be coded prior to a syntax element indicating a level value of the current transform coefficient, whether the number of context coding bins coded / decoded in residual data coding exceeds the maximum available context coding bin number set in a context coding bin constraint algorithm, whether lossless or near lossless coding is performed, or quantization coefficient information. The selection information for the corresponding Rice parameter lookup table may be signaled implicitly or explicitly. For example, the selection information may correspond to a syntax element for transmitting information indicating a Rice parameter lookup table selected by the encoding device.
[0259] The Rice parameter derivation unit 241 of the entropy encoding unit 240 may derive a Rice parameter for information (abs_remainder[n] or dec_abs_level[n]) indicating a level value of the current transform coefficient based on the selected Rice parameter lookup table and the neighboring (or reference) transform coefficients of the current transform coefficient (S620). Specifically, the Rice parameter derivation unit 241 of the entropy encoding unit 240 may derive a Rice parameter for (the coefficient of) the current scanning position using the selected Rice parameter lookup table based on the above-mentioned locSumAbs. The locSumAbs may be derived based on AbsLevel and / or sig_coeff_flag of the neighboring transform coefficients. It is obvious to those skilled in the art that the procedure of deriving the Rice parameter may be omitted for sig_coeff_flag, par_level_flag, abs_level_gtx_flag, etc. that are binarized to a fixed length without using a Rice parameter. For the sig_coeff_flag, par_level_flag, abs_level_gtx_flag, etc., binarization other than the Rice parameter-based binarization may be performed.
[0260] The binarization unit 242 of the entropy encoding unit 240 may perform binarization based on the derived Rice parameter to derive a bin string for information indicating a level value of the transform coefficient (abs_remainder[n] or dec_abs_level[n]) (S630). The length of the bin string may be adaptively determined according to the derived Rice parameter.
[0261] The entropy encoding processor 243 of the entropy encoding unit 240 may perform entropy encoding based on a bin string for information indicating a level value of the transform coefficient (abs_remainder[n] or dec_abs_level[n]) (S640). The entropy encoding processor 243 of the entropy encoding unit 240 may perform context-based entropy encoding on the bin string based on a context-adaptive arithmetic coding (CABAC) entropy coding technique, and the output may be included in a bitstream. In this case, the encoding apparatus may derive context information for each bin of the bin string, perform entropy coding, and update the context information for each bin. The bitstream may include various information for image / video decoding, such as prediction information, in addition to residual information including information on abs_remainder[n] or dec_abs_level[n] as described above. The bitstream may further include selection information for the Rice parameter table. The bitstream can be transmitted to a decoding device via a (digital) storage medium or a network.
[0262] FIG. 8 illustrates a schematic diagram of an example of an entropy encoding method according to another embodiment of the present document.
[0263] The Rice parameter derivation method disclosed in Fig. 8 may be performed by the encoding apparatus 200 disclosed in Fig. 2 and Fig. 7. Specifically, for example, S800 to S820 in Fig. 8 may be performed by the Rice parameter derivation unit 241 of the entropy encoding unit 240. S830 in Fig. 8 may be performed by the binarization unit 242 of the entropy encoding unit 240, and S840 in Fig. 8 may be performed by the entropy encoding processing unit 243 of the entropy encoding unit 240.
[0264] The entropy encoding method disclosed in FIG. 8 may include embodiments detailed in this document.
[0265] 7 and 8, the entropy encoding unit 240 performs a residual coding procedure on the (quantized) transform coefficients. Here, the transform coefficients may be mixed with the residual coefficients. The entropy encoding unit 240 may perform residual coding on the (quantized) transform coefficients in the current block (current CB or current TB) according to a scan order. The entropy encoding unit 240 may generate and encode various syntax elements related to residual information, for example, as shown in Tables 1 to 4. As an example, the Rice parameter derivation unit 241 of the entropy encoding unit 240 may generate (or derive) information indicating a level value of a current transform coefficient (quantized transform coefficient or current (quantized) residual coefficient) in the current block (S800). Here, the information indicating the level value of the current transform coefficient may include at least one of abs_remainder[n] or dec_abs_level[n]. The value of abs_remainder[n] may be derived based on values of sig_coeff_flag[xC][yC], abs_level_gtx_flag[n][0], par_level_flag[n], abs_level_gtx_flag[n][1], etc. The value of dec_abs_level[n] may be derived as a level value of the transform coefficient. When a predetermined number of context coding bins in a corresponding block (CU or TU) according to a scanning order reaches a certain threshold, the encoding apparatus may code the level value of the subsequent transform coefficients based on dec_abs_level[n].
[0266] The Rice parameter derivation unit 241 of the entropy encoding unit 240 may determine an index value of the Rice lookup table for information indicating the level value of the transform coefficient (S810). For example, the Rice parameter derivation unit 241 of the entropy encoding unit 240 may determine an index value of the Rice parameter lookup table based on a configuration (presence or absence, type, etc.) of a syntax element coded prior to a syntax element indicating a level value of a current transform coefficient, whether the number of context coding bins coded / decoded in residual data coding exceeds the maximum available context coding bin number set in a context coding bin constraint algorithm, whether lossless or near lossless coding is performed, quantization coefficient information (value) or whether the sum of the level values of the peripheral coefficients of the current transform coefficient is greater than a threshold value (or the size of the Rice parameter lookup table), etc., or may change the corresponding index value by adding a shift to the index value.
[0267] The Rice parameter derivation unit 241 of the entropy encoding unit 240 may derive a Rice parameter for information indicating a level value of the current transform coefficient (abs_remainder[n] or dec_abs_level[n]) from the Rice parameter lookup table based on the index value (S820). For example, the Rice parameter derivation unit 241 of the entropy encoding unit 240 may derive a Rice parameter for (a coefficient at) the current scanning position using the above-described Equation 15 and / or Equation 16 based on the index value. When Equation 16 is used, the value of the Rice parameter may be changed to a value obtained by adding an offset value to the index value. The index value may be derived based on the above-described locSumAbs or may be determined based on an index selection method set by default. The locSumAbs may be derived based on AbsLevel and / or sig_coeff_flag of the surrounding transform coefficients.
[0268] It is obvious to those skilled in the art that the procedure for deriving the Rice parameters can be omitted for sig_coeff_flag, par_level_flag, abs_level_gtx_flag, etc., which are binarized to a fixed length without using the Rice parameters. For the sig_coeff_flag, par_level_flag, abs_level_gtx_flag, etc., binarization using other methods other than Rice parameter-based binarization can be performed.
[0269] The binarization unit 242 of the entropy encoding unit 240 may perform binarization based on the derived Rice parameter to derive a bin string for information indicating a level value of the transform coefficient (abs_remainder[n] or dec_abs_level[n]) (S830). The length of the bin string may be adaptively determined according to the derived Rice parameter.
[0270] The entropy encoding processor 243 of the entropy encoding unit 240 may perform entropy encoding based on a bin string for information indicating a level value of the transform coefficient (abs_remainder[n] or dec_abs_level[n]) (S840). The entropy encoding processor 243 of the entropy encoding unit 240 may perform context-based entropy encoding on the bin string based on a context-adaptive arithmetic coding (CABAC) entropy coding technique, and the output may be included in a bitstream. In this case, the encoding apparatus may derive context information for each bin of the bin string, perform entropy coding, and update the context information for each bin. The bitstream may include various information for image / video decoding, such as prediction information, in addition to residual information including information on abs_remainder[n] or dec_abs_level[n] as described above. The bitstream may further include selection information for the Rice parameter table. The bitstream can be transmitted to a decoding device via a (digital) storage medium or a network.
[0271] 9 and 10 show a schematic diagram of an example of an entropy decoding method and associated components according to an embodiment of the present document.
[0272] The Rice parameter derivation method disclosed in Fig. 9 may be performed by the decoding apparatus 300 disclosed in Fig. 3 and Fig. 10. Specifically, for example, S900 to S920 in Fig. 9 may be performed by the Rice parameter derivation unit 311 of the entropy decoding unit 310. S930 in Fig. 9 may be performed by the binarization unit 312 of the entropy decoding unit 310, and S940 in Fig. 9 may be performed by the entropy decoding processing unit 313 of the entropy decoding unit 310.
[0273] The entropy decoding method disclosed in FIG. 9 may include the embodiments detailed in this document.
[0274] 9 and 10, the entropy decoding unit may decode the encoded residual information to derive the (quantized) transform coefficients. Here, the transform coefficients may be mixed with the residual coefficients. The decoding device may decode the encoded residual information for the current block (current CB or current TB) to derive the (quantized) transform coefficients. The decoding device may decode various syntax elements related to the residual information, for example, as shown in Tables 1 to 4, analyze the values of the related syntax elements, and derive the (quantized) transform coefficients based on the values.
[0275] Specifically, the Rice parameter derivation unit 311 of the entropy decoding unit 310 may obtain information (abs_remainder[n] or dec_abs_level[n]) indicating a level value of a current transform coefficient (a quantized transform coefficient or a current (quantized) residual coefficient) from a bitstream (S900), and may select a Rice parameter lookup table for the information indicating the level value from among a plurality of Rice parameter lookup tables (S910).
[0276] For example, the Rice parameter derivation unit 311 of the entropy decoding unit 310 may configure a plurality of Rice parameter lookup tables and select one of the tables. For this purpose, selection information for selecting one of the tables may be explicitly signaled. The selection information may correspond to a syntax element for transmitting information indicating the Rice parameter lookup table selected by the encoding apparatus. In this case, the Rice parameter derivation unit 311 of the entropy decoding unit 310 may obtain a syntax element indicating information regarding the Rice parameter lookup table from the bitstream and select one of the plurality of Rice parameter lookup tables based on the obtained syntax element.
[0277] Alternatively, the Rice parameter derivation unit 311 of the entropy decoding unit 310 may select a Rice parameter lookup table to be used for level coding of a current transform coefficient (or a coefficient group, or a transform block, or a coding block) from among a number of Rice parameter lookup tables based on at least one of previously given information such as a configuration of a syntax element (presence or absence of a syntax element, type, etc.) coded preceding a syntax element indicating a level value of a current transform coefficient, information on the maximum available context coding bin number set by a context coding bin constraint algorithm, whether a current block is lossless or near lossless coded, or quantization coefficient information for a current transform coefficient, etc. Here, the syntax element coded preceding a syntax element indicating a level value of the current transform coefficient may include at least one of sig_coeff_flag, abs_level_gtx_flag, par_level_flag, or coeff_sign_flag, and a Rice parameter lookup table may be selected based on whether at least one of them is decoded.
[0278] The Rice parameter derivation unit 311 of the entropy decoding unit 310 may derive a Rice parameter for information (abs_remainder[n] or dec_abs_level[n]) indicating a level value of the current transform coefficient based on the selected Rice parameter lookup table and the neighboring (or reference) transform coefficients of the current transform coefficient (S920). Specifically, the Rice parameter derivation unit 311 of the entropy decoding unit 310 may derive a Rice parameter for (the coefficient of) the current scanning position using the selected Rice parameter lookup table based on the above-mentioned locSumAbs. The locSumAbs may be derived based on AbsLevel and / or sig_coeff_flag of the neighboring transform coefficients. It is obvious to those skilled in the art that the procedure of deriving the Rice parameter may be omitted for sig_coeff_flag, par_level_flag, abs_level_gtx_flag, etc. that are binarized to a fixed length without using a Rice parameter. For the sig_coeff_flag, par_level_flag, abs_level_gtx_flag, etc., binarization other than the Rice parameter-based binarization may be performed.
[0279] The binarization unit 312 of the entropy decoding unit 310 may perform binarization based on the derived Rice parameter to derive a bin string for information indicating a level value of the transform coefficient (abs_remainder[n] or dec_abs_level[n]) (S930). As an example, the binarization unit 312 of the entropy decoding unit 310 may derive an available bin string for an available value of the abs_remainder[n] or dec_abs_level[n] through the binarization procedure. The length of the available bin string may be adaptively determined according to the derived Rice parameter.
[0280] The entropy decoding processor 313 of the entropy decoding unit 310 may derive the level value of the transform coefficient by performing entropy decoding based on a bin string for information (abs_remainder[n] or dec_abs_level[n]) indicating the level value of the transform coefficient (S940). For example, the entropy decoding processor 313 of the entropy decoding unit 310 may sequentially parse and decode each bin / bit for the abs_remainder[n] or dec_abs_level[n] and compare the derived bin string with the available bin string. If the derived bin string is the same as one of the available bin strings, a value corresponding to the corresponding bin string may be derived as the value of the abs_remainder[n]. If not, the comparison procedure may be performed after further parsing and decoding the next bit in the bitstream. Through this process, it is possible to signal specific information (specific syntax element) using variable length bits without using start bits or end bits for the specific information in the bitstream. This allows relatively fewer bits to be allocated to low values, improving overall coding efficiency.
[0281] The decoding apparatus may perform context-based entropy decoding on each bin in the bin string from the bitstream based on the CABAC entropy coding technique. In this case, the decoding apparatus may derive context information for each bin of the bin string, perform entropy coding, and update the context information for each bin. The entropy decoding procedure may be performed by the entropy decoding processor 313 in the entropy decoding unit 310. As described above, the bitstream may include various information for image / video decoding, such as prediction information in addition to residual information including information on abs_remainder[n] or dec_abs_level[n]. The bitstream may further include selection information for a Rice parameter lookup table. As described above, the bitstream may be transmitted to the decoding apparatus via a (digital) storage medium or a network.
[0282] The decoding apparatus may derive (quantized) transform / residual coefficients based on the entropy decoding, and may derive residual samples for the current block by performing inverse quantization and / or inverse transform procedures based on the transform / residual coefficients, if necessary. Reconstructed samples may be generated based on the residual samples and prediction samples derived through inter prediction and / or intra prediction, and a reconstructed block / picture including the reconstructed samples may be generated.
[0283] FIG. 11 illustrates a schematic diagram of an example of an entropy decoding method according to another embodiment of the present document.
[0284] The Rice parameter derivation method disclosed in Fig. 11 may be performed by the decoding apparatus 300 disclosed in Fig. 3 and Fig. 10. Specifically, for example, S1100 to S1120 in Fig. 11 may be performed by the Rice parameter derivation unit 311 of the entropy decoding unit 310. S1130 in Fig. 11 may be performed by the binarization unit 312 of the entropy decoding unit 310, and S1140 in Fig. 11 may be performed by the entropy decoding processing unit 313 of the entropy decoding unit 310.
[0285] The entropy decoding method disclosed in FIG. 11 may include the embodiments detailed in this document.
[0286] 10 and 11, the entropy decoding unit may decode the encoded residual information to derive the (quantized) transform coefficients. Here, the transform coefficients may be mixed with the residual coefficients. The decoding device may decode the encoded residual information for the current block (current CB or current TB) to derive the (quantized) transform coefficients. The decoding device may decode various syntax elements related to the residual information, for example, as shown in Tables 1 to 4, analyze the values of the related syntax elements, and derive the (quantized) transform coefficients based on the values.
[0287] Specifically, the Rice parameter derivation unit 311 of the entropy decoding unit 310 may obtain information (abs_remainder[n] or dec_abs_level[n]) indicating a level value of a current transform coefficient (quantized transform coefficient or current (quantized) residual coefficient) from a bitstream (S1100). Then, the Rice parameter derivation unit 311 of the entropy decoding unit 310 may determine an index value of a Rice parameter lookup table for the information indicating the level value (S1110). For example, the Rice parameter derivation unit 311 of the entropy decoding unit 310 may determine and / or change an index value of a Rice parameter lookup table based on at least one of a configuration of syntax elements (presence or absence, type, etc. of syntax elements) coded prior to the information indicating the level value of the current transform coefficient, the number of maximum available context coding bins set in a context coding bin constraint algorithm, whether lossless or near lossless coding is performed, quantization coefficient information (value) for the transform coefficient, or level value(s) of neighboring coefficient(s) of the current transform coefficient.
[0288] For example, the Rice parameter derivation unit 311 of the entropy decoding unit 310 may determine the index value or change the index value by adding a shift to the index value based on whether at least one of syntax elements sig_coeff_flag, abs_level_gtx_flag, par_level_flag, or coeff_sign_flag is decoded from a bitstream, or may determine the index value or change the index value by adding a shift to the index value based on whether a sum of level values of neighboring transform coefficients of the transform coefficient is equal to or greater than a threshold.
[0289] The Rice parameter derivation unit 311 of the entropy decoding unit 310 may derive a Rice parameter for information indicating a level value of the current transform coefficient (abs_remainder[n] or dec_abs_level[n]) from the Rice parameter lookup table based on the index value (S1120). For example, the Rice parameter derivation unit 311 of the entropy decoding unit 310 may derive a Rice parameter for (a coefficient of) the current scanning position using the above-mentioned Equation 15 and / or Equation 16 based on the index value. When Equation 16 is used, the value of the Rice parameter may be changed to a value obtained by adding an offset value to the index value. The index value may be derived based on the above-mentioned locSumAbs or may be determined based on an index selection method set by default.
[0290] It is obvious to those skilled in the art that the procedure for deriving the Rice parameters can be omitted for sig_coeff_flag, par_level_flag, abs_level_gtx_flag, etc., which are binarized to a fixed length without using the Rice parameters. For the sig_coeff_flag, par_level_flag, abs_level_gtx_flag, etc., binarization using other methods other than Rice parameter-based binarization can be performed.
[0291] The binarization unit 312 of the entropy decoding unit 310 may perform binarization based on the derived Rice parameter to derive a bin string for information indicating a level value of the transform coefficient (abs_remainder[n] or dec_abs_level[n]) (S1130). As an example, the binarization unit 312 of the entropy decoding unit 310 may derive an available bin string for an available value of the abs_remainder[n] or dec_abs_level[n] through the binarization procedure. The length of the available bin string may be adaptively determined according to the derived Rice parameter.
[0292] The entropy decoding processor 313 of the entropy decoding unit 310 may derive the level value of the transform coefficient by performing entropy decoding based on a bin string for information (abs_remainder[n] or dec_abs_level[n]) indicating the level value of the transform coefficient (S940). For example, the entropy decoding processor 313 of the entropy decoding unit 310 may sequentially parse and decode each bin / bit for the abs_remainder[n] or dec_abs_level[n] and compare the derived bin string with the available bin string. If the derived bin string is the same as one of the available bin strings, a value corresponding to the corresponding bin string may be derived as the value of the abs_remainder[n]. If not, the comparison procedure may be performed after further parsing and decoding the next bit in the bitstream. Through this process, it is possible to signal specific information (specific syntax element) using variable length bits without using start bits or end bits for the specific information in the bitstream. This allows relatively fewer bits to be allocated to low values, improving overall coding efficiency.
[0293] The decoding apparatus may perform context-based entropy decoding on each bin in the bin string from the bitstream based on the CABAC entropy coding technique. In this case, the decoding apparatus may derive context information for each bin of the bin string, perform entropy coding, and update the context information for each bin. The entropy decoding procedure may be performed by the entropy decoding processor 313 in the entropy decoding unit 310. As described above, the bitstream may include various information for image / video decoding, such as prediction information in addition to residual information including information on abs_remainder[n] or dec_abs_level[n]. The bitstream may further include selection information for a Rice parameter lookup table. As described above, the bitstream may be transmitted to the decoding apparatus via a (digital) storage medium or a network.
[0294] The decoding apparatus may derive (quantized) transform / residual coefficients based on the entropy decoding, and may derive residual samples for the current block by performing inverse quantization and / or inverse transform procedures based on the transform / residual coefficients, if necessary. Reconstructed samples may be generated based on the residual samples and prediction samples derived through inter prediction and / or intra prediction, and a reconstructed block / picture including the reconstructed samples may be generated.
[0295] FIG. 12 illustrates a video / image encoding method according to an embodiment of the present document.
[0296] The video / image encoding method disclosed in Fig. 12 may be performed by the encoding apparatus 200 disclosed in Fig. 2. Specifically, for example, S1200 in Fig. 12 may be performed by the prediction unit 220 of the encoding apparatus, and S1210 may be performed by the subtraction unit 231 of the encoding apparatus. S1220 may be performed by the transformation unit 232 of the encoding apparatus, S1230 may be performed by the quantization unit 233 of the encoding apparatus, and S1240 may be performed by the entropy encoding unit 240 of the encoding apparatus. S800 to S830 detailed in Fig. 8 may be included in the S1240 procedure.
[0297] 12, an encoding apparatus may derive a prediction sample through prediction for a current block (S1200). The encoding apparatus may determine whether to perform inter prediction or intra prediction for the current block, and may determine a specific inter prediction mode or a specific intra prediction mode based on an RD cost. Depending on the determined mode, the encoding apparatus may derive a prediction sample for the current block.
[0298] The encoding apparatus may derive residual samples by comparing original samples and the predicted samples for the current block (S1210).
[0299] The encoding apparatus derives transform coefficients through a transform procedure on the residual samples (S1220), and quantizes the derived transform coefficients to derive quantized transform coefficients (S1230).
[0300] The encoding apparatus may encode image information including prediction information and residual information, and output the encoded image information in the form of a bitstream (S1240). The prediction information may include prediction mode information and information on motion information (e.g., when inter-prediction is applied) as information related to the prediction procedure. The residual information may include, for example, the information disclosed in Tables 1 to 4 above as information on the quantized transform coefficients.
[0301] The output bitstream can be transmitted to a decoding device via a storage medium or a network.
[0302] FIG. 13 illustrates a video / image decoding method according to an embodiment of the present document.
[0303] The video / image decoding method disclosed in FIG. 13 may be performed by the decoding apparatus 300 disclosed in FIG. 3. Specifically, for example, S1300 in FIG. 13 may be performed by the prediction unit 330 of the decoding apparatus. In S1300, the procedure of decoding prediction information included in the bitstream to derive values of related syntax elements may be performed by the entropy decoding unit 310 of the decoding apparatus. S1310, S1320, S1330, and S1340 may be performed by the entropy decoding unit 310, the inverse quantization unit 321, the inverse transform unit 322, and the addition unit 340 of the decoding apparatus, respectively. S1000 to S1030 detailed in FIG. 10 may be included in the S1310 procedure.
[0304] The decoding apparatus may perform operations corresponding to those performed by the encoding apparatus. The decoding apparatus may derive a prediction sample by performing inter prediction or intra prediction on a current block based on received prediction information (S1300).
[0305] The decoding apparatus may derive quantized transform coefficients for the current block based on the received residual information (S1310).
[0306] The decoding apparatus can derive the transform coefficients by inverse quantizing the quantized transform coefficients (S1320).
[0307] The decoding apparatus can derive residual samples through an inverse transform procedure on the transform coefficients (S1330).
[0308] The decoding apparatus may generate a reconstructed sample for the current block based on the predicted sample and the residual sample, and may generate a reconstructed picture based on the reconstructed sample (S1340). Thereafter, an in-loop filtering procedure may be further applied to the reconstructed picture, as described above.
[0309] In the above embodiments, the method is described based on a flow chart with a series of steps or blocks, but the embodiments are not limited to the order of steps, and certain steps may occur in a different order or simultaneously with other steps than those described. In addition, those skilled in the art will understand that the steps shown in the flow chart are not exclusive, and other steps may be included, or one or more steps of the flow chart may be deleted without affecting the scope of the embodiments of this document.
[0310] The methods according to the embodiments of the present document described above may be embodied in software form, and the encoding device and / or decoding device according to the present document may be included in a device that performs video processing, such as a TV, a computer, a smartphone, a set-top box, or a display device.
[0311] In this document, when the embodiments are implemented in software, the methods described above may be implemented with modules (processes, functions, etc.) that perform the functions described above. The modules may be stored in a memory and executed by a processor. The memory may be internal or external to the processor and may be coupled to the processor in various well-known ways. The processor may include an application-specific integrated circuit (ASIC), other chipsets, logic circuits, and / or data processing devices. The memory may include a read-only memory (ROM), a random access memory (RAM), a flash memory, a memory card, a storage medium, and / or other storage devices. That is, the embodiments described in this document may be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional units shown in each drawing may be implemented and executed on a computer, processor, microprocessor, controller, or chip. In this case, information (e.g., information on instructions) or algorithms for implementation may be stored on a digital storage medium.
[0312] In addition, the decoding device and encoding device to which the embodiment(s) of this document are applied may be included in a multimedia broadcast transmitting / receiving device, a mobile communication terminal, a home cinema video device, a digital cinema video device, a surveillance camera, a video interactive device, a real-time communication device such as video communication, a mobile streaming device, a storage medium, a camcorder, a custom video (VoD) service providing device, an over-the-top video (OTT) device, an internet streaming service providing device, a three-dimensional (3D) video device, a virtual reality (VR) device, an argument reality (AR) device, an image telephone video device, a transportation means terminal (e.g., a vehicle (including an autonomous vehicle) terminal, an airplane terminal, a ship terminal, etc.), and a medical video device, and may be used to process a video signal or a data signal. For example, an over-the-top video (OTT) device may include a game console, a Blu-ray player, an internet-connected TV, a home theater system, a smartphone, a tablet PC, a digital video recorder (DVR), etc.
[0313] In addition, the processing method to which the embodiment(s) of the present document are applied may be produced in the form of a program executed by a computer and may be stored in a computer-readable recording medium. In addition, multimedia data having a data structure according to the embodiment(s) of the present document may also be stored in a computer-readable recording medium. The computer-readable recording medium includes all kinds of storage devices and distributed storage devices in which computer-readable data is stored. The computer-readable recording medium may include, for example, a Blu-ray Disc (BD), a Universal Serial Bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. In addition, the computer-readable recording medium may include a medium embodied in the form of a carrier wave (e.g., transmission via the Internet). In addition, a bitstream generated by the encoding method may be stored in a computer-readable recording medium or transmitted via a wired or wireless communication network.
[0314] Furthermore, the embodiment(s) of the present document may be embodied in a computer program product by program code, the program code being executable by a computer in accordance with the embodiment(s) of the present document, and the program code may be stored on a computer readable carrier.
[0315] FIG. 14 illustrates an example of a content streaming system in which the embodiments disclosed herein can be applied.
[0316] Referring to FIG. 14, a content streaming system to which the embodiments of this document are applied may include broadly an encoding server, a streaming server, a web server, a media repository, a user device, and a multimedia input device.
[0317] The encoding server compresses content input from a multimedia input device such as a smartphone, camera, camcorder, etc. into digital data to generate a bitstream and transmits the bitstream to the streaming server. As another example, if a multimedia input device such as a smartphone, camera, camcorder, etc. directly generates a bitstream, the encoding server may be omitted.
[0318] The bitstream may be generated by an encoding method or a bitstream generation method applied to an embodiment of this document, and the streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.
[0319] The streaming server transmits multimedia data to a user device based on a user request via a web server, and the web server acts as a medium to inform the user of available services. When a user requests a desired service from the web server, the web server transmits the request to the streaming server, and the streaming server transmits multimedia data to the user. In this case, the content streaming system may include a separate control server, and in this case, the control server controls commands / responses between devices in the content streaming system.
[0320] The streaming server can receive content from a media repository and / or an encoding server. For example, when the content is received from the encoding server, the content can be received in real time. In this case, the streaming server can store the bitstream for a certain period of time to provide a smooth streaming service.
[0321] Examples of the user devices include mobile phones, smart phones, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glass, head mounted displays (HMDs), digital TVs, desktop computers, and digital signage.
[0322] Each server in the content streaming system can be operated as a distributed server, in which case data received by each server can be processed in a distributed manner.
Claims
1. 1. A video decoding method performed by a decoding device, comprising: receiving a bitstream comprising information related to level values of transform coefficients in a current block; determining an index value of a Rice parameter lookup table for said information related to said level values of said transform coefficients; deriving a Rice parameter for the information related to the level value of the transform coefficient based on the Rice parameter lookup table and the index value; deriving a bin string for the information related to the level values of the transform coefficients based on the Rice parameters; and deriving the level values of the transform coefficients based on the bin string; The step of deriving the Rice parameters comprises: deriving temporary Rice parameters based on the index values and the Rice parameter lookup table; and deriving the Rice parameters by adding an offset to the temporary Rice parameters.
2. The step of deriving the Rice parameters comprises: deriving a modified index value based on the index value; deriving the Rice parameters utilizing the modified index values and the Rice parameter lookup table; The video decoding method of claim 1 , wherein the step of deriving the modified index value comprises the step of deriving the modified index value based on whether a sum of level values of adjacent transform coefficients of the transform coefficient is less than a threshold.
3. The method of claim 2 , wherein deriving the modified index value comprises deriving the modified index value based on the index value and a shift value.
4. The step of deriving the Rice parameters comprises: deriving a modified index value based on the index value and a shift value; deriving temporary Rice parameters based on the modified index values and the Rice parameter lookup table; 2. The video decoding method of claim 1, further comprising the step of: deriving the Rice parameters by adding an offset to the temporary Rice parameters.
5. 5. The video decoding method of claim 4, wherein the modified index value is derived using the shift value and the Rice parameter is derived using the offset based on a specific syntax element that is coded before the information related to the level value of the transform coefficient.
6. The video decoding method of claim 5 , wherein the index value is derived based on a sum of level values of neighboring transform coefficients of the transform coefficient.
7. The video decoding method of claim 1 , wherein the information related to the level value of the transform coefficient includes a syntax element dec_abs_level indicating the level value of the transform coefficient or a syntax element abs_remainder indicating a remaining level value of the transform coefficient.
8. 1. A video encoding method performed by an encoding device, comprising: generating information related to the level values of the transform coefficients in the current block; determining an index value of a Rice parameter lookup table for said information related to said level values of said transform coefficients; deriving a Rice parameter for the information related to the level value of the transform coefficient based on the index value and the Rice parameter lookup table; deriving a bin string for the information related to the level values of the transform coefficients based on the Rice parameters; encoding the bin string; The step of deriving the Rice parameters comprises: deriving temporary Rice parameters based on the index values and the Rice parameter lookup table; and deriving the Rice parameters by adding an offset to the temporary Rice parameters.
9. The step of deriving the Rice parameters comprises: deriving a modified index value based on the index value; deriving the Rice parameters using the modified index values and the Rice parameter lookup table; The video encoding method of claim 8 , wherein the step of deriving the modified index value comprises the step of deriving the modified index value based on whether a sum of level values of adjacent transform coefficients of the transform coefficient is smaller than a threshold value.
10. The video encoding method of claim 9 , wherein deriving the modified index value comprises deriving the modified index value based on the index value and a shift value.
11. The step of deriving the Rice parameters comprises: deriving a modified index value based on the index value and a shift value; deriving temporary Rice parameters based on the modified index values and the Rice parameter lookup table; 9. The video encoding method of claim 8, further comprising the step of: deriving the Rice parameters by adding an offset to the temporary Rice parameters.
12. The video encoding method of claim 11, wherein the modified index value is derived using the shift value and the Rice parameter is derived using the offset based on a specific syntax element that is coded before the information related to the level value of the transform coefficient.
13. The video encoding method of claim 12 , wherein the index value is derived based on a sum of level values of neighboring transform coefficients of the transform coefficient.
14. 9. The video encoding method of claim 8, wherein the information related to the level value of the transform coefficient comprises a syntax element dec_abs_level indicating the level value of the transform coefficient, or a syntax element abs_remainder indicating a remaining level value of the transform coefficient.
15. 1. A method for transmitting data for a video, comprising: Obtaining a bitstream for the video, comprising: The bitstream comprises: generating information related to the level values of the transform coefficients in the current block; determining an index value of a Rice parameter lookup table for said information related to said level values of said transform coefficients; deriving a Rice parameter for the information related to the level value of the transform coefficient based on the index value and the Rice parameter lookup table; deriving a bin string for the information related to the level values of the transform coefficients based on the Rice parameters; encoding the bin string; transmitting the data including the bitstream; The step of deriving the Rice parameters comprises: deriving temporary Rice parameters based on the index values and the Rice parameter lookup table; and deriving the Rice parameter by adding an offset to the temporary Rice parameter.
Citation Information
Patent Citations
High throughput coding for cabac in hevc
WO2013108639A1
Method, apparatus and system for encoding and decoding a subset of transform units of encoded video data
WO2013152401A1
Rice parameter initialization for coefficient level coding in video coding process
WO2015006602A2
Coefficient level coding in video coding
WO2017040828A1