Image decoding method and apparatus therefor

Abstract

To provide an image decoding method.SOLUTION: An image decoding method implemented with a decoding apparatus according to the present document comprises the steps of: acquiring image information including a block-based delta pulse code modulation (BDPCM) constraint flag indicating whether BDPCM is constrained with respect to an image; acquiring BDPCM-associated information regarding a current block on the basis of the BDPCM constraint flag; and generating reconstructed samples for the current block on the basis of the BDPCM-associated information.SELECTED DRAWING: Figure 11

Description

Technical Field

[0001] This document relates to image coding technology, and more particularly, to an image decoding method and apparatus for coding a current block performing BDPCM in an image coding system.

Background Art

[0002] Recently, the demand for high-resolution and high-quality images such as HD (High Definition) images and UHD (Ultra High Definition) images has been increasing in various fields. As the image data becomes higher in resolution and quality, the amount of information or bits transmitted relatively increases compared to existing image data. Therefore, when transmitting image data using a medium such as an existing wired or wireless broadband line, or storing image data using an existing storage medium, the transmission cost and storage cost increase.

[0003] Accordingly, a highly efficient image compression technology is required to effectively transmit, store, and reproduce information of high-resolution and high-quality images.

Summary of the Invention

Problems to be Solved by the Invention

[0004] The technical problem of this document is to provide a method and apparatus for increasing image coding efficiency.

[0005] Another technical problem of this document is to provide a method and apparatus for increasing the efficiency of BDPCM.

Means for Solving the Problems

[0006] According to an embodiment of this document, an image decoding method performed by a decoding device is provided. The method includes: obtaining image information including a BDPCM constraint flag indicating whether BDPCM (Block-based Delta Pulse Code Modulation) is constrained for an image; obtaining BDPCM-related information for a current block based on the BDPCM constraint flag; and generating a restored sample for the current block based on the BDPCM-related information.

[0007] According to another embodiment of this document, a decoding device for performing image decoding is provided. The decoding device includes an entropy decoding unit that obtains image information including a BDPCM constraint flag indicating whether BDPCM (Block-based Delta Pulse Code Modulation) is constrained for an image, and obtains BDPCM-related information for a current block based on the BDPCM constraint flag, and an addition unit that generates a restored sample for the current block based on the BDPCM-related information.

[0008] According to still another embodiment of this document, a video encoding method performed by an encoding device is provided. The method includes: generating a BDPCM constraint flag indicating whether BDPCM (Block-based Delta Pulse Code Modulation) is constrained for an image; and encoding image information including the BDPCM constraint flag, where when the value of the BDPCM constraint flag is 1, the BDPCM constraint flag indicates that the BDPCM is not available for the image, and when the value of the BDPCM constraint flag is 0, the BDPCM constraint flag indicates that the BDPCM is not constrained for the image.

[0009] According to still another embodiment of the present document, a video encoding apparatus is provided. The encoding apparatus includes a prediction unit that generates a BDPCM constraint flag indicating whether BDPCM (Block-based Delta Pulse Code Modulation) is restricted for an image, and an entropy encoding unit that encodes image information including the BDPCM constraint flag. When the value of the BDPCM constraint flag is 1, the BDPCM constraint flag indicates that BDPCM cannot be used for the image, and when the value of the BDPCM constraint flag is 0, the BDPCM constraint flag indicates that BDPCM is not restricted for the image.

Advantages of the Invention

[0010] According to the present document, the BDPCM flag can be signaled based on the current block size and the maximum transform block size, and the current block size and the maximum transform block size can be considered for BDPCM flag signaling and BDPCM applicability determination. Through this, the amount of bits for BDPCM can be reduced, and the overall coding efficiency can be improved.

[0011] According to the present document, a syntax element indicating whether BDPCM is restricted for an image can be signaled, and through this, it is possible to determine whether BDPCM can be executed for an image with one syntax element, and the overall image coding efficiency can be improved.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Mode for Carrying Out the Invention

[0013] This document can be modified in various ways, can have various embodiments, and will illustrate and describe in detail specific embodiments with reference to the drawings. However, this is not intended to limit this document to specific embodiments. The terms commonly used in this specification are merely used to describe specific embodiments and are not used with the intention of limiting the technical idea of this document. Singular expressions include plural expressions unless the context clearly indicates otherwise. Terms such as "including" or "having" in this specification are intended to specify the existence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and it should be understood that the existence or possibility of addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof, etc. is not precluded in advance.

[0014] On the other hand, each configuration in the drawings described in this document is independently illustrated for the convenience of explaining different characteristic functions, and it does not mean that each configuration is realized by separate hardware or separate software. For example, among each configuration, two or more configurations can be combined to form one configuration, and one configuration can also be divided into a plurality of configurations. Embodiments in which each configuration is integrated and / or separated are included in the scope of rights of this document as long as they do not deviate from the essence of this document.

[0015] Hereinafter, with reference to the accompanying drawings, preferred embodiments of this document will be described in more detail. Hereinafter, the same reference numerals will be used for the same components in the drawings, and overlapping descriptions of the same components can be omitted.

[0016] FIG. 1 schematically shows an example of a video / image coding system to which embodiments of this document can be applied.

[0017] As shown in FIG. 1, a video / image coding system can include a first device (source device) and a second device (receiver device). The source device can transmit encoded video / image information or data in a file or streaming form to the receiver device via a digital storage medium or a network.

[0018] The source device can include a video source, an encoding device, and a transmitter. The receiver device can include a receiver, a decoding device, and a renderer. The encoding device can be referred to as a video / image encoding device, and the decoding device can be referred to as a video / image decoding device. A transmitter can be included in the encoding device. A receiver can be included in the decoding device. The renderer can include a display unit, and the display unit can also be composed of a separate device or an external component.

[0019] The video source can obtain video / images through processes such as capture, synthesis, or generation of video / images. The video source can include a video / image capture device and / or a video / image generation device. The video / image capture device can include, for example, one or more cameras, a video / image archive containing previously captured video / images, etc. The video / image generation device can include, for example, a computer, a tablet, and a smartphone, etc., and can (electronically) generate video / images. For example, virtual video / images can be generated via a computer or the like, in which case the video / image capture process can be replaced during the process of generating related data.

[0020] The encoding device can encode input video / images. The encoding device can execute a series of procedures such as prediction, transformation, quantization, etc. for compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0021] The transmitting unit can transmit the encoded video / image information or data output in the form of a bitstream to the receiving unit of the receiving device via a digital storage medium or network in file or streaming form. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitting unit can include elements for generating media files via a predetermined file format and can include elements for transmission via a broadcast / communication network. The receiving unit can receive / extract the bitstream and transmit it to the decoding device.

[0022] The decoding device can execute a series of procedures such as inverse quantization, inverse transformation, prediction, etc. corresponding to the operation of the encoding device to decode the video / image.

[0023] The renderer can render the decoded video / image. The rendered video / image can be displayed via the display unit.

[0024] This document relates to video / image coding. For example, the methods / embodiments disclosed in this document can be applied to the methods disclosed in the VVC (versatile video coding) standard, EVC (essential video coding) standard, AV1 (AOMedia Video 1) standard, AVS2 (2nd generation of audio video coding standard), or next-generation video / image coding standards (e.g., H.267 or H.268, etc.).

[0025] This document presents various embodiments related to video / image coding, and unless otherwise mentioned, the embodiments can also be implemented in combination with each other.

[0026] In this document, "video" may mean a collection of a series of "images" over time. "Picture" generally means a unit indicating one image in a specific time period, and "subpicture / slice / tile" is a unit that constitutes a part of a picture in coding. A subpicture / slice / tile may include one or more CTUs (Coding Tree Units). One picture may be composed of one or more subpictures / slices / tiles. One picture may be composed of one or more groups of tiles. One tile group may include one or more tiles. A brick represents a rectangular region of CTU rows within a tile in a picture. A tile may be partitioned into multiple bricks, each of which consists of one or more CTU rows within the tile. A tile that is not partitioned into multiple bricks may also be referred to as a brick.A brick scan shows a specific sequential ordering of CTUs partitioning a picture in which the CTUs are ordered consecutively in CTU raster scan in a brick, bricks within a tile are ordered consecutively in a raster scan of the bricks of the tile, and tiles in a picture are ordered consecutively in a raster scan of the tiles of the picture. Also, a subpicture may represent a rectangular region of one or more slices within a picture. That is, a subpicture contains one or more slices that collectively cover a rectangular region of a picture. A tile is a rectangular region of CTUs within a particular tile column and a particular tile row in a picture.The tile column is a rectangular region of CTUs having a height equal to the height of the picture and a width specified by syntax elements in the picture parameter set. The tile row is a rectangular region of CTUs having a height specified by syntax elements in the picture parameter set and a width equal to the width of the picture. A tile scan is a specific sequential ordering of CTUs partitioning a picture in which the CTUs are ordered consecutively in CTU raster scan in a tile whereas tiles in a picture are ordered consecutively in a raster scan of the tiles of the picture.A slice includes an integer number of bricks of a picture that maybe exclusively contained in a single NAL unit. A slice may consists of either a number of complete tiles or only a consecutive sequence of complete bricks of one tile. In this document, tile groups and slices may be used interchangeably. For example, in this document, a tile group / tile group header may be referred to as a slice / slice header.

[0027] A pixel or pel can mean the smallest unit that makes up a picture (or image). Also, the term "sample" can be used as the term corresponding to a pixel. A sample can generally indicate a pixel or a pixel value, and can also indicate only the pixel / pixel value of the luma component, or only the pixel / pixel value of the chroma component.

[0028] A unit can indicate the basic unit of image processing. A unit can include at least one of a specific region of a picture and information related to that region. One unit can include one luma block and two chroma (e.g., cb, cr) blocks. A unit can, in some cases, be used interchangeably with terms such as block or area. In a general case, an M×N block can include a sample (or sample array) consisting of M columns and N rows, or a set (or array) of transform coefficients.

[0029] In this specification, "A or B" may mean "only A", "only B", or "both A and B". In other words, in this specification, "A or B" may be construed as "A and / or B". For example, in this specification, "A, B, or C" may mean "only A", "only B", "only C", or "any combination of A, B, and C".

[0030] The slashes ( / ) and commas used in this specification may mean "and / or". For example, "A / B" may mean "A and / or B". Thus, "A / B" may mean "only A", "only B", or "both A and B". For example, "A, B, C" may mean "A, B, or C".

[0031] In this specification, "at least one of A and B" may mean "only A", "only B", or "both A and B". Also, in this specification, expressions such as "at least one of A or B" and "at least one of A and / or B" may be interpreted in the same way as "at least one of A and B".

[0032] Also, in this specification, "at least one of A, B and C" may mean "only A", "only B", "only C" or "any combination of A, B and C". Also, "at least one of A, B or C" and "at least one of A, B and / or C" may mean "at least one of A, B and C".

[0033] Also, the parentheses used in this specification may mean "for example". Specifically, when it is displayed as "prediction (intra prediction)", "intra prediction" may be proposed as an example of "prediction". In other words, "prediction" in this specification is not limited to "intra prediction", and "intra prediction" may be proposed as an example of "prediction". Also, when it is displayed as "prediction (i.e., intra prediction)", "intra prediction" may be proposed as an example of "prediction".

[0034] The technical features separately described within one drawing in this specification may be realized individually or simultaneously.

[0035] The following drawings are created to illustrate a specific example of this specification. Since the names of specific devices and the names of specific signals / messages / fields described in the drawings are presented exemplarily, the technical features of this specification are not limited to the specific names used in the following drawings.

[0036] FIG. 2 is a diagram schematically explaining the configuration of a video / image encoding device to which the embodiment of this document can be applied. Hereinafter, the video encoding device can include an image encoding device.

[0037] As shown in FIG. 2, the encoding device 200 can be configured to include an image partitioner 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 can include an inter-prediction unit 221 and an intra-prediction unit 222. The residual processor 230 can include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 can further include a subtractor 231. The adder 250 can be called a reconstructor or a reconstructed block generator. The aforementioned image partitioner 210, predictor 220, residual processor 230, entropy encoder 240, adder 250, and filter 260 can be configured by one or more hardware components (e.g., an encoder chipset or a processor) according to an embodiment. Also, the memory 270 can include a DPB (decoded picture buffer) and can also be configured by a digital storage medium. The hardware component can further include the memory 270 as an internal / external component.

[0038] The image segmentation unit 210 can divide an input image (or picture, frame) input to the encoding device 200 into one or more processing units. As an example, the processing unit can be called a coding unit (CU). In this case, the coding unit can be recursively divided from a coding tree unit (CTU) or a largest coding unit (LCU) by a QTBTTT (Quad-tree binary-tree ternary-tree) structure. For example, one coding unit can be divided into multiple coding units with a deeper depth based on a quad-tree structure, a binary-tree structure, and / or a ternary structure. In this case, for example, the quad-tree structure can be applied first, and the binary-tree structure and / or the ternary structure can be applied later. Or the binary-tree structure can also be applied first. The coding procedure according to this document can be executed based on the final coding unit that cannot be further divided. In this case, based on the coding efficiency according to the image characteristics, etc., the largest coding unit can be immediately used as the final coding unit, or, if necessary, the coding unit can be recursively divided into coding units with a deeper depth, and the coding unit with the optimal size can be used as the final coding unit. Here, the coding procedure can include procedures such as prediction, transformation, and restoration described later. As another example, the processing unit can further include a prediction unit (PU: Prediction Unit) or a transform unit (TU: Transform Unit). In this case, the prediction unit and the transform unit can each be divided or partitioned from the aforementioned final coding unit.The prediction unit is a unit of sample prediction, and the conversion unit is a unit for deriving a conversion coefficient and / or a unit for deriving a residual signal from the conversion coefficient.

[0039] The term "unit" can, in some cases, be used interchangeably with terms such as "block" or "area". In general, an M×N block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or a pixel value, and can represent only the pixel / pixel value of the luma component, or only the pixel / pixel value of the chroma component. A sample can be used as a term corresponding to a pixel or a pel in one picture (or image).

[0040] The encoding device 200 can subtract a prediction signal (predicted block, predicted sample array) output from the inter prediction unit 221 or the intra prediction unit 222 from an input image signal (original block, original sample array) to generate a residual signal (residual signal, residual block, residual sample array), and the generated residual signal is transmitted to the conversion unit 232. In this case, as shown in the figure, the unit that subtracts the prediction signal (predicted block, predicted sample array) from the input image signal (original block, original sample array) within the encoder 200 can be called the subtraction unit 231. The prediction unit can perform a prediction on a processing target block (hereinafter referred to as the current block) and generate a predicted block including predicted samples for the current block. The prediction unit can determine whether intra prediction or inter prediction is applied in units of the current block or CU. The prediction unit can generate various information related to prediction, such as prediction mode information, and transmit it to the entropy encoding unit 240 as described later in the description of each prediction mode. The information related to prediction can be encoded by the entropy encoding unit 240 and output in the form of a bit stream.

[0041] The intra prediction unit 222 can predict the current block by referring to samples within the current picture. The samples to be referred to can be located in the neighborhood of the current block according to the prediction mode, or can also be located remotely. In intra prediction, the prediction mode can include a plurality of non-directional modes and a plurality of directional modes. The non-directional modes can include, for example, the DC mode and the Planar Mode. The directional modes can include, for example, 33 directional prediction modes or 65 directional prediction modes depending on the degree of fineness of the prediction direction. However, this is merely an example, and more or fewer directional prediction modes can be used depending on the setting. The intra prediction unit 222 can also determine the prediction mode to be applied to the current block by using the prediction mode applied to the adjacent blocks.

[0042] The inter prediction unit 221 can derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information can be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between adjacent blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, the adjacent blocks can include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block can be called by names such as a collocated reference block and a collocated CU (colCU), and the reference picture including the temporal neighboring block can also be called a collocated picture (colPic). For example, the inter prediction unit 221 can construct a motion information candidate list based on adjacent blocks and generate information indicating which candidates are used to derive the motion vector and / or reference picture index of the current block. Inter prediction can be executed based on various prediction modes. For example, in the case of the skip mode and the merge mode, the inter prediction unit 221 can use the motion information of adjacent blocks as the motion information of the current block. In the case of the skip mode, a residual signal may not be transmitted, which is different from the merge mode.In the case of the motion information prediction (motion vector prediction, MVP) mode, the motion vector of an adjacent block can be used as a motion vector predictor, and the motion vector of the current block can be indicated by signaling the motion vector difference.

[0043] The prediction unit 220 can generate a prediction signal based on various prediction methods described later. For example, the prediction unit can apply not only intra prediction or inter prediction for the prediction of one block, but also can apply intra prediction and inter prediction simultaneously. This can be called combined inter and intra prediction (CIIP). Also, the prediction unit can be based on the intra block copy (IBC) prediction mode for the prediction of a block, or can be based on the palette mode. The IBC prediction mode or the palette mode can be used for content image / video coding such as games, for example, like SCC (screen content coding). IBC basically performs prediction within the current picture, but can be executed similarly to inter prediction in terms of deriving a reference block within the current picture. That is, IBC can utilize at least one of the inter prediction techniques described in this document. The palette mode can be regarded as an example of intra coding or intra prediction. When the palette mode is applied, the sample values in the picture can be signaled based on the information regarding the palette table and the palette index.

[0044] The prediction signal generated via the prediction unit (including the inter prediction unit 221 and / or the intra prediction unit 222) can be used to generate a restored signal or can be used to generate a residual signal. The conversion unit 232 can apply a conversion technique to the residual signal to generate transform coefficients. For example, the conversion technique can include at least one of DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT (Karhunen-Loeve Transform), GBT (Graph-Based Transform), or CNT (Conditionally Non-linear Transform). Here, GBT means a transform obtained from a graph when representing the relationship information between pixels as a graph. CNT means a transform obtained based on generating a prediction signal using all previously reconstructed pixels. Also, the conversion process can be applied to a pixel block having the same size of a square and can also be applied to a non-square, variable-size block.

[0045] The quantization unit 233 quantizes the transform coefficients and transmits them to the entropy encoding unit 240. The entropy encoding unit 240 can encode the quantized signal (information regarding the quantized transform coefficients) and output it as a bitstream. The information regarding the quantized transform coefficients can be referred to as residual information. The quantization unit 233 can reorder the quantized transform coefficients in block form into a one-dimensional vector form based on the coefficient scan order, and can also generate the information regarding the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form. The entropy encoding unit 240 can execute various encoding methods such as, for example, exponential Golomb, CAVLC (context-adaptive variable length coding), CABAC (context-adaptive binary arithmetic coding), etc. The entropy encoding unit 240 can also encode, together or separately, information necessary for video / image restoration (e.g., values of syntax elements, etc.) in addition to the quantized transform coefficients. The encoded information (e.g., encoded video / image information) can be transmitted or stored in the form of a bitstream in units of NAL (network abstraction layer) units. The video / image information can further include information regarding various parameter sets such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). Also, the video / image information can further include general constraint information. The information and / or syntax elements transmitted / signaled from the encoding device to the decoding device in this document can be included in the video / image information. The video / image information can be encoded through the encoding procedure described above and included in the bitstream.The bitstream can be transmitted via a network or stored in a digital storage medium. Here, the network can include a broadcast network and / or a communication network, etc., and the digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The signal output from the entropy encoding unit 240 can be configured as an internal / external element of the encoding device 200 by a transmission unit (not shown) for transmission and / or a storage unit (not shown) for storage, or the transmission unit can also be included in the entropy encoding unit 240.

[0046] The quantized transform coefficients output from the quantization unit 233 can be used to generate a prediction signal. For example, a residual signal (residual block or residual sample) can be restored by applying inverse quantization and inverse transformation to the quantized transform coefficients via the inverse quantization unit 234 and the inverse transform unit 235. The addition unit 250 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the restored residual signal to the prediction signal output from the inter prediction unit 221 or the intra prediction unit 222. When there is no residual for the block to be processed, as in the case where the skip mode is applied, the predicted block can be used as the reconstructed block. The addition unit 250 can be called a restoration unit or a reconstructed block generation unit. The generated reconstructed signal can be used for intra prediction of the next block to be processed within the current picture, and as will be described later, can also be used for inter prediction of the next picture after passing through filtering.

[0047] On the other hand, LMCS (luma mapping with chroma scaling) can also be applied in the picture encoding and / or restoration process.

[0048] The filtering unit 260 can apply filtering to the restored signal to improve the subjective / objective image quality. For example, the filtering unit 260 can apply various filtering methods to the restored picture to generate a modified restored picture, and can store the modified restored picture in the memory 270, specifically, in the DPB of the memory 270. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filtering unit 260 can generate various information related to filtering and transmit it to the entropy encoding unit 240 as will be described later in the description of each filtering method. The information related to filtering can be encoded by the entropy encoding unit 240 and output in the form of a bit stream.

[0049] The modified restored picture transmitted to the memory 270 can be used as a reference picture in the inter prediction unit 221. The encoding device can avoid prediction mismatches between the encoding device 200 and the decoding device 300 and improve the encoding efficiency when inter prediction is applied through this.

[0050] The memory 270 DPB can store the modified restored picture for use as a reference picture in the inter prediction unit 221. The memory 270 can store the motion information of the block where the motion information in the current picture was derived (or encoded) and / or the motion information of the block in the already restored picture. The stored motion information can be transmitted to the inter prediction unit 221 for utilization as the motion information of spatially adjacent blocks or temporally adjacent blocks. The memory 270 can store the restored samples of the restored blocks in the current picture and transmit them to the intra prediction unit 222.

[0051] FIG. 3 is a diagram schematically illustrating a configuration of a video / image decoding apparatus to which an embodiment of this document can be applied.

[0052] As shown in FIG. 3, the decoding apparatus 300 can be configured to include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 can include an inter-predictor 331 and an intra-predictor 332. The residual processor 320 can include a dequantizer 321 and an inverse transformer 322. The entropy decoder 310, the residual processor 320, the predictor 330, the adder 340, and the filter 350 described above can be configured by one hardware component (for example, a decoder chipset or a processor) according to an embodiment. Also, the memory 360 can include a DPB (decoded picture buffer) and can also be configured by a digital storage medium. The hardware component can further include the memory 360 as an internal / external component.

[0053] When a bitstream including video / image information is input, the decoding device 300 can restore an image corresponding to the process in which the video / image information was processed by the encoding device of FIG. 2. For example, the decoding device 300 can derive units / blocks based on block splitting related information obtained from the bitstream. The decoding device 300 can execute decoding using the processing units applied in the encoding device. Therefore, the processing unit for decoding is, for example, a coding unit, and the coding unit can be split according to a quad-tree structure, a binary tree structure, and / or a ternary tree structure from a coding tree unit or a maximum coding unit. One or more transform units can be derived from the coding unit. Then, the restored image signal decoded and output via the decoding device 300 can be played back via a playback device.

[0054] The decoding device 300 can receive the signal output from the encoding device in FIG. 2 in the form of a bitstream, and the received signal can be decoded via the entropy decoding unit 310. For example, the entropy decoding unit 310 can parse the bitstream to derive information (such as video / image information) necessary for image restoration (or picture restoration). The video / image information can further include information regarding various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). Also, the video / image information can further include general constraint information. The decoding device can further decode a picture based on the information regarding the parameter set and / or the general constraint information. The signaling / received information and / or syntax elements described later in this document can be decoded via the decoding procedure and obtained from the bitstream. For example, the entropy decoding unit 310 can decode the information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output values of syntax elements necessary for image restoration, quantized values of transform coefficients regarding residuals, etc. More specifically, the CABAC entropy decoding method receives bins corresponding to each syntax element in the bitstream, determines a context model using the syntax element information to be decoded, the surrounding and decoded information of the block to be decoded, or the information of symbols / bins decoded in the previous step, predicts the occurrence probability of a bin according to the determined context model, and executes arithmetic decoding of the bin to generate a symbol corresponding to the value of each syntax element. At this time, after determining the context model, the CABAC entropy decoding method can update the context model using the information of the symbol / bin decoded for the context model of the next symbol / bin.Of the information decoded by the entropy decoding unit 310, the information related to prediction is provided to the prediction unit (inter prediction unit 332 and intra prediction unit 331), and the residual value for which entropy decoding is performed by the entropy decoding unit 310, that is, the quantized transform coefficient and related parameter information, can be input to the residual processing unit 320. The residual processing unit 320 can derive a residual signal (residual block, residual sample, residual sample array). Also, among the information decoded by the entropy decoding unit 310, the information related to filtering can be provided to the filtering unit 350. On the other hand, a receiving unit (not shown) that receives the signal output from the encoding device can be further configured as an internal / external element of the decoding device 300, or the receiving unit is a component of the entropy decoding unit 310. On the other hand, the decoding device according to this document can be called a video / image / picture decoding device, and the decoding device can be classified into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder can include the entropy decoding unit 310, and the sample decoder can include at least one of the inverse quantization unit 321, inverse transform unit 322, addition unit 340, filtering unit 350, memory 360, inter prediction unit 332, and intra prediction unit 331.

[0055] In the inverse quantization unit 321, the quantized transform coefficient can be inverse quantized to output a transform coefficient. The inverse quantization unit 321 can reorder the quantized transform coefficients in a two-dimensional block form. In this case, the reordering can be performed based on the coefficient scan order executed by the encoding device. The inverse quantization unit 321 can perform inverse quantization on the quantized transform coefficient using a quantization parameter (for example, quantization step size information) to obtain a transform coefficient.

[0056] In the inverse conversion unit 322, the conversion coefficients are inversely converted to obtain a residual signal (residual block, residual sample array).

[0057] The prediction unit can perform prediction on the current block and generate a predicted block including prediction samples for the current block. The prediction unit can determine whether intra prediction or inter prediction is applied to the current block based on the information regarding the prediction output from the entropy decoding unit 310, and can determine a specific intra / inter prediction mode.

[0058] The prediction unit 320 can generate a prediction signal based on various prediction methods described later. For example, the prediction unit can apply not only intra prediction or inter prediction for prediction of one block, but also can apply intra prediction and inter prediction simultaneously. This can be called combined inter and intra prediction (CIIP). Also, the prediction unit can be based on the intra block copy (IBC) prediction mode or the palette mode for prediction of a block. The IBC prediction mode or the palette mode can be used for content image / video coding such as games, for example, like SCC (screen content coding). IBC basically performs prediction within the current picture, but can be executed similarly to inter prediction in terms of deriving a reference block within the current picture. That is, IBC can utilize at least one of the inter prediction techniques described in this document. The palette mode can be regarded as an example of intra coding or intra prediction. When the palette mode is applied, information regarding the palette table and the palette index can be included in and signaled in the video / image information.

[0059] The intra prediction unit 331 can predict the current block by referring to samples within the current picture. The samples to be referred to can be located in the neighborhood (neighbor) of the current block or at a distance from it depending on the prediction mode. In intra prediction, the prediction mode can include a plurality of non-directional modes and a plurality of directional modes. The intra prediction unit 331 can also determine the prediction mode to be applied to the current block by using the prediction mode applied to an adjacent block.

[0060] The inter prediction unit 332 can derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, in order to reduce the amount of motion information transmitted from the inter prediction mode, the motion information can be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the adjacent block and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, the adjacent blocks can include spatial neighboring blocks existing within the current picture and temporal neighboring blocks existing in the reference picture. For example, the inter prediction unit 332 can construct a motion information candidate list based on the adjacent blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Inter prediction can be executed based on various prediction modes, and the information regarding the prediction can include information indicating the mode of inter prediction for the current block.

[0061] The adder 340 can generate a restored signal (restored picture, restored block, restored sample array) by adding the obtained residual signal to the predicted signal (predicted block, predicted sample array) output from the prediction unit (including the inter prediction unit 332 and / or the intra prediction unit 331). When there is no residual for the block to be processed, as in the case where the skip mode is applied, the predicted block can be used as the restored block.

[0062] The adder 340 can be referred to as a restoration unit or a restored block generation unit. The generated restored signal can be used for intra prediction of the next block to be processed in the current picture, can be output after being filtered as described later, or can also be used for inter prediction of the next picture.

[0063] On the other hand, LMCS (luma mapping with chroma scaling) can also be applied during the picture decoding process.

[0064] The filtering unit 350 can apply filtering to the restored signal to improve the subjective / objective image quality. For example, the filtering unit 350 can apply various filtering methods to the restored picture to generate a modified restored picture, and can transmit the modified restored picture to the memory 360, specifically, to the DPB of the memory 360. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0065] The (corrected) restored picture stored in the DPB of the memory 360 can be used as a reference picture in the inter prediction unit 332. The memory 360 can store the motion information of the block from which the motion information in the current picture was derived (or decoded) and / or the motion information of the block in the already restored picture. The stored motion information can be transmitted to the inter prediction unit 260 for utilization as the motion information of spatially adjacent blocks or temporally adjacent blocks. The memory 360 can store the restored samples of the restored blocks in the current picture and can transmit them to the intra prediction unit 331.

[0066] In this specification, the embodiments described in the filtering unit 260, the inter prediction unit 221, and the intra prediction unit 222 of the encoding device 200 can be applied to the filtering unit 350, the inter prediction unit 332, and the intra prediction unit 331 of the decoding device 300 in the same or corresponding manner, respectively.

[0067] In this document, at least one of quantization / inverse quantization and / or transformation / inverse transformation can be omitted. When the quantization / inverse quantization is omitted, the quantized transform coefficients can be referred to as transform coefficients. When the transformation / inverse transformation is omitted, the transform coefficients can be referred to as coefficients or residual coefficients, or can still be referred to as transform coefficients for the sake of consistency of expression.

[0068] In this document, the quantized transform coefficients and transform coefficients can be referred to as transform coefficients and scaled transform coefficients, respectively. In this case, the residual information can include information about the transform coefficients (etc.), and the information about the transform coefficients (etc.) can be signaled via a residual coding syntax. The transform coefficients can be derived based on the residual information (or the information about the transform coefficients (etc.)), and the scaled transform coefficients can be derived via an inverse transform (scaling) for the transform coefficients. The residual samples can be derived based on an inverse transform (transformation) for the scaled transform coefficients. This can be applied / expressed similarly in other parts of this document.

[0069] FIG. 4 exemplarily shows a hierarchical structure for a coded image / video.

[0070] As shown in FIG. 4, a coded image / video is divided into a VCL (video coding layer) that performs decoding processing of the image / video and handles itself, a lower system that transmits and stores the encoded information, and a NAL (network abstraction layer) that exists between the VCL and the lower system and is responsible for network adaptation functions.

[0071] In the VCL, VCL data including compressed image data (slice data) can be generated, or parameter sets including information such as a Picture Parameter Set (PPS), a Sequence Parameter Set (SPS), a Video Parameter Set (VPS), or a SEI (Supplemental Enhancement Information) message that is additionally required in the decoding process of the image can be generated.

[0072] In the NAL, a NAL unit can be generated by adding header information (NAL unit header) to the RBSP (Raw Byte Sequence Payload) generated by the VCL. At this time, the RBSP refers to slice data, parameter sets, SEI messages, etc. generated by the VCL. The NAL unit header can include NAL unit type information specified by the RBSP data included in the NAL unit.

[0073] As shown in the above drawings, the NAL unit can be divided into a VCL NAL unit and a Non-VCL NAL unit by the RBSP generated by the VCL. The VCL NAL unit can mean a NAL unit containing information related to an image (slice data), and the Non-VCL NAL unit can mean a NAL unit containing information (parameter set or SEI message) necessary for decoding an image.

[0074] As described above, the above-mentioned VCL NAL unit and Non-VCL NAL unit can be transmitted via a network with header information attached according to the data standard of the lower system. For example, the NAL unit can be transformed into a data form of a predetermined standard such as the H.266 / VVC file format, RTP (Real-time Transport Protocol), TS (Transport Stream), etc., and transmitted via various networks.

[0075] As described above, the NAL unit type can be specified by the RBSP data structure included in the NAL unit, and information regarding such a NAL unit type can be stored in the NAL unit header and signaled.

[0076] For example, NAL units can be broadly classified into VCL NAL unit types and Non-VCL NAL unit types depending on whether the NAL unit contains information related to an image (slice data). The VCL NAL unit types can be classified according to the properties and types of pictures included in the VCL NAL unit, and the Non-VCL NAL unit types can be classified according to the types of parameter sets, etc.

[0077] The following is an example of NAL unit types specified according to the types of parameter sets included in Non-VCL NAL unit types, etc.

[0078] · APS (Adaptation Parameter Set) NAL unit: The type for NAL units containing APS

[0079] · DPS (Decoding Parameter Set) NAL unit: The type for NAL units containing DPS

[0080] · VPS (Video Parameter Set) NAL unit: The type for NAL units containing VPS

[0081] · SPS (Sequence Parameter Set) NAL unit: The type for NAL units containing SPS

[0082] · PPS (Picture Parameter Set) NAL unit: The type for NAL units containing PPS

[0083] · PH (Picture header) NAL unit: The type for NAL units containing PH

[0084] The above-mentioned NAL unit type has syntax information for the NAL unit type, and the syntax information can be stored in the NAL unit header and signaled. For example, the syntax information can be nal_unit_type, and the NAL unit type can be specified by the nal_unit_type value.

[0085] On the other hand, as described above, the encoding device can perform various encoding methods such as exponential Golomb, CAVLC (context-adaptive variable length coding), CABAC (context-adaptive binary arithmetic coding), etc. Also, the decoding device can decode the information in the bitstream based on coding methods such as exponential Golomb coding, CAVLC, or CABAC, and output values of syntax elements necessary for image restoration, quantized values of transform coefficients related to residuals, etc.

[0086] For example, the above-described coding methods, etc. can be performed as described later.

[0087] FIG. 5 exemplarily shows context-adaptive binary arithmetic coding (CABAC) for encoding a syntax element. For example, in the encoding process of CABAC, when the input signal is a syntax element that is not a binary value, the encoding device can binarize the value of the input signal to convert the input signal into a binary value. Also, when the input signal is already a binary value (i.e., when the value of the input signal is a binary value), binarization is not performed and it can be bypassed. Here, each binary number 0 or 1 that constitutes a binary value can be called a bin. For example, when the binary string after binarization is 110, each of 1, 1, and 0 is called one bin. The bin (etc.) for one syntax element can represent the value of the syntax element.

[0088] Thereafter, the binarized bin etc. of the syntax element can be input as a regular coding engine or a bypass coding engine. The regular coding engine of the encoding device can assign a context model that reflects a probability value to the bin, and can encode the bin based on the assigned context model. The regular coding engine of the encoding device can update the context model for the bin after encoding each bin. The bin encoded as described above can be represented as a context-coded bin.

[0089] On the one hand, when the binary bins of the syntax element, etc. are input to the bypass encoding engine, they can be coded as follows. For example, the bypass encoding engine of the encoding device omits the procedure of estimating the probability for the input bin and the procedure of updating the probability model applied to the bin after encoding. When bypass encoding is applied, the encoding device can apply a uniform probability distribution instead of assigning a context model to encode the input bin, thereby improving the encoding speed. The bin encoded as described above can be represented as a bypass bin.

[0090] Entropy decoding can represent the process of performing the same process as the above-described entropy encoding in reverse order.

[0091] For example, when the syntax element is decoded based on the context model, the decoding device can receive the bin corresponding to the syntax element via the bitstream, determine the context model using the decoding information of the decoding target block or the peripheral block of the syntax element or the information of the symbol / bin decoded in the previous step, predict the occurrence probability of the received bin according to the determined context model, and perform arithmetic decoding of the bin to derive the value of the syntax element. Thereafter, the context model of the bin to be decoded next can be updated in the determined context model.

[0092] Also, for example, when the syntax element is bypass decoded, the decoding device can receive the bin corresponding to the syntax element via the bitstream and decode the input bin by applying a uniform probability distribution. In this case, the decoding device can omit the procedure of deriving the context model of the syntax element and the procedure of updating the context model applied to the bin after decoding.

[0093] Also, as described above, in performing video coding, prediction is performed to increase the compression efficiency. Through this, a predicted block including prediction samples for the current block, which is a block to be coded, can be generated. Here, the predicted block includes prediction samples in the spatial domain (or pixel domain). The predicted block is derived identically in the encoding device and the decoding device, and the encoding device can increase the image coding efficiency by signaling information (residual information) regarding the residual between the original block and the predicted block, which is not the original sample value of the original block, to the decoding device. The decoding device can derive a residual block including residual samples based on the residual information, and combine the residual block and the predicted block to generate a restored block including restored samples, and can generate a restored picture including the restored block.

[0094] The residual information can be generated through the conversion and quantization procedures. For example, an encoding device can derive a residual block between the original block and the predicted block, perform a conversion procedure on the residual samples (residual sample array) included in the residual block to derive conversion coefficients, perform a quantization procedure on the conversion coefficients to derive quantized conversion coefficients, and signal the related residual information to a decoding device (via a bitstream). Here, the residual information can include information such as value information, position information, conversion technique, conversion kernel, quantization parameter, etc. of the quantized conversion coefficients. The decoding device can perform an inverse quantization / inverse conversion procedure based on the residual information to derive residual samples (or a residual block). The decoding device can generate a restored picture based on the predicted block and the residual block. The encoding device can further inverse quantize / inverse convert the quantized conversion coefficients to derive a residual block for reference in the inter prediction of subsequent pictures, and generate a restored picture based on this.

[0095] Intra prediction can represent a prediction that generates prediction samples for a current block based on reference samples within a picture (hereinafter, the current picture) to which the current block belongs. When intra prediction is applied to the current block, peripheral reference samples used for intra prediction of the current block can be derived. The peripheral reference samples of the current block can include a total of 2×nH samples adjacent to the left boundary of the current block of size nW×nH and adjacent to the bottom - left, samples adjacent to the top boundary of the current block and a total of 2×nW samples adjacent to the top - right, and 1 sample adjacent to the top - left of the current block. Or, the peripheral reference samples of the current block can also include a plurality of columns of upper - peripheral samples and a plurality of rows of left - peripheral samples. Also, the peripheral reference samples of the current block can include a total of nH samples adjacent to the right boundary of the current block of size nW×nH, a total of nW samples adjacent to the bottom boundary of the current block, and 1 sample adjacent to the bottom - right of the current block.

[0096] However, some of the peripheral reference samples of the current block may not have been decoded yet or may not be available. In this case, the decoder can substitute unavailable samples with available samples to form the peripheral reference samples used for prediction. Or, the peripheral reference samples used for prediction can be formed through interpolation of available samples.

[0097] When neighboring reference samples are derived, (i) prediction samples can be derived based on the average or interpolation of neighboring reference samples of the current block, and (ii) the prediction samples can also be derived based on reference samples that exist in a specific (prediction) direction with respect to the prediction samples among the neighboring reference samples of the current block. In the case of (i), it can be called a non-directional mode or a non-angular mode, and in the case of (ii), it can be called a directional mode or an angular mode.

[0098] Also, among the neighboring reference samples, the prediction samples can be generated by interpolation between a first neighboring sample located in the prediction direction of the intra prediction mode of the current block and a second neighboring sample located in the direction opposite to the prediction direction with respect to the prediction samples of the current block. In the case described above, it can be called Linear interpolation intra prediction (LIP). Also, a chroma prediction sample can be generated based on luma samples using a linear model (LM). In this case, it can be called the LM mode or the CCLM (chroma component LM) mode.

[0099] Also, a temporary prediction sample of the current block is derived based on filtered neighboring reference samples, and a prediction sample of the current block is derived by weighted sum of at least one reference sample derived by the intra prediction mode and the temporary prediction sample among the existing neighboring reference samples, that is, the neighboring reference samples that have not been filtered. In the case described above, it can be called PDPC (Position dependent intra prediction).

[0100] Also, among the surrounding multiple-reference sample lines of the current block, the reference sample line with the highest prediction accuracy is selected, and a prediction sample is derived using the reference sample located in the prediction direction on this line. At this time, intra prediction coding can be performed by a method of instructing (signaling) the used reference sample line to the decoding device. In the above-described case, it can be called multi-reference line intra prediction or MRL-based intra prediction.

[0101] Also, the current block is divided into vertical or horizontal sub-partitions and intra prediction is performed based on the same intra prediction mode, but the surrounding reference samples can be derived and used in units of the sub-partitions. That is, in this case, although the intra prediction mode for the current block is similarly applied to the sub-partitions, by deriving and using the surrounding reference samples in units of the sub-partitions, the intra prediction performance can be enhanced in some cases. Such a prediction method can be called ISP (intra sub-partitions)-based intra prediction.

[0102] The intra prediction methods described above can be called intra prediction types, distinguished from the intra prediction modes. The intra prediction types can be called by various terms such as intra prediction techniques or additional intra prediction modes. For example, the intra prediction type (or additional intra prediction mode, etc.) can include at least one of LIP, PDPC, MRL, and ISP described above. The general intra prediction method excluding the specific intra prediction types such as LIP, PDPC, MRL, and ISP can be called the normal intra prediction type. The normal intra prediction type can be generally applied when the above specific intra prediction types are not applicable, and prediction can be performed based on the intra prediction mode described above. On the other hand, post-processing filtering for the prediction samples derived as needed can also be performed.

[0103] Specifically, the intra prediction procedure can include an intra prediction mode / type determination step, a neighboring reference sample derivation step, and an intra prediction mode / type-based prediction sample derivation step. Further, if necessary, a post-filtering step for the derived prediction samples can also be performed.

[0104] FIG. 6 shows an example of an intra prediction-based video / image encoding method.

[0105] As shown in FIG. 6, the encoding device performs intra prediction on the current block (S600). The encoding device can derive an intra prediction mode / type for the current block and derive neighboring reference samples of the current block, and generate prediction samples within the current block based on the intra prediction mode / type and the neighboring reference samples. Here, the intra prediction mode / type determination, neighboring reference sample derivation, and prediction sample generation procedures can be performed simultaneously, or any one of the procedures can be performed prior to the other procedures. The encoding device can determine the mode / type applied to the current block among a plurality of intra prediction modes / types. The encoding device can compare the RD cost for the intra prediction mode / type and determine the optimal intra prediction mode / type for the current block.

[0106] On the other hand, the encoding device can also perform a prediction sample filtering procedure. The prediction sample filtering can be referred to as post-filtering. Some or all of the prediction samples can be filtered by the prediction sample filtering procedure. In some cases, the prediction sample filtering procedure can be omitted.

[0107] The encoding device generates a residual sample for the current block based on the (filtered) prediction sample (S610). The encoding device can compare the prediction sample with the original sample of the current block on a phase basis to derive the residual sample.

[0108] The encoding device can encode image information including the information related to the intra prediction (prediction information) and the residual information related to the residual sample (S620). The prediction information can include the intra prediction mode information and the intra prediction type information. The encoding device can output the encoded image information in the form of a bitstream. The output bitstream can be transmitted to the decoding device via a storage medium or a network.

[0109] The residual information can include a residual coding syntax to be described later. The encoding device can convert / quantize the residual sample to derive quantized transform coefficients. The residual information can include information related to the quantized transform coefficients.

[0110] On the other hand, as described above, the encoding device can generate a reconstructed picture (including reconstructed samples and reconstructed blocks). For this purpose, the encoding device can perform inverse quantization / inverse transformation processing on the quantized transform coefficients again to derive a (corrected) residual sample. As described above, the reason for performing inverse quantization / inverse transformation again after converting / quantizing the residual sample is to derive the same residual sample as the residual sample derived from the decoding device. The encoding device can generate a reconstructed block including a reconstructed sample for the current block based on the prediction sample and the (corrected) residual sample. A reconstructed picture for the current picture can be generated based on the reconstructed block. As described above, an in-loop filtering procedure or the like can be further applied to the reconstructed picture.

[0111] FIG. 7 shows an example of an intra prediction-based video / image encoding method.

[0112] The decoding device can perform operations corresponding to the operations performed by the encoding device.

[0113] Prediction information and residual information can be obtained from the bitstream. Residual samples for the current block can be derived based on the residual information. Specifically, based on the quantized transform coefficients derived based on the residual information, inverse quantization is performed to derive transform coefficients, and inverse transform is performed on the transform coefficients to derive residual samples for the current block.

[0114] Specifically, the decoding device can derive the intra prediction mode / type for the current block based on the received prediction information (intra prediction mode / type information) (S700). The decoding device can derive the peripheral reference samples of the current block (S710). The decoding device generates prediction samples within the current block based on the intra prediction mode / type and the peripheral reference samples (S720). In this case, the decoding device can perform a prediction sample filtering procedure. The prediction sample filtering can be called post-filtering. Some or all of the prediction samples can be filtered by the prediction sample filtering procedure. In some cases, the prediction sample filtering procedure can be omitted.

[0115] The decoding device generates residual samples for the current block based on the received residual information (S730). The decoding device can generate restored samples for the current block based on the prediction samples and the residual samples, and derive a restored block including the restored samples (S740). A restored picture for the current picture can be generated based on the restored block. As described above, in-loop filtering procedures and the like can be further applied to the restored picture.

[0116] The intra prediction mode information can include, for example, flag information (e.g., intra_luma_mpm_flag) indicating whether the most probable mode (MPM) is applied to the current block or whether the remaining mode is applied. When the MPM is applied to the current block, the prediction mode information can further include index information (e.g., intra_luma_mpm_idx) indicating one of the intra prediction mode candidates (MPM candidates). The intra prediction mode candidates (MPM candidates) can be composed of an MPM candidate list or an MPM list. Also, when the MPM is not applied to the current block, the intra prediction mode information can further include remaining mode information (e.g., intra_luma_mpm_remainder) indicating one of the remaining intra prediction modes excluding the intra prediction mode candidates (MPM candidates). The decoding device can determine the intra prediction mode of the current block based on the intra prediction mode information.

[0117] In addition, the intra prediction type information can be realized in various forms. As an example, the intra prediction type information can include intra prediction type index information indicating one of the intra prediction types. As another example, the intra prediction type information may include reference sample line information (e.g., intra_luma_ref_idx) indicating whether the MRL is applied to the current block and, if applied, which reference sample line is used, ISP flag information (e.g., intra_subpartitions_mode_flag) indicating whether the ISP is applied to the current block, or ISP type information (e.g., intra_subpartitions_split_flag) indicating the split type of the subpartition when the ISP is applied, and can include at least one of them. Also, the intra prediction type information can include an MIP flag indicating whether MIP (matrix-based intra prediction) is applied to the current block.

[0118] The intra prediction mode information and / or the intra prediction type information can be encoded / decoded by the coding method described in this document. For example, the intra prediction mode information and / or the intra prediction type information can be encoded / decoded via entropy coding (e.g., CABAC, CAVLC).

[0119] FIG. 8 exemplarily shows the intra prediction procedure.

[0120] Referring to FIG. 8, as described above, the intra prediction procedure can include an intra prediction mode / type determination step, a peripheral reference sample derivation step, and an intra prediction execution (prediction sample generation) step. The intra prediction procedure can be performed by an encoding device and a decoding device as described above. In this document, the coding device can include an encoding device and / or a decoding device.

[0121] As shown in FIG. 8, the coding device determines an intra prediction mode / type (S800).

[0122] The encoding device can determine the intra prediction mode / type applied to the current block among the various intra prediction modes / types described above, and can generate prediction-related information. The prediction-related information can include intra prediction mode information representing the intra prediction mode applied to the current block and / or intra prediction type information representing the intra prediction type applied to the current block. The decoding device can determine the intra prediction mode / type applied to the current block based on the prediction-related information.

[0123] The intra prediction mode information can include, for example, flag information (e.g., intra_luma_mpm_flag) indicating whether MPM (most probable mode) is applied to the current block or whether the remaining mode is applied. When the MPM is applied to the current block, the prediction mode information can further include index information (e.g., intra_luma_mpm_idx) indicating one of the intra prediction mode candidates (MPM candidates). The intra prediction mode candidates (MPM candidates) can be composed of an MPM candidate list or an MPM list. Also, when the MPM is not applied to the current block, the intra prediction mode information can further include remaining mode information (e.g., intra_luma_mpm_remainder) indicating one of the remaining intra prediction modes excluding the intra prediction mode candidates (MPM candidates). The decoding device can determine the intra prediction mode of the current block based on the intra prediction mode information.

[0124] Also, the intra prediction type information can be realized in various forms. As an example, the intra prediction type information can include intra prediction type index information indicating one of the intra prediction types. As another example, the intra prediction type information includes reference sample line information (e.g., intra_luma_ref_idx) indicating whether the MRL is applied to the current block and, if applied, which reference sample line is used, ISP flag information (e.g., intra_subpartitions_mode_flag) indicating whether the ISP is applied to the current block, or ISP type information (e.g., intra_subpartitions_split_flag) indicating the split type of the subpartition when the ISP is applied, and can include at least one of them. Also, the intra prediction type information can include a MIP flag indicating whether MIP (matrix-based intra prediction) is applied to the current block.

[0125] For example, when intra prediction is applied, the intra prediction mode applied to the current block can be determined using the intra prediction modes of the neighboring blocks. For example, the coding device can select one of the MPM (most probable mode) candidates in the MPM list derived based on the intra prediction modes of the neighboring blocks (e.g., left and / or upper neighboring blocks) of the current block and / or additional candidate modes based on the received MPM index, or can select one of the remaining intra prediction modes not included in the MPM candidates (and the planar mode) based on the MPM reminder information (remaining intra prediction mode information). The MPM list can be configured to include or not include the planar mode as a candidate. For example, when the MPM list includes the planar mode as a candidate, the MPM list can have six candidates, and when the MPM list does not include the planar mode as a candidate, the MPM list can have five candidates. When the MPM list does not include the planar mode as a candidate, a not planar flag (e.g., intra_luma_not_planar_flag) indicating whether the intra prediction mode of the current block is not the planar mode can be signaled. For example, the MPM flag can be signaled first, and the MPM index and the not planar flag can be signaled when the value of the MPM flag is 1. Also, the MPM index can be signaled when the value of the not planar flag is 1. Here, the configuration that the MPM list does not include the planar mode as a candidate is to signal the flag (not planar flag) first to confirm whether it is the planar mode first because the planar mode is always considered as an MPM rather than not being an MPM.

[0126] For example, whether the intra prediction mode currently applied to a block is among the MPM candidates (and the planar mode) or among the remaining modes can be indicated based on the MPM flag (e.g., intra_luma_mpm_flag). A value of 1 for the MPM flag can indicate that the intra prediction mode for the current block is within the MPM candidates (and the planar mode), and a value of 0 for the MPM flag can indicate that the intra prediction mode for the current block is not within the MPM candidates (and the planar mode). A value of 0 for the not planar flag (e.g., intra_luma_not_planar_flag) can indicate that the intra prediction mode for the current block is the planar mode, and a value of 1 for the not planar flag can indicate that the intra prediction mode for the current block is not the planar mode. The MPM index can be signaled in the form of the mpm_idx or intra_luma_mpm_idx syntax element, and the remaining intra prediction mode information can be signaled in the form of the rem_intra_luma_pred_mode or intra_luma_mpm_remainder syntax element. For example, the remaining intra prediction mode information can index the remaining intra prediction modes not included in the MPM candidates (and the planar mode) among all the intra prediction modes in ascending order of the prediction mode number and point to one of them. The intra prediction mode can be the intra prediction mode for the luma component (samples). Hereinafter, the intra prediction mode information can include at least one of the MPM flag (e.g., intra_luma_mpm_flag), the not planar flag (e.g., intra_luma_not_planar_flag), the MPM index (e.g., mpm_idx or intra_luma_mpm_idx), and the remaining intra prediction mode information (rem_intra_luma_pred_mode or intra_luma_mpm_remainder).In this document, the MPM list can be referred to by various terms such as the MPM candidate list, candModeList, etc.

[0127] When MIP is currently applied to a block, a separate MPM flag (e.g., intra_mip_mpm_flag) for MIP, an MPM index (e.g., intra_mip_mpm_idx), and remaining intra prediction mode information (e.g., intra_mip_mpm_remainder) can be signaled, and the not planar flag can be not signaled.

[0128] In other words, generally when block partitioning is performed on an image, the current block to be coded and neighboring blocks tend to have similar image characteristics. Therefore, the current block and neighboring blocks are likely to be identical to each other or have similar intra prediction modes. Thus, the encoder can use the intra prediction mode of neighboring blocks to encode the intra prediction mode of the current block.

[0129] The coding device can construct an MPM (most probable modes) list for the current block. The MPM list can also be referred to as the MPM candidate list. Here, MPM can mean a mode used to improve coding efficiency by considering the similarity between the current block and neighboring blocks during intra prediction mode coding. As described above, the MPM list can be configured to include the planar mode or can be configured excluding the planar mode. For example, when the MPM list includes the planar mode, the number of candidates in the MPM list can be 6. And when the MPM list does not include the planar mode, the number of candidates in the MPM list can be 5.

[0130] The encoding device can perform prediction based on various intra prediction modes, and can determine the optimal intra prediction mode based on rate-distortion optimization (RDO) based on this. In this case, the encoding device can determine the optimal intra prediction mode using only the MPM candidates and the planar mode configured in the MPM list, or can also determine the optimal intra prediction mode using not only the MPM candidates and the planar mode configured in the MPM list but also the remaining intra prediction modes. Specifically, for example, if the intra prediction type of the current block is a specific type (e.g., LIP, MRL, or ISP) that is not the normal intra prediction type, the encoding device can determine the optimal intra prediction mode by considering only the MPM candidates and the planar mode as intra prediction mode candidates for the current block. That is, in this case, the intra prediction mode for the current block can be determined among the MPM candidates and the planar mode, and in this case, the MPM flag cannot be encoded / signaled. In this case, the decoding device can assume that the MPM flag is 1 without being separately signaled.

[0131] On the other hand, generally, when the intra prediction mode of the current block is not the planar mode and is one of the MPM candidates in the MPM list, the encoding device generates an MPM index (mpm idx) indicating one of the MPM candidates. If the intra prediction mode of the current block is not in the MPM list, MPM remainder information (remaining intra prediction mode information) indicating the same mode as the intra prediction mode of the current block is generated from among the remaining intra prediction modes not included in the MPM list (and the planar mode). The MPM remainder information can include, for example, the intra_luma_mpm_remainder syntax element.

[0132] The decoding device acquires intra prediction mode information from the bitstream. As described above, the intra prediction mode information can include at least one of an MPM flag, a not planar flag, an MPM index, and MPM remainder information (remaining intra prediction mode information). The decoding device can construct an MPM list. The MPM list is configured in the same manner as the MPM list configured by the encoding device. That is, the MPM list can include intra prediction modes of peripheral blocks and can further include a specific intra prediction mode by a predetermined method.

[0133] The decoding device can determine the intra prediction mode of the current block based on the MPM list and the intra prediction mode information. As an example, when the value of the MPM flag is 1, the decoding device can derive the planar mode as the intra prediction mode of the current block (based on the not planar flag), or can derive the candidate pointed to by the MPM index among the MPM candidates in the MPM list as the intra prediction mode of the current block. Here, the MPM candidates can represent only the candidates included in the MPM list, or can include not only the candidates included in the MPM list but also the planar mode that can be applied when the value of the MPM flag is 1.

[0134] As another example, when the value of the MPM flag is 0, the decoding device can derive, as the intra prediction mode of the current block, the intra prediction mode pointed to by the remaining intra prediction mode information (which can be referred to as mpm remainder information) among the remaining intra prediction modes not included in the MPM list and the planner mode. On the other hand, as yet another example, when the intra prediction type of the current block is a specific type (e.g., LIP, MRL, or ISP, etc.), the decoding device can derive, without parsing / decoding / verifying the MPM flag, the candidate pointed to by the MPM flag in the planner mode or the MPM list as the intra prediction mode of the current block.

[0135] The coding device derives the surrounding reference samples of the current block (S810). When intra prediction is applied to the current block, the surrounding reference samples used for intra prediction of the current block can be derived. The surrounding reference samples of the current block can include a total of 2×nH samples adjacent to the left boundary of the current block with a size of nW×nH and adjacent to the bottom - left, samples adjacent to the top boundary of the current block and a total of 2×nW samples adjacent to the top - right, and 1 sample adjacent to the top - left of the current block. Alternatively, the surrounding reference samples of the current block can also include a plurality of rows of upper - side surrounding samples and a plurality of columns of left - side surrounding samples. Also, the surrounding reference samples of the current block can include a total of nH samples adjacent to the right boundary of the current block with a size of nW×nH, a total of nW samples adjacent to the bottom boundary of the current block, and 1 sample adjacent to the bottom - right of the current block.

[0136] On the one hand, when the MRL is applied (i.e., when the value of the MRL index is greater than 0), the peripheral reference samples can be located on lines 1 or 2, not line 0, adjacent to the current block on the left side / upper side. In this case, the number of peripheral reference samples can be further increased. On the other hand, when the ISP is applied, the peripheral reference samples can be derived in sub-partition units.

[0137] The coding device performs intra prediction on the current block to derive prediction samples (S820). The coding device can derive the prediction samples based on the intra prediction mode / type and the peripheral samples. The coding device can derive the reference samples according to the intra prediction mode of the current block among the peripheral reference samples of the current block, and derive the prediction samples of the current block based on the reference samples.

[0138] On the other hand, according to one embodiment, the BDPCM (block differential pulse coded modulation or Block-based Delta Pulse Code Modulation) technique can be used. BDPCM can also be named RDPCM (quantized Residual block-based Delta Pulse Code Modulation).

[0139] When predicting a block by applying BDPCM, the restored samples can be utilized to predict the lines or columns of the block line by line. At this time, the reference samples used can be samples that are not filtered. The BDPCM direction can indicate whether vertical or horizontal prediction is used. That is, when BDPCM is applied, the vertical or horizontal direction can be selected as the BDPCM direction, and prediction can be performed in the said BDPCM direction. The prediction error can be quantized in the spatial domain, and the samples can be restored by adding the prediction error inverse quantized to the prediction (i.e., the predicted samples). The said prediction error can mean the residual. As an alternative to such BDPCM, a quantized residual domain BDPCM can be proposed, and the prediction direction and signaling can be the same as those of the BDPCM applied in the spatial domain. That is, after stacking the quantized coefficients themselves like DPCM (Delta Pulse Code Modulation) via the quantized residual domain BDPCM, the residual can be restored via inverse quantization. Therefore, the quantized residual domain BDPCM can be used in the sense of applying DPCM at the residual coding end. The quantized residual domain used hereinafter is one in which the residual derived based on prediction is quantized without conversion, and means the domain for the quantized residual samples. For example, the quantized residual domain can include the quantized residual (or the quantized residual coefficients) for which the transform skip is applied, that is, the transform is skipped for the residual samples but quantization is applied. Or, for example, the quantized residual domain can include the quantized transform coefficients.

[0140] For a block of size M×N, for the left or upper boundary samples (i.e., the left peripheral samples or the upper peripheral samples), the intra prediction in the horizontal direction (copying the left peripheral sample line line by line to the prediction block) or the intra prediction in the vertical direction (copying the upper peripheral sample line line by line to the prediction block) is performed using the non-filtered samples among them, and the derived residual is r (i、j) (0 ≤ i ≤ M - 1, 0 ≤ j ≤ N - 1) can be assumed. Here, M can represent the number of rows or height, and N can represent the number of columns or width. And the residual r (i、j) The quantized value of can be assumed to be Q(r (i、j) )(0 ≤ i ≤ M - 1, 0 ≤ j ≤ N - 1). Here, the residual means the value of the difference between the original block and the prediction block value.

[0141] Then, if BDPCM is applied to the quantized residual samples, A modified array of M×N that constitutes JPEG2025106492000002.jpg10109 JPEG2025106492000003.jpg566 can be derived.

[0142] For example, if vertical BDPCM is signaled (i.e., when vertical BDPCM is applied), JPEG2025106492000004.jpg1082 can be derived as follows.

[0143]

Equation

[0144] That is, for example, when vertical BDPCM is applied, the encoding device can perform vertical intra prediction based on upper peripheral samples, and the quantized residual samples for the current block can be derived as described in Equation 1 above. Referring to Equation 1 above, the quantized residual samples of the rows excluding the first row of the current block can be derived as the difference between the quantized value for that position and the quantized value for the position in the previous row of that position (i.e., the upper peripheral position of that position).

[0145] Also, when applied similarly to horizontal prediction (i.e., when horizontal BDPCM is applied), the residual quantized samples can be derived as in the following equation.

[0146]

Equation

[0147] That is, for example, when horizontal BDPCM is applied, the encoding device can perform horizontal intra prediction based on left peripheral samples, and the quantized residual samples for the current block can be derived as described in Equation 2 above. Referring to Equation 2 above, the quantized residual samples of the columns excluding the first column of the current block can be derived as the difference between the quantized value for that position and the quantized value for the position in the previous column of that position (i.e., the left peripheral position of that position).

[0148] The quantized residual samples JPEG2025106492000007.jpg798 can be transmitted to the decoding device.

[0149] In the decoding device, in order to derive Q(r (i、j) )(0 ≤ i ≤ M - 1, 0 ≤ j ≤ N - 1), the operation can be performed in reverse.

[0150] For vertical prediction, the following formula can be applied.

[0151]

Equation

[0152] Also, for horizontal prediction, the following formula can be applied.

[0153]

Equation

[0154] The inverse-quantized quantized residual JPEG2025106492000010.jpg11116 is combined with the intra-block prediction value to derive the restored sample value.

[0155] The main advantage of such a technique is that inverse BDPCM can be performed by simply adding predictors during or after coefficient parsing.

[0156] As described above, BDPCM can be applied to the quantized residual domain, and the quantized residual domain can include quantized residuals (or quantized residual coefficients). At this time, conversion skip can be applied to the residuals. That is, when BDPCM is applied, conversion is skipped for the residual samples, and quantization can be applied. Alternatively, the quantized residual domain can also include quantized transform coefficients. The flag for whether BDPCM is applicable can be signaled at the sequence level (SPS), and such a flag can also be signaled only when it is signaled in the SPS that the conversion skip mode is possible. The said flag can be called the BDPCM usable flag or the SPS BDPCM usable flag.

[0157] When applying BDPCM, intra prediction can be performed on the entire block by sample copy in a prediction direction similar to the intra prediction direction (for example, vertical prediction or horizontal prediction). The residual, which is the difference value between the original and the predicted block, is skipped for conversion and quantized, and is the delta value, that is, the difference value, between the quantized residual and the predictor for the horizontal or vertical direction (that is, the quantized residual in the horizontal or vertical direction). JPEG2025106492000011.jpg995 can be coded.

[0158] If BDPCM is applicable, the CU size is smaller than or equal to MaxTsSize (maximum transform skip block size) for luma samples. When the CU is coded with intra prediction, flag information can be sent at the CU level. The flag information can be referred to as the BDPCM flag. Here, MaxTsSize can mean the maximum block size for which the transform skip mode is allowed. The flag information can indicate whether normal intra coding is applied or BDPCM is applied. If BDPCM is applied, a BDPCM prediction direction flag indicating whether the prediction direction is horizontal or vertical can be sent. The BDPCM prediction direction flag can also be referred to as the BDPCM direction flag. Thereafter, the block can be predicted through a normal horizontal or vertical intra prediction process using the reference samples that have not been filtered. Also, the residual is quantized, and the value of the difference between each quantized residual and its predictor, for example, the quantized residuals at the surrounding positions in the horizontal or vertical direction according to the BDPCM prediction direction, can be coded.

[0159] On the other hand, as will be described later, the above-mentioned BDPCM can be described in a standard document format.

[0160] For example, the syntax element for the above-mentioned BDPCM usable flag and the semantics for the syntax element can be represented as shown in the following table.

[0161]

Table 1

[0162]

Table 2

[0163] Table 1 shows the sps_bdpcm_enabled_flag signaled in the SPS (Sequence Parameter Set). If the syntax element sps_bdpcm_enabled_flag is 1, it represents flag information indicating whether BDPCM is applied to the coding unit where intra prediction is performed, that is, it indicates the existence of "intra_bdpcm_luma_flag" and "intra_bdpcm_chroma_flag" in the coding unit. The syntax element sps_bdpcm_enabled_flag can be a syntax element for the BDPCM usable flag described above. Also, if the syntax element "sps_bdpcm_enabled_flag" does not exist, its value can be regarded as 0.

[0164] Also, for example, the syntax elements for the above-mentioned BDPCM flag and BDPCM direction flag and the semantics for the syntax elements can be represented as shown in the following table.

[0165] [Table 3]

[0166] [Table 4]

[0167] The syntax element bdpcm_flag in Table 3 can indicate whether BDPCM is applied to the current block. The syntax element bdpcm_flag can be a syntax element for the BDPCM flag described above. For example, if the value of bdpcm_flag is 1, the BDPCM is applied to the current block, the transformation for the current block is skipped, and there may be a bdpcm_dir_flag indicating the prediction direction for the current block. Also, for example, if the value of bdpcm_flag is 0, it cannot be applied to the current block. Also, for example, if bdpcm_flag does not exist, this value can be regarded as 0. The current block can be a coding block. The bdpcm_dir_flag can indicate the prediction direction for the current block. For example, referring to Table 4, if the value of bdpcm_dir_flag is 1, the prediction direction for the current block can be the vertical direction, and if the value of bdpcm_dir_flag is 0, the prediction direction for the current block can be the horizontal direction. The syntax element bdpcm_flag can be a syntax element for the BDPCM flag described above, and the syntax element bdpcm_dir_flag can be a syntax element for the BDPCM direction flag described above.

[0168] Also, for example, the syntax elements for the BDPCM flag and BDPCM direction flag described above can be signaled separately for the luma component and chroma component. For example, the semantics for the syntax element can be represented as in the following table.

[0169] [Table 5]

[0170] [Table 6]

[0171] As described above, the syntax element intra_bdpcm_luma_flag in Table 5 can indicate whether BDPCM is applied to the current luma block, and intra_bdpcm_chroma_flag can indicate whether BDPCM is applied to the current luma block or the current chroma block. For example, if the value of intra_bdpcm_luma_flag or intra_bdpcm_chroma_flag is 1, the transformation for the coding block is skipped, and the prediction mode for the coding block can be set to horizontal or vertical directions by intra_bdpcm_luma_dir_flag or intra_bdpcm_chroma_dir_flag that represents the prediction direction. If intra_bdpcm_luma_flag or intra_bdpcm_chroma_flag does not exist, this value can be regarded as 0.

[0172] Also, for example, if the value of intra_bdpcm_luma_dir_flag or intra_bdpcm_chroma_dir_flag that represents the prediction direction is 0, it can indicate that the BDPCM prediction direction is horizontal, and if the value of intra_bdpcm_luma_dir_flag or intra_bdpcm_chroma_dir_flag is 1, it can indicate that the BDPCM prediction direction is vertical.

[0173] Also, when BDPCM is applied, an example of the inverse quantization process can be represented as shown in the following table.

[0174]

Table 7-1

Table 7-2

Table 7-3

[0175] Alternatively, when BDPCM is applied, an example of the inverse quantization process can also be expressed as shown in the following table.

[0176] [Table 8-1] [Table 8-2] [Table 8-3] [Table 8-4]

[0177] Referring to Table 7 or Table 8, if the value of bdpcm_flag is 1, the inverse quantized residual value d[x][y] can be derived based on the intermediate variable dz[x][y]. Here, x is the horizontal coordinate, increasing from left to right, y is the vertical coordinate, increasing from top to bottom, and the position within the two-dimensional block can be represented by (x, y). Also, the position within the two-dimensional block represents the (x, y) position when the upper left corner position of the block is set to (0, 0).

[0178] For example, if the value of bdpcm_dir_flag is 0, that is, if horizontal BDPCM is applied, the variable dz[x][y] can be derived based on TransCoeffLevel[xTbY][yTbY][cIdx][x][y] when x is 0, and based on dz[x - 1][y] + dz[x][y] when x is not 0. That is, when horizontal BDPCM is applied (the value of bdpcm_dir_flag is 0), the variable dz[x][y] of the sample located in the first column where x is 0 can be derived to TransCoeffLevel[xTbY][yTbY][cIdx][x][y] derived based on the residual information of the sample, and the variable dz[x][y] of the sample located in a column other than the first column where x is not 0 can be derived as the sum of dz[x - 1][y] of the left neighboring sample of the sample and dz[x][y] for the sample. Here, dz[x][y] for the sample added to the dz[x - 1][y] can be derived based on the residual information for the sample to be signaled.

[0179] Also, for example, if the value of bdpcm_dir_flag is 1, that is, if vertical BDPCM is applied, the variable dz[x][y] can be derived based on dz[x][y - 1] + dz[x][y]. That is, when vertical BDPCM is applied (the value of bdpcm_dir_flag is 1), the variable dz[x][y] of the sample located in the first row where y is 0 can be derived to TransCoeffLevel[xTbY][yTbY][cIdx][x][y] derived based on the residual information of the sample, and the variable dz[x][y] of the sample located in a row other than the first row where y is not 0 can be derived as the sum of dz[x][y - 1] of the upper neighboring sample of the sample and dz[x][y] for the sample. Here, dz[x][y] for the sample added to the dz[x][y - 1] can be derived based on the residual information for the sample to be signaled.

[0180] As described above, the residual at a specific position can be derived based on the sum of the residual at the previous position (i.e., the left or upper side) in the horizontal or vertical direction and the value received with the residual information at the specific position. This is because when BDPCM is applied, the difference value between the residual sample value at a specific position (x, y) and the residual sample value at the previous position in the horizontal or vertical direction (i.e., (x - 1, y) or (x, y - 1)) is signaled as residual information.

[0181] On the other hand, this document proposes the following solution regarding the method of applying BDPCM between residual signals in the process of coding the residual signal with conversion skipped. Different from the coefficients to which the conversion is applied, the residual signal with conversion skipped can be evenly distributed within a TU (transform unit), and the probability that the residual coefficient with conversion skipped is similar to the residual coefficients around the component is extremely high. Also, in the case of an intra-predicted conversion skip block, due to the distance to the prediction reference sample, the larger the block size, the higher the probability that the level of the residual generated at the lower right end of the block is larger than the level of the residual generated at the upper left end of the block. Therefore, according to BDPCM, in order to improve the coding efficiency by utilizing the characteristics of the above-described residual distribution, as described above, prediction between line-by-line residuals can be performed in the row or column direction. And the residual of the block to which the BDPCM is applied can be encoded / decoded with the residual coding syntax for conversion skip.

[0182] Therefore, BDPCM can be reasonably regarded as yet another transform skip method. Accordingly, this document proposes a method for determining whether BDPCM is applicable based on the conditions of transform skip. For example, as one embodiment, a scheme can be proposed to signal / parse / code a BDPCM flag indicating whether BDPCM is applicable when transform_skip_enabled_flag is true (i.e., when the value of transform_skip_enabled_flag is 1). Also, this embodiment can propose a scheme to signal / parse / code the BDPCM flag based on the size of the current block. For example, when the size of the current block is less than or equal to the maximum transform block size (i.e., when the width and height of the current block are less than or equal to the maximum transform block size), the BDPCM flag can be signaled / parsed / coded. Also, this embodiment can propose a scheme to signal / parse / code the BDPCM flag based on transform_skip_enabled_flag and / or the size of the current block. For example, when transform_skip_enabled_flag is true (i.e., when the value of transform_skip_enabled_flag is 1) and the size of the current block is less than or equal to the maximum transform block size (i.e., when the width and height of the current block are less than or equal to the maximum transform block size), the BDPCM flag can be signaled / parsed / coded.

[0183] For example, the syntax proposed in this embodiment can be as follows in the following table.

[0184]

Table 9

[0185] Referring to Table 9, the syntax element of the BDPCM flag can be intra_bdpcm_flag. Or, for example, the syntax element of the BDPCM flag can be intra_bdpcm_luma_flag. Referring to Table 9, the BDPCM flag for the current block can be signaled based on the transform_skip_enabled_flag and / or the size of the current block. For example, if the width and height of the current block are less than or equal to the maximum transform block size, the BDPCM flag can be signaled; if the width or height of the current block is greater than the maximum transform block size, the BDPCM flag cannot be signaled. Here, for example, the maximum transform block size can be derived based on the information representing the maximum transform block size. Also, for example, if the BDPCM flag does not exist (i.e., the BDPCM flag is not signaled), the BDPCM flag can be inferred as 0.

[0186] In addition, this document proposes the following solution regarding the method of applying BDPCM between residual signals in the process of coding a transform-skipped residual signal. As described above, the residual signal with transform skipped, unlike the coefficients to which the transform is applied, can be evenly distributed within a TU (transform unit), and the probability that the residual coefficients with transform skipped are similar to the residual coefficients around the component is extremely high. Also, in the case of an intra-predicted transform-skip block, due to the distance from the prediction reference sample, the larger the block size, the higher the probability that the level of the residual generated at the lower right end of the block is greater than the level of the residual generated at the upper left end of the block.

[0187] Accordingly, although this embodiment regards BDPCM as yet another conversion skip method, it regards BDPCM as a coding tool separate from the conversion skip mode and proposes a method for determining whether BDPCM is applicable. The method proposed in this embodiment is based on the fact that, as in the prior art, when it is in the BDPCM mode, it is inferred as the conversion skip mode.

[0188] For example, the syntax and semantics proposed in this embodiment can be as shown in the following table.

[0189]

Table 10

[0190]

Table 11

[0191] For example, according to this embodiment, the syntax element no_bdpcm_constraint_flag can be signaled. The syntax element no_bdpcm_constraint_flag can indicate whether BDPCM is constrained. For example, no_bdpcm_constraint_flag can be called the BDPCM constraint flag.

[0192] For example, referring to Table 11, no_bdpcm_constraint_flag being 1 can indicate that the value of the BDPCM availability flag is 0. That is, for example, no_bdpcm_constraint_flag being 1 can indicate that BDPCM is not available (for the entire image). no_bdpcm_constraint_flag being 0 can mean that no constraint conditions are imposed on BDPCM. Also, for example, the syntax element of the BDPCM availability flag can be bdpcm_enabled_flag. On the other hand, for example, the BDPCM availability flag can be defined in one or more of SPS (sequence parameter set), PPS (picture parameter set), VPS (Video Parameter Set), and Slice header. That is, for example, the BDPCM availability flag can be signaled via SPS (sequence parameter set), PPS (picture parameter set), VPS (video parameter set), and / or Slice header.

[0193] Also, for example, referring to Table 11, if the value of the BDPCM usable flag is 0 or the BDPCM flag does not exist, the BDPCM flag can be regarded as 0. Therefore, for example, if the value of no_bdpcm_constraint_flag described above is 1, the BDPCM usable flag can be 0, and the BDPCM flag can also be regarded as 0. Also, for example, the BDPCM flag can indicate whether BDPCM is applied to the current block. For example, a BDPCM flag of 0 can indicate that BDPCM is not applied to the current block, and a BDPCM flag of 1 can indicate that BDPCM is applied to the current block. That is, a BDPCM flag of 1 can indicate that the conversion for the current block is skipped and prediction for the current block is performed in the intra prediction mode represented by the BDPCM direction flag. The syntax element of the BDPCM direction flag can be intra_bdpcm_dir_flag, intra_bdpcm_luma_dir_flag, or intra_bdpcm_chroma_dir_flag.

[0194] Also, for example, referring to Table 11, if the conversion skip flag does not exist, the value of the BDPCM usable flag is 1, and if the value of BdcpmFlag is 1, the conversion skip flag can be regarded as 1. Also, for example, if the conversion skip flag does not exist, if the value of the BDPCM usable flag is 0 or the value of BdcpmFlag is 0, the conversion skip flag can be regarded as 0. Here, the value of BdcpmFlag can be set in the same way as the value of the BDPCM flag.

[0195] In addition, in the process of coding the conversion-skipped residual signal, this document proposes the following solution regarding the method of applying BDPCM between residual signals. As described above, different from the coefficients to which the conversion is applied, the residual signals with the conversion skipped can be evenly distributed within a TU (transform unit), and the probability that the residual coefficients with the conversion skipped are similar to the residual coefficients around the component is extremely high. Also, in the case of an intra-predicted conversion skip block, due to the distance from the prediction reference samples, the larger the block size, the higher the probability that the level of the residual generated at the lower right end of the block is larger than the level of the residual generated at the upper left end of the block.

[0196] Accordingly, although this embodiment regards BDPCM as another conversion skip method, it regards BDPCM as a separate coding tool from the conversion skip mode and proposes another method for determining whether BDPCM can be applied. The method proposed in this embodiment is based on the fact that BDPCM is performed completely independently of the conversion skip mode.

[0197] For example, the syntax and semantics proposed in this embodiment can be as shown in the following table.

[0198]

Table 12-1

Table 12-2

[0199]

Table 13

[0200] For example, according to this embodiment, the syntax element no_bdpcm_constraint_flag can be signaled. The syntax element no_bdpcm_constraint_flag can indicate whether BDPCM is constrained or not.

[0201] For example, referring to Table 13, a no_bdpcm_constraint_flag of 1 can indicate that the value of the BDPCM available flag is 0. A no_bdpcm_constraint_flag of 0 can impose no constraint conditions on BDPCM. Also, for example, the syntax element of the BDPCM available flag can be bdpcm_enabled_flag. On the other hand, for example, the BDPCM available flag can be defined in one or more of SPS (sequence parameter set), PPS (picture parameter set), VPS (video parameter set), and slice header. That is, for example, the BDPCM available flag can be signaled via SPS (sequence parameter set), PPS (picture parameter set), VPS (video parameter set), and / or slice header.

[0202] Also, for example, referring to Table 13, if the value of the BDPCM available flag is 0 or the BDPCM flag does not exist, the BDPCM flag can be regarded as 0. Therefore, for example, if the value of no_bdpcm_constraint_flag described above is 0, the BDPCM available flag can be 0, and the BDPCM flag can also be regarded as 0. Also, for example, the BDPCM flag can indicate whether BDPCM is applied to the current block. For example, a BDPCM flag of 0 can indicate that BDPCM is not applied to the current block, and a BDPCM flag of 1 can indicate that BDPCM is applied to the current block. That is, a BDPCM flag of 1 can indicate that the conversion of the current block is skipped and prediction for the current block is performed in the intra prediction mode represented by the BDPCM direction flag. The syntax element of the BDPCM direction flag can be intra_bdpcm_dir_flag, intra_bdpcm_luma_dir_flag, or intra_bdpcm_chroma_dir_flag.

[0203] Also, for example, referring to Table 13, if the conversion skip flag does not exist, the conversion skip flag can be regarded as 0. That is, the conversion skip mode and BDPCM can be performed independently.

[0204] In addition, in the process of coding the conversion-skipped residual signal, this document proposes the following solution regarding the method of applying BDPCM between residual signals. As described above, different from the coefficients to which the conversion is applied, the residual signal with the conversion skipped can be evenly distributed within a TU (transform unit), and the probability that the residual coefficient with the conversion skipped is similar to the residual coefficients around the component is extremely high. Also, in the case of an intra-predicted conversion skip block, due to the distance from the prediction reference sample, the larger the block size, the higher the probability that the level of the residual generated at the lower right end of the block is larger than the level of the residual generated at the upper left end of the block.

[0205] Accordingly, although this embodiment regards BDPCM as yet another conversion skip method, it regards it as a separate coding tool from the conversion skip mode and proposes another method for determining whether BDPCM can be applied. The method proposed in this embodiment is based on the fact that BDPCM is performed completely independently of the conversion skip mode, and different from the previously disclosed embodiments, a flag signaled in the High level syntax, that is, no_bdpcm_constraint_flag, can be not defined separately.

[0206] For example, the syntax and semantics proposed in this embodiment can be as shown in the following table.

[0207]

Table 14

[0208]

Table 15

[0209] Referring to Table 14, residual coding can be branched based on the BDPCM flag. That is, different syntax elements can be used for residual coding based on the value of the BDPCM flag (based on whether BDPCM is applicable).

[0210] For example, referring to Table 14, when the value of the conversion skip flag is 0 and the value of the BDPCM flag is 0 (that is, when conversion is applied and BDPCM is not applied), regular residual coding (RRC) can be applied to the current block. When the value of the conversion skip flag is 1 or the value of the BDPCM flag is 1 (that is, when conversion is skipped or BDPCM is applied), transform skip residual coding (TSRC) can be applied to the current block. That is, for example, when the value of the conversion skip flag is 0 and the value of the BDPCM flag is 0 (that is, when conversion is applied and BDPCM is not applied), the syntax elements of regular residual coding (RRC) can be signaled for the current block. When the value of the conversion skip flag is 1 or the value of the BDPCM flag is 1 (that is, when conversion is skipped or BDPCM is applied), the syntax elements of transform skip residual coding (TSRC) can be signaled for the current block. The regular residual coding can also be called general residual coding. Also, the regular residual coding can be called the regular residual coding syntax structure, and the transform skip residual coding can be called the transform skip residual coding syntax structure.

[0211] Also, for example, referring to Table 15, when there is no conversion skip flag, the conversion skip flag can be regarded as 0. That is, the conversion skip mode and BDPCM can be performed independently.

[0212] FIG. 9 schematically shows an image encoding method by an encoding device according to this document. The method disclosed in FIG. 9 can be performed by the encoding device disclosed in FIG. 2. Specifically, for example, S900 in FIG. 9 can be performed by the prediction unit of the encoding device, and S910 can be performed by the entropy encoding unit of the encoding device.

[0213] The encoding device generates a BDPCM constraint flag indicating whether BDPCM (Block-based Delta Pulse Code Modulation) is restricted for an image (S900). The encoding device can determine whether BDPCM (Block-based Delta Pulse Code Modulation) is restricted for an image. For example, the encoding device can determine whether the BDPCM is restricted in consideration of the characteristics of the image and the coding efficiency.

[0214] Thereafter, the encoding device can generate a BDPCM constraint flag indicating whether the BDPCM is constrained or not. The BDPCM constraint flag can indicate whether the BDPCM is constrained or not. For example, a BDPCM constraint flag with a value of 1 can indicate that the value of the BDPCM available flag is 0. That is, for example, a BDPCM constraint flag with a value of 1 can indicate that the BDPCM is not available (for the entire image). In other words, for example, when the value of the BDPCM constraint flag is 1, the BDPCM constraint flag can indicate that the BDPCM is not available (for the entire image). Therefore, when the value of the BDPCM constraint flag is 1, the BDPCM flag can not be signaled and the BDPCM flag can be inferred as 0. Also, for example, a BDPCM constraint flag with a value of 0 can impose no constraint conditions on the BDPCM. In other words, for example, when the value of the BDPCM constraint flag is 0, the BDPCM constraint flag can impose no constraint conditions on the BDPCM. The syntax element of the BDPCM constraint flag can be no_bdpcm_constraint_flag.

[0215] The encoding device encodes the image information including the BDPCM constraint flag (S910). The encoding device can encode the image information including the BDPCM constraint flag. That is, for example, the encoding device can encode a BDPCM constraint flag indicating whether the BDPCM is constrained or not for an image.

[0216] On the other hand, although not shown in the figure, the encoding device can generate a reconstructed picture in the image based on the BDPCM, and can generate and encode BDPCM related information. The image information can include the BDPCM related information.

[0217] For example, when the BDPCM is not restricted, that is, when the value of the BDPCM restriction flag is 0, the encoding device can generate a restored picture in the image based on the BDPCM, and can generate and encode BDPCM-related information. The image information can include the BDPCM-related information.

[0218] For example, when the value of the BDPCM restriction flag is 0, the encoding device can determine whether the width and height of the current block are smaller than or equal to the maximum transform block size. The encoding device can determine the maximum transform block size, and can generate and encode information representing the maximum transform block size. For example, the information representing the maximum transform block size can be signaled via high-level syntax. For example, the information representing the maximum transform block size can be signaled via SPS (Sequence Parameter Set, SPS). For example, the syntax element of the information representing the maximum transform block size can be sps_log2_transform_skip_max_size_minus2.

[0219] Also, for example, when the width and the height are smaller than or equal to the maximum transform block size, the encoding device can determine whether BDPCM is applicable to the current block, and can generate a BDPCM flag indicating whether BDPCM (Block-based Delta Pulse Code Modulation) is applicable to the current block. For example, when the width and the height are smaller than or equal to the maximum transform block size, the BDPCM-related information can include the BDPCM flag.

[0220] Also, for example, when the width or the height is greater than the maximum transform block size, that is, when at least one of the width and the height is greater than the maximum transform block size, the encoding device can determine that BDPCM is not applicable to the current block and cannot generate a BDPCM flag indicating whether BDPCM (Block-based Delta Pulse Code Modulation) is applicable to the current block. The BDPCM flag can be regarded as 0. That is, for example, when the width or the height is greater than the maximum transform block size, that is, when at least one of the width and the height is greater than the maximum transform block size, the BDPCM flag indicating whether BDPCM (Block-based Delta Pulse Code Modulation) is applicable to the current block cannot be signaled, and BDPCM cannot be applied to the current block. For example, when the width or the height is greater than the maximum transform block size, that is, when at least one of the width and the height is greater than the maximum transform block size, the BDPCM-related information can exclude the BDPCM flag for the current block.

[0221] Also, for example, when the value of the BDPCM flag is 0, the BDPCM flag can indicate that BDPCM is not applied to the current block. When the value of the BDPCM flag is 1, it can indicate that BDPCM is applied to the current block and that there is a BDPCM direction flag for the current block. That is, for example, when the value of the BDPCM flag is 0, the BDPCM flag can indicate that BDPCM is not applied to the current block, and that general intra prediction, IBC prediction, inter prediction, or palette prediction is performed. When the value of the BDPCM flag is 1, it can indicate that BDPCM is applied to the current block and that there is a BDPCM direction flag for the current block. For example, the syntax element of the BDPCM flag can be the above-described bdpcm_flag, intra_bdpcm_luma_flag, or intra_bdpcm_chroma_flag. Also, for example, the BDPCM flag can be signaled in units of CU (coding unit). For example, the current block can be a coding block.

[0222] Also, for example, the encoding device can determine whether BDPCM is applied to the current block and can determine the direction in which the BDPCM is performed. Also, for example, the encoding device can generate and encode a BDPCM direction flag that represents the prediction direction for the current block. The BDPCM-related information can include the BDPCM direction flag.

[0223] For example, the BDPCM direction flag can represent the prediction direction for the current block. For example, when the value of the BDPCM flag is 1, the encoding device can generate and encode the BDPCM direction flag. For example, the BDPCM direction flag can represent the vertical direction or the horizontal direction as the prediction direction for the current block. For example, when the value of the BDPCM direction flag is 0, the BDPCM direction flag can represent that the prediction direction for the current block is the horizontal direction, and when the value of the BDPCM direction flag is 1, the BDPCM direction flag can represent that the prediction direction for the current block is the vertical direction. For example, the syntax element of the BDPCM direction flag can be the above-mentioned bdpcm_dir_flag, intra_bdpcm_luma_dir_flag, or intra_bdpcm_chroma_dir_flag.

[0224] Also, for example, the encoding device can perform intra prediction on the current block based on the prediction direction in which BDPCM is performed to derive prediction samples. For example, the prediction direction can be the vertical direction or the horizontal direction, and prediction samples for the current block can be generated by the intra prediction mode accordingly.

[0225] For example, when the value of the BDPCM direction flag is 0, that is, for example, when the prediction direction for the current block is derived as the horizontal direction, the encoding device can derive the prediction samples of the current block based on the horizontal intra prediction mode. In other words, for example, when the value of the BDPCM direction flag is 0, that is, for example, when the prediction direction for the current block is derived as the horizontal direction, the encoding device can perform intra prediction based on the left peripheral samples of the current block to derive the prediction samples of the current block. For example, when the prediction direction for the current block is derived as the horizontal direction, the encoding device can derive the sample value of the left peripheral sample in the same row as the prediction sample as the sample value of the prediction sample.

[0226] Also, for example, when the value of the BDPCM direction flag is 1, that is, for example, when the prediction direction for the current block is derived as the vertical direction, the encoding device can derive the prediction samples of the current block based on the vertical intra prediction mode. In other words, for example, when the value of the BDPCM direction flag is 1, that is, for example, when the prediction direction for the current block is derived as the vertical direction, the encoding device can derive the prediction samples of the current block based on the upper peripheral samples of the current block. For example, when the prediction direction for the current block is derived as the vertical direction, the encoding device can derive the sample value of the upper peripheral sample in the same column as the prediction sample as the sample value of the prediction sample.

[0227] On the one hand, the tree type of the current block can be classified into SINGLE_TREE or DUAL_TREE depending on whether the luma block and the corresponding chroma block have individual split structures. If the chroma block has the same split structure as the luma block, it can be represented as a single tree; if the chroma component block has a different split structure from the luma component block, it can be represented as a dual tree. By way of example, BDPCM can be individually applied to the luma block or chroma block of the current block.

[0228] When the tree structure of the current block is a dual tree, BDPCM can be applied to only one of the component blocks. Also, when the current block has a single tree structure, BDPCM can be applied to only one of the component blocks.

[0229] Also, for example, the encoding device can derive the residual sample of the current block based on the prediction sample. For example, the encoding device can derive the residual sample by subtracting the prediction sample from the original sample for the current block.

[0230] On the other hand, for example, the image information can include the residual information. For example, the encoding device can derive the residual coefficient of the current block based on the residual sample. For example, when BDPCM is applied to the current block, the encoding device can determine that no transformation is applied to the current block. In this case, for example, the encoding device can quantize the residual sample to derive the residual coefficient. Here, for example, the block to which no transformation is applied can be represented as a transform skip block. That is, for example, the current block can be a transform skip block.

[0231] Thereafter, for example, the encoding device can encode residual information for the residual coefficient. For example, the residual information can include residual information for the residual coefficient of the residual sample.

[0232] For example, the residual information may include syntax elements for the residual samples of the current block, and a difference between the residual coefficient value of the target residual sample and the residual coefficient value of the left neighboring residual sample or the upper neighboring residual sample of the target residual sample may be derived based on the syntax elements for the target residual sample. For example, when the prediction direction of the current block is horizontal, a difference between the residual coefficient value of the target residual sample and the residual coefficient value of the left neighboring residual sample of the target residual sample may be derived based on the syntax elements for the target residual sample. That is, for example, when the prediction direction of the current block is horizontal, the syntax element for the target residual sample can represent a difference between the residual coefficient value of the target residual sample and the residual coefficient value of the left neighboring residual sample of the target residual sample. Also, for example, when the prediction direction of the current block is vertical, a difference between the residual coefficient value of the target residual sample and the residual coefficient value of the upper neighboring residual sample of the target residual sample may be derived based on the syntax elements for the target residual sample. That is, for example, when the prediction direction of the current block is vertical, the syntax element for the target residual sample can represent a difference between the residual coefficient value of the target residual sample and the residual coefficient value of the upper neighboring residual sample of the target residual sample. Further, when the target residual sample is located in the first row or column of the current block, the residual coefficient value of the target residual sample may be derived based on the syntax elements for the target residual sample. That is, when the target residual sample is located in the first row or column of the current block, the syntax element for the target residual sample can represent the residual coefficient value of the target residual sample.

[0233] For example, the residual information can include syntax elements such as transform_skip_flag, last_sig_coeff_x_prefix, last_sig_coeff_y_prefix, last_sig_coeff_x_suffix, last_sig_coeff_y_suffix, coded_sub_block_flag, sig_coeff_flag, par_level_flag, abs_level_gt1_flag, abs_level_gtX_flag, abs_remainder, coeff_sign_flag, dec_abs_level, and / or mts_idx.

[0234] Specifically, for example, the residual information can include a transform skip flag for the current block. The transform skip flag can indicate whether the transform is applicable to the current block. That is, the transform skip flag can indicate whether the transform has been applied to the residual coefficients of the current block. Also, for example, when the BDPCM is applied to the current block, the transform skip flag for the current block can be not signaled, and the value of the transform skip flag can be inferred as 1. That is, when the BDPCM is applied to the current block, the residual information can not include the transform skip flag for the current block, the value of the transform skip flag can be inferred as 1, and the current block can be a transform skip block. The syntax element representing the transform skip flag can be the transform_skip_flag described above.

[0235] Also, for example, the residual information may include position information representing the position of the last non-zero residual coefficient in the residual coefficient array of the current block. That is, the residual information may include position information representing the position of the last non-zero residual coefficient in the scanning order of the current block. The position information may include information representing a prefix of the column position of the last non-zero residual coefficient, information representing a prefix of the row position of the last non-zero residual coefficient, information representing a suffix of the column position of the last non-zero residual coefficient, and information representing a suffix of the row position of the last non-zero residual coefficient. The syntax elements for the position information may be last_sig_coeff_x_prefix, last_sig_coeff_y_prefix, last_sig_coeff_x_suffix, and last_sig_coeff_y_suffix. On the other hand, non-zero residual coefficients may also be referred to as significant coefficients. Also, for example, when the current block is a transform skip block, the residual information may not include position information representing the position of the last non-zero residual coefficient in the residual coefficient array of the current block.

[0236] Also, for example, the residual information may include a valid coefficient flag indicating whether the residual coefficient of the residual sample of the current block is a non-zero residual coefficient, a parity level flag for the parity of the coefficient level with respect to the residual coefficient, a first coefficient level flag indicating whether the coefficient level is greater than a first threshold, and a second coefficient level flag indicating whether the coefficient level of the residual coefficient is greater than a second threshold. Here, the valid coefficient flag can be sig_coeff_flag, the parity level flag can be par_level_flag, the first coefficient level flag can be abs_level_gt1_flag, and the second coefficient level flag can be abs_level_gt3_flag or abs_level_gtx_flag.

[0237] Also, for example, the residual information may include a sign flag indicating the sign of the residual coefficient of the residual sample of the current block. The sign flag can be coeff_sign_flag.

[0238] Also, for example, the residual information may include coefficient value related information with respect to the value of the residual coefficient of the residual sample of the current block. The coefficient value related information can be abs_remainder and / or dec_abs_level.

[0239] On the other hand, the bitstream including the image information can be transmitted to a decoding device via a network or a (digital) storage medium. Here, the network can include a broadcast network and / or a communication network, etc., and the digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.

[0240] FIG. 10 schematically shows an encoding apparatus that performs the image encoding method according to this document. The method disclosed in FIG. 9 can be performed by the encoding apparatus disclosed in FIG. 10. Specifically, for example, the prediction unit of the encoding apparatus in FIG. 10 can perform S900 in FIG. 9, and the entropy encoding unit of the encoding apparatus can perform S910.

[0241] FIG. 11 schematically shows an image decoding method by the decoding apparatus according to this document. The method disclosed in FIG. 11 can be performed by the decoding apparatus disclosed in FIG. 3. Specifically, for example, S1100 to S1110 in FIG. 11 can be performed by the entropy decoding unit of the decoding apparatus, and S1120 in FIG. 11 can be performed by the prediction unit, residual processing unit, and addition unit of the decoding apparatus.

[0242] The decoding apparatus acquires image information including a BDPCM constraint flag indicating whether BDPCM (Block-based Delta Pulse Code Modulation) is constrained for an image (S1100).

[0243] The decoding device can acquire image information via a bitstream. For example, the decoding device can acquire image information including a BDPCM constraint flag indicating whether BDPCM (Block-based Delta Pulse Code Modulation) is restricted for an image. The image information can include a BDPCM constraint flag indicating whether the BDPCM is restricted. That is, for example, the BDPCM constraint flag can be signaled. The BDPCM constraint flag can indicate whether the BDPCM is restricted. For example, a BDPCM constraint flag with a value of 1 can indicate that the value of the BDPCM available flag is 0. That is, for example, a BDPCM constraint flag with a value of 1 can indicate that (for the entire image) the BDPCM is not available. In other words, for example, when the value of the BDPCM constraint flag is 1, the BDPCM constraint flag can indicate that (for the entire image) the BDPCM is not available. Therefore, when the value of the BDPCM constraint flag is 1, the BDPCM flag can not be signaled and the BDPCM flag can be inferred as 0. Also, for example, a BDPCM constraint flag with a value of 0 can not impose a constraint condition on the BDPCM. In other words, for example, when the value of the BDPCM constraint flag is 0, the BDPCM constraint flag can not impose a constraint condition on the BDPCM. The syntax element of the BDPCM constraint flag can be no_bdpcm_constraint_flag.

[0244] The decoding device acquires BDPCM-related information for the current block based on the BDPCM constraint flag (S1110).

[0245] For example, when the value of the BDPCM constraint flag is 1, the BDPCM-related information for the current block can not be signaled.

[0246] Also, for example, when the value of the BDPCM constraint flag is 0, the BDPCM-related information for the current block can be signaled. For example, the BDPCM-related information can include a BDPCM flag indicating whether BDPCM (Block-based Delta Pulse Code Modulation) can be applied to the current block.

[0247] Specifically, for example, the decoding device can determine whether the width and height of the current block are smaller than or equal to the maximum transform block size. The maximum transform block size can be derived based on the information representing the maximum transform block size. For example, the information representing the maximum transform block size can be signaled via high-level syntax. For example, the information representing the maximum transform block size can be signaled via SPS (Sequence Parameter Set). For example, the syntax element of the information representing the maximum transform block size can be sps_log2_transform_skip_max_size_minus2.

[0248] For example, when the width and the height are smaller than or equal to the maximum transform block size, the decoding device can obtain a BDPCM flag indicating whether BDPCM (Block-based Delta Pulse Code Modulation) can be applied to the current block. That is, for example, when the width and the height are smaller than or equal to the maximum transform block size, the BDPCM-related information can include the BDPCM flag. Or, when the width or the height is larger than the maximum transform block size, that is, when at least one of the width and the height is larger than the maximum transform block size, the BDPCM flag indicating whether BDPCM (Block-based Delta Pulse Code Modulation) can be applied to the current block cannot be obtained. For example, when the width or the height is larger than the maximum transform block size, that is, when at least one of the width and the height is larger than the maximum transform block size, the BDPCM flag indicating whether BDPCM (Block-based Delta Pulse Code Modulation) can be applied to the current block cannot be signaled, and BDPCM cannot be applied to the current block. That is, for example, when the width or the height is larger than the maximum transform block size, that is, when at least one of the width and the height is larger than the maximum transform block size, the image information can exclude the BDPCM flag for the current block.

[0249] Also, for example, when the value of the BDPCM flag is 0, the BDPCM flag can indicate that BDPCM is not applied to the current block. When the value of the BDPCM flag is 1, BDPCM is applied to the current block, and it can indicate that there is a BDPCM direction flag for the current block. That is, for example, when the value of the BDPCM flag is 0, the BDPCM flag can indicate that BDPCM is not applied to the current block, and general intra prediction, IBC prediction, inter prediction, or palette prediction is performed. When the value of the BDPCM flag is 1, BDPCM is applied to the current block, and it can indicate that there is a BDPCM direction flag for the current block. The BDPCM-related information can include the BDPCM direction flag. For example, the syntax element of the BDPCM flag can be the above-described bdpcm_flag, intra_bdpcm_luma_flag, or intra_bdpcm_chroma_flag. Also, for example, the BDPCM flag can be signaled in units of CU (coding unit). For example, the current block can be a coding block.

[0250] Also, for example, the BDPCM direction flag can represent a vertical direction or a horizontal direction as the prediction direction for the current block. For example, when the value of the BDPCM direction flag is 0, the BDPCM direction flag can indicate that the prediction direction for the current block is the horizontal direction. When the value of the BDPCM direction flag is 1, the BDPCM direction flag can indicate that the prediction direction for the current block is the vertical direction. For example, the syntax element of the BDPCM direction flag can be the above-described bdpcm_dir_flag, intra_bdpcm_luma_dir_flag, or intra_bdpcm_chroma_dir_flag.

[0251] The decoding device generates a restored sample for the current block based on the BDPCM-related information (S1120). The decoding device can generate a restored sample for the current block based on the BDPCM-related information.

[0252] For example, the decoding device can generate a predicted sample for the current block based on the BDPCM-related information.

[0253] For example, when the value of the BDPCM direction flag is 0, that is, for example, when the BDPCM direction flag indicates that the prediction direction for the current block is the horizontal direction, the decoding device can derive the predicted sample of the current block based on the horizontal intra prediction mode. In other words, for example, when the value of the BDPCM direction flag is 0, that is, for example, when the BDPCM direction flag indicates that the prediction direction for the current block is the horizontal direction, the decoding device can perform intra prediction based on the left peripheral samples of the current block to derive the predicted sample of the current block. For example, when the prediction direction for the current block is derived as the horizontal direction, the decoding device can derive the sample value of the left peripheral sample in the same row as the predicted sample as the sample value of the predicted sample.

[0254] Also, for example, when the value of the BDPCM direction flag is 1, that is, for example, when the BDPCM direction flag indicates that the prediction direction for the current block is the vertical direction, the decoding device can derive the prediction samples of the current block based on the vertical intra prediction mode. In other words, for example, when the value of the BDPCM direction flag is 1, that is, for example, when the BDPCM direction flag indicates that the prediction direction for the current block is the vertical direction, the decoding device can derive the prediction samples of the current block based on the upper peripheral samples of the current block. For example, when the prediction direction for the current block is derived as the vertical direction, the decoding device can derive the sample value of the upper peripheral sample in the same column as the prediction sample as the sample value of the prediction sample.

[0255] Thereafter, the decoding device can derive the restored samples or the restored picture of the current block based on the prediction samples. For example, the decoding device can derive the restored samples through the addition of the prediction samples and the residual samples of the current block.

[0256] On the other hand, for example, the decoding device can derive the residual samples of the current block based on the residual information.

[0257] For example, when BDPCM is applied to the current block, the residual information can include syntax elements for the residual samples of the current block (i.e., when BDPCM is applied to the current block, the residual information can include syntax elements for the target residual samples of the current block), and the syntax elements for the target residual samples can represent the difference between the residual coefficient value of the target residual sample and the residual coefficient value of the left neighboring residual sample or the upper neighboring residual sample of the target residual sample. That is, for example, when BDPCM is applied to the current block, the residual information can include syntax elements for the target residual samples of the current block, and based on the syntax elements for the target residual samples, the difference between the residual coefficient value of the target residual sample and the residual coefficient value of the left neighboring residual sample or the upper neighboring residual sample of the target residual sample can be derived.

[0258] For example, when BDPCM is applied to the current block and the prediction direction for the current block is the horizontal direction, the syntax element for the target residual sample can represent the difference between the residual coefficient value of the target residual sample and the residual coefficient value of the left neighboring residual sample of the target residual sample. That is, for example, based on the syntax element for the target residual sample, the difference between the residual coefficient value of the target residual sample and the residual coefficient value of the left neighboring residual sample of the target residual sample can be derived. Then, the residual coefficient of the target residual sample can be derived as the sum of the residual coefficient value of the left neighboring residual sample of the target residual sample and the difference. Here, the target residual sample can be a residual sample within a column other than the first column of the current block. For example, the residual coefficient of the target residual sample can be derived based on the above-described Equation 4. On the other hand, for example, when the target residual sample is a residual sample within the first column of the current block, the residual coefficient of the target residual sample can be derived based on the syntax element of the target residual sample.

[0259] Also, for example, when BDPCM is applied to the current block and the prediction direction for the current block is the vertical direction, the syntax element for the target residual sample can represent the difference between the residual coefficient value of the target residual sample and the residual coefficient value of the upper neighboring residual sample of the target residual sample. That is, for example, based on the syntax element for the target residual sample, the difference between the residual coefficient value of the target residual sample and the residual coefficient value of the upper neighboring residual sample of the target residual sample can be derived. Then, the residual coefficient of the target residual sample can be derived by summing the residual coefficient value of the upper neighboring residual sample of the target residual sample and the difference. Here, the target residual sample can be a residual sample within a row other than the first row of the current block. For example, the residual coefficient of the target residual sample can be derived based on the above-described Equation 3. On the other hand, for example, when the target residual sample is a residual sample within the first row of the current block, the residual coefficient of the target residual sample can be derived based on the syntax element of the target residual sample.

[0260] Thereafter, for example, the decoding device can derive the target residual sample by inverse quantizing the residual coefficient. That is, for example, the target residual sample can be derived by inverse quantizing the residual coefficient.

[0261] On the other hand, although not illustrated in the drawings, for example, the decoding device can obtain the residual information for the current block based on the BDPCM flag. For example, when the BDPCM flag indicates that BDPCM is applied to the current block, that is, when BDPCM is applied to the current block, the residual information can include syntax elements for the residual samples of the current block, and based on the syntax elements for the target residual sample, the difference between the residual coefficient value of the target residual sample and the residual coefficient value of the left peripheral residual sample or the upper peripheral residual sample of the target residual sample can be derived. For example, when the prediction direction of the current block is the horizontal direction, that is, when the prediction direction of the current block is derived as the horizontal direction based on the BDPCM direction flag, the difference between the residual coefficient value of the target residual sample and the residual coefficient value of the left peripheral residual sample of the target residual sample can be derived based on the syntax elements for the target residual sample. Also, for example, when the prediction direction of the current block is the vertical direction, that is, when the prediction direction of the current block is derived as the vertical direction based on the BDPCM direction flag, the difference between the residual coefficient value of the target residual sample and the residual coefficient value of the upper peripheral residual sample of the target residual sample can be derived based on the syntax elements for the target residual sample. Further, when the target residual sample is located in the first row or column of the current block, the residual coefficient value of the target residual sample can be derived based on the syntax elements for the target residual sample.

[0262] For example, the residual information can include syntax elements such as transform_skip_flag, last_sig_coeff_x_prefix, last_sig_coeff_y_prefix, last_sig_coeff_x_suffix, last_sig_coeff_y_suffix, coded_sub_block_flag, sig_coeff_flag, par_level_flag, abs_level_gt1_flag, abs_level_gtX_flag, abs_remainder, coeff_sign_flag, dec_abs_level, and / or mts_idx.

[0263] Specifically, for example, the residual information can include a transform skip flag for the current block. The transform skip flag can indicate whether transformation can be applied to the current block. That is, the transform skip flag can indicate whether transformation has been applied to the residual coefficients of the current block. Also, for example, when the value of the BDPCM flag is 1, that is, when BDPCM is applied to the current block, the transform skip flag for the current block can be not signaled, and the value of the transform skip flag can be inferred as 1. That is, when the value of the BDPCM flag is 1, that is, when BDPCM is applied to the current block, the image information can not include the transform skip flag for the current block, the value of the transform skip flag can be inferred as 1, and the current block can be a transform skip block. The syntax element representing the transform skip flag can be the above-mentioned transform_skip_flag.

[0264] Also, for example, the residual information may include position information representing the position of the last non-zero residual coefficient in the residual coefficient array of the current block. That is, the residual information may include position information representing the position of the last non-zero residual coefficient in the scanning order of the current block. The position information may include information representing the prefix of the column position of the last non-zero residual coefficient, information representing the prefix of the row position of the last non-zero residual coefficient, information representing the suffix of the column position of the last non-zero residual coefficient, and information representing the suffix of the row position of the last non-zero residual coefficient. The syntax elements for the position information may be last_sig_coeff_x_prefix, last_sig_coeff_y_prefix, last_sig_coeff_x_suffix, and last_sig_coeff_y_suffix. On the other hand, the non-zero residual coefficient may also be called a significant coefficient. Also, for example, when the current block is a transform skip block, the residual information may not include position information representing the position of the last non-zero residual coefficient in the residual coefficient array of the current block.

[0265] Also, for example, the residual information can include a valid coefficient flag indicating whether the residual coefficient of the residual sample of the current block is a non-zero residual coefficient, a parity level flag for the parity of the coefficient level with respect to the residual coefficient, a first coefficient level flag indicating whether the coefficient level is greater than a first threshold, and a second coefficient level flag indicating whether the coefficient level of the residual coefficient is greater than a second threshold. Here, the valid coefficient flag can be sig_coeff_flag, the parity level flag can be par_level_flag, the first coefficient level flag can be abs_level_gt1_flag, and the second coefficient level flag can be abs_level_gt3_flag or abs_level_gtx_flag.

[0266] Also, for example, the residual information can include a sign flag indicating the sign of the residual coefficient of the residual sample of the current block. The sign flag can be coeff_sign_flag.

[0267] Also, for example, the residual information can include coefficient value related information for the value of the residual coefficient of the residual sample of the current block. The coefficient value related information can be abs_remainder and / or dec_abs_level.

[0268] The decoding device can derive the restored sample through addition of the predicted sample and the residual sample. Thereafter, in-loop filtering procedures such as deblocking filtering, SAO, and / or ALF procedures can be applied to the restored sample, as described above, in order to improve the subjective / objective image quality as necessary.

[0269] FIG. 12 schematically shows a decoding apparatus that performs the image decoding method according to this document. The method disclosed in FIG. 11 can be performed by the decoding apparatus disclosed in FIG. 12. Specifically, for example, the entropy decoding unit of the decoding apparatus in FIG. 12 can perform S1100 to S1110 in FIG. 11, and the prediction unit, residual processing unit, and addition unit of the decoding apparatus in FIG. 12 can perform S1120 in FIG. 11.

[0270] According to the above-mentioned document, the BDPCM flag is signaled based on the size of the current block and the maximum transform block size, and the size of the current block and the maximum transform block size can be considered for BDPCM flag signaling and determination of whether BDPCM can be applied. Through this, the amount of bits for BDPCM can be reduced, and the overall coding efficiency can be improved.

[0271] Also, according to this document, a syntax element indicating whether BDPCM is restricted for an image can be signaled, and through this, it is possible to determine whether BDPCM can be executed for the image with one syntax element, whereby the overall image coding efficiency can be improved.

[0272] In the foregoing embodiments, the method has been described based on a flowchart in a series of steps or blocks, but this document is not limited to the order of the steps, and a certain step can occur in a different order or simultaneously with steps different from the foregoing. Also, those skilled in the art can understand that the steps shown in the flowchart are not exclusive, other steps are included, or one or more steps of the flowchart can be deleted without affecting the scope of this document.

[0273] The embodiments described in this document can be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional units illustrated in each drawing can be implemented and executed on a computer, processor, microprocessor, controller, or chip. In this case, information for implementation (e.g., information on instructions) or algorithms can be stored in a digital storage medium.

[0274] In addition, the decoding device and encoding device to which the embodiments of this document are applied can be included in multimedia broadcast transmission / reception devices, mobile communication terminals, home cinema video devices, digital cinema video devices, surveillance cameras, video intercom devices, real-time communication devices such as video communication, mobile streaming devices, storage media, camcorders, video-on-demand (VoD) service providing devices, over-the-top (OTT) video devices, Internet streaming service providing devices, three-dimensional (3D) video devices, picture phone video devices, transportation means terminals (e.g., vehicle terminals, airplane terminals, ship terminals, etc.), and medical video devices, etc., and can be used to process video signals or data signals. For example, as an over-the-top (OTT) video device, it can be equipped with a game console, Blu-ray player, Internet-connected TV, home theater system, smartphone, tablet PC, digital video recorder (DVR), etc.

[0275] In addition, the processing method to which the embodiments of this document are applied can be produced in the form of a program executed by a computer and can be stored in a computer-readable recording medium. Multimedia data having the data structure according to this document can also be stored in a computer-readable recording medium. The computer-readable recording medium includes all types of storage devices and distributed storage devices in which data that can be read by a computer is stored. The computer-readable recording medium can include, for example, Blu-ray Disc (BD), Universal Serial Bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices. Further, the computer-readable recording medium includes a medium realized in the form of a carrier wave (for example, transmission via the Internet). Also, a bitstream generated by an encoding method can be stored in a computer-readable recording medium or transmitted via a wired or wireless communication network.

[0276] In addition, the embodiments of this document can be realized by a computer program product with program code, and the program code can be executed by a computer according to the embodiments of this document. The program code can be stored on a carrier readable by a computer.

[0277] FIG. 13 exemplarily shows a structural diagram of a content streaming system to which the embodiments of this document are applied.

[0278] The content streaming system to which the embodiments of this document are applied can generally include an encoding server, a streaming server, a web server, a media repository, a user device, and a multimedia input device.

[0279] The encoding server compresses the content input from a multimedia input device such as a smartphone, camera, camcorder, etc. into digital data to generate a bitstream, and serves to transmit this to the streaming server. As another example, when a multimedia input device such as a smartphone, camera, camcorder, etc. directly generates a bitstream, the encoding server can be omitted.

[0280] The bitstream can be generated by an encoding method or a bitstream generation method to which the embodiments of this document are applied, and the streaming server can temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0281] The streaming server transmits multimedia data to a user device based on a user request via a web server, and the web server serves as a medium to inform the user of what services are available. When the user requests a desired service from the web server, the web server transmits this to the streaming server, and the streaming server transmits multimedia data to the user. At this time, the content streaming system can include another control server, and in this case, the control server serves to control commands / responses between each device in the content streaming system.

[0282] The streaming server can receive content from a media repository and / or an encoding server. For example, when it comes to receiving content from the encoding server, the content can be received in real time. In this case, in order to provide a smooth streaming service, the streaming server can store the bitstream for a certain period of time.

[0283] Examples of the user device include mobile phones, smartphones, laptop computers, digital broadcast terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation devices, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, head mounted displays (HMDs)), digital TVs, desktop computers, digital signage, etc. Each server in the content streaming system can be operated as a distributed server, and in this case, the data received by each server can be distributedly processed.

[0284] The claims described in this specification can be combined in various ways. For example, the technical features of the method claims in this specification can be combined and realized as a device, and the technical features of the device claims in this specification can be combined and realized as a method. Also, the technical features of the method claims in this specification and the technical features of the device claims can be combined and realized as a device, and the technical features of the method claims in this specification and the technical features of the device claims can be combined and realized as a method.

Claims

1. In a method for decoding an image performed by a decoding apparatus, a step of obtaining image information including a BDPCM constraint flag indicating whether BDPCM (Block-based Delta Pulse Code Modulation) is restricted for the image; a step of obtaining BDPCM-related information for a current block, wherein the BDPCM-related information includes at least one of a BDPCM flag indicating whether the BDPCM is applied to the current block or a BDPCM direction flag indicating a prediction direction for the current block; a step of generating a restored sample for the current block based on the BDPCM-related information; comprising, wherein the BDPCM constraint flag equal to 1 indicates that a constraint that the BDPCM usable flag must be equal to 0 is imposed, and the BDPCM constraint flag equal to 0 indicates that the constraint is not imposed; the BDPCM usable flag equal to 1 indicates that the BDPCM is usable, and the BDPCM usable flag equal to 0 indicates that the BDPCM is not usable; the BDPCM flag equal to 1 indicates that the BDPCM is applied to the current block, and the BDPCM flag equal to 0 indicates that the BDPCM is not applied to the current block; the BDPCM direction flag is signaled based on the BDPCM flag being equal to 1; when the BDPCM constraint flag is equal to 1, the BDPCM usable flag is equal to 0; a method, wherein when the BDPCM usable flag is equal to 0, the BDPCM flag is regarded as equal to 0.

2. The method according to claim 1, wherein when the BDPCM constraint flag is equal to 1, the BDPCM-related information for the current block is not signaled.

3. The step of generating the restored sample for the current block based on the BDPCM-related information includes: a step of deriving a prediction sample of the current block based on an intra prediction mode derived based on the BDPCM direction flag; a step of deriving the restored sample based on the prediction sample; The method according to claim 1.

4. The BDPCM direction flag equal to 0 indicates that the prediction direction for the current block is the horizontal direction, The BDPCM direction flag equal to 1 indicates that the prediction direction for the current block is the vertical direction, according to the method of claim 3.

5. Based on the BDPCM direction flag being equal to 0, the intra prediction mode of the current block is derived as a horizontal intra prediction mode, Based on the BDPCM direction flag being equal to 1, the intra prediction mode of the current block is derived as a vertical intra prediction mode, according to the method of claim 3.

6. The step of generating the restored sample based on the predicted sample includes deriving a target residual sample of the current block based on the residual information regarding the current block, and deriving a restored sample by adding the predicted sample for the target residual sample and the target residual sample, according to the method of claim 4.

7. When BDPCM is applied to the current block and the prediction direction for the current block is the vertical direction, the residual information includes a syntax element for the target residual sample of the current block, The syntax element of the target residual sample represents the difference between the residual coefficient value of the target residual sample and the residual coefficient value of the upper peripheral residual sample of the target residual sample, according to the method of claim 6.

8. The difference is derived based on the syntax element for the target residual sample, The residual coefficient value of the target residual sample is derived by summing the residual coefficient value of the upper peripheral residual sample and the difference, according to the method of claim 7.

9. When BDPCM is applied to the current block and the prediction direction for the current block is the horizontal direction, the residual information includes a syntax element for the target residual sample of the current block, The syntax element for the target residual sample represents the difference between the residual coefficient value of the target residual sample and the residual coefficient value of the left peripheral residual sample of the target residual sample. The method according to claim 6.

10. In a method of encoding an image performed by an encoding device, generating a BDPCM constraint flag indicating whether BDPCM (Block-based Delta Pulse Code Modulation) is restricted for the image; encoding BDPCM-related information for a current block based on the BDPCM constraint flag, the BDPCM-related information including at least one of a BDPCM flag indicating whether the BDPCM is applied to the current block or a BDPCM direction flag indicating a prediction direction for the current block; including The BDPCM constraint flag equal to 1 indicates that a constraint that the BDPCM usable flag must be equal to 0 is imposed, and the BDPCM constraint flag equal to 0 indicates that the constraint is not imposed. The BDPCM usable flag equal to 1 indicates that the BDPCM is usable, and the BDPCM usable flag equal to 0 indicates that the BDPCM is not usable. The BDPCM flag equal to 1 indicates that the BDPCM is applied to the current block, and the BDPCM flag equal to 0 indicates that the BDPCM is not applied to the current block. The BDPCM direction flag is encoded based on the BDPCM flag being equal to 1. When the BDPCM constraint flag is equal to 1, the BDPCM usable flag is equal to 0. When the BDPCM usable flag is equal to 0, the BDPCM flag is regarded as equal to 0. A method.

11. A method for transmitting data for image information, Generating a BDPCM constraint flag indicating whether BDPCM (Block-based Delta Pulse Code Modulation) is restricted for an image, and encoding BDPCM-related information for a current block based on the BDPCM constraint flag, where the BDPCM-related information includes at least one of a BDPCM flag indicating whether the BDPCM is applicable to the current block or a BDPCM direction flag indicating a prediction direction for the current block, thereby generating a bitstream of the image information; Transmitting the bitstream of the image information; The BDPCM constraint flag equal to 1 indicates that a constraint that the BDPCM usable flag must be equal to 0 is imposed, and the BDPCM constraint flag equal to 0 indicates that the constraint is not imposed; The BDPCM usable flag equal to 1 indicates that the BDPCM is usable, and the BDPCM usable flag equal to 0 indicates that the BDPCM is not usable; The BDPCM flag equal to 1 indicates that the BDPCM is applied to the current block, and the BDPCM flag equal to 0 indicates that the BDPCM is not applied to the current block; The BDPCM direction flag is encoded based on the BDPCM flag being equal to 1; When the BDPCM constraint flag is equal to 1, the BDPCM usable flag is equal to 0; When the BDPCM usable flag is equal to 0, the BDPCM flag is regarded as equal to 0.

Images