Image decoding method and apparatus based on bdpcm

Abstract

According to the present disclosure, a method of image decoding performed by a decoding device includes determining whether a width and a height of a current block are less than or equal to a maximum transform block size, obtaining a block-based delta pulse code modulation (BDPCM) flag indicating whether BDPCM is applied to the current block when the width and the height are less than or equal to the maximum transform block size, obtaining a BDPCM direction flag indicating a prediction direction for the current block based on the BDPCM flag, deriving prediction samples of the current block based on an intra prediction mode derived based on the BDPCM direction flag, and deriving reconstructed samples of the current block based on the prediction samples.

Description

Technical Field

[0001] This disclosure relates to image compilation technology, and more specifically, to an image decoding method and apparatus that performs BDPCM to compile the current block in an image compilation system. Background Technology

[0002] Recently, the demand for high-resolution, high-quality images, such as HD (high-definition) and UHD (ultra-high-definition) images, has been growing across various fields. Because image data is high-resolution and high-quality, the amount of information or bits to be transmitted increases compared to traditional image data. Therefore, transmission and storage costs increase when using media such as traditional wired / wireless broadband lines to send image data or when storing image data using existing storage media.

[0003] Therefore, there is a need for efficient image compression technology to effectively transmit, store, and reproduce information from high-resolution, high-quality images. Summary of the Invention

[0004] Technical issues

[0005] The technical objective of this disclosure is to provide a method and apparatus for improving image compilation efficiency.

[0006] Another technical objective of this disclosure is to provide a method and apparatus for improving the efficiency of BDPCM.

[0007] Technical solution

[0008] According to embodiments of this disclosure, an image decoding method performed by a decoding device is provided. The method includes: determining whether the width and height of a current block are less than or equal to a maximum transform block size; when the width and height are less than or equal to the maximum transform block size, obtaining a BDPCM flag indicating whether block-based incremental pulse code modulation (BDPCM) is applied to the current block; based on the BDPCM flag, obtaining a BDPCM direction flag indicating a prediction direction for the current block; deriving prediction samples of the current block based on an intra-frame prediction mode derived from the BDPCM direction flag; and deriving reconstructed samples of the current block based on the prediction samples.

[0009] According to another embodiment of this disclosure, a decoding apparatus for performing image decoding is provided. The decoding apparatus includes: an entropy decoder for determining whether the width and height of a current block are less than or equal to a maximum transform block size; when the width and height are less than or equal to the maximum transform block size, obtaining a BDPCM flag indicating whether block-based incremental pulse code modulation (BDPCM) is applied to the current block; and obtaining a BDPCM direction flag indicating a prediction direction for the current block based on the BDPCM flag; a predictor for deriving prediction samples of the current block based on an intra-frame prediction mode derived from the BDPCM direction flag; and an adder for deriving reconstructed samples of the current block based on the prediction samples.

[0010] According to another embodiment of this disclosure, an image encoding method performed by an encoding device is provided. The method includes: determining whether the width and height of a current block are less than or equal to a maximum transform block size; when the width and height are less than or equal to the maximum transform block size, generating a BDPCM flag indicating whether block-based incremental pulse code modulation (BDPCM) is applied to the current block; generating a BDPCM direction flag indicating a prediction direction for the current block; and encoding image information including the BDPCM flag and the BDPCM direction flag.

[0011] According to another embodiment of this disclosure, an encoding apparatus for performing image encoding is provided. The encoding apparatus includes: a predictor for determining whether the width and height of a current block are less than or equal to a maximum transform block size; when the width and height are less than or equal to the maximum transform block size, generating a BDPCM flag indicating whether block-based incremental pulse code modulation (BDPCM) is applied to the current block, and generating a BDPCM direction flag indicating a prediction direction for the current block; and an entropy encoder for encoding image information including the BDPCM flag and the BDPCM direction flag.

[0012] Beneficial effects

[0013] According to this disclosure, by signaling the BDPCM flag based on the current block size and the maximum transform block size, the current block size and the maximum transform block size can be taken into account in the BDPCM flag signaling and in determining whether to apply BDPCM. In this way, the number of bits in BDPCM can be reduced and the overall compilation efficiency can be improved.

[0014] According to this disclosure, a syntax element indicating whether to constrain BDPCM for an image can be signaled, and in this way, a syntax element can be used to determine whether to perform BDPCM on an image, thereby improving the overall image compilation efficiency. Attached Figure Description

[0015] Figure 1 Examples of video / image compilation devices to which embodiments of the present disclosure can be applied are briefly illustrated.

[0016] Figure 2 This is a schematic diagram illustrating the configuration of a video / image encoding apparatus to which embodiments of the present disclosure can be applied.

[0017] Figure 3 This is a schematic diagram illustrating the configuration of a video / image decoding apparatus to which embodiments of the present disclosure can be applied.

[0018] Figure 4 Examples of video / image coding methods based on intra-frame prediction are shown.

[0019] Figure 5 Examples of video / image coding methods based on intra-frame prediction are shown.

[0020] Figure 6 The intra-frame prediction process is illustrated schematically.

[0021] Figure 7 An image encoding method according to the encoding apparatus of this document is illustrated schematically.

[0022] Figure 8 An encoding apparatus for performing an image encoding method according to this document is illustrated schematically.

[0023] Figure 9 An image decoding method according to the decoding apparatus of this document is illustrated schematically.

[0024] Figure 10 A decoding apparatus for performing an image decoding method according to this document is illustrated schematically.

[0025] Figure 11 A structural diagram of a content streaming system applying this disclosure is shown. Detailed Implementation

[0026] This disclosure can be modified in various forms, and specific embodiments thereof will be described and illustrated in the accompanying drawings. However, the embodiments are not intended to limit this disclosure. The terminology used in the following description is for the purpose of describing specific embodiments only and is not intended to limit this disclosure. Singular expressions include plural expressions, provided that they are clearly understood in different ways. Terms such as “comprising” and “having” are intended to indicate the presence of the features, numbers, steps, operations, elements, components, or combinations thereof used in the following description, and therefore should be understood to mean that the possibility of having or adding one or more different features, numbers, steps, operations, elements, components, or combinations thereof is not excluded.

[0027] Furthermore, the elements in the accompanying drawings described in this disclosure are drawn independently for the convenience of explaining different specific functions and do not imply that these elements are embodied by independent hardware or independent software. For example, two or more elements may be combined to form a single element, or a single element may be divided into multiple elements. The implementation of combining and / or dividing elements is part of this disclosure and does not depart from its concept.

[0028] In the following, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Furthermore, throughout the drawings, similar reference numerals are used to indicate similar elements, and identical descriptions of similar elements will be omitted.

[0029] Figure 1 Examples of video / image compilation devices to which embodiments of the present disclosure can be applied are briefly illustrated.

[0030] Reference Figure 1 A video / image encoding system may include a first device (source device) and a second device (receiving device). The source device may send encoded video / image information or data to the receiving device in the form of a file or stream via a digital storage medium or network.

[0031] The source device may include a video source, an encoding device, and a transmitter. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display, and the display may be configured as a separate device or an external component.

[0032] Video sources can acquire video / images through processes that capture, synthesize, or generate video / images. Video sources can include video / image capture devices and / or video / image generation devices. Video / image capture devices can include, for example, one or more cameras, video / image archives including previously captured video / images, etc. Video / image generation devices can include, for example, computers, tablets, and smartphones, and can generate video / images (electronically). For example, virtual video / images can be generated using computers, etc. In this case, the video / image capture process can be replaced by a process that generates related data.

[0033] An encoding device can encode input video / images. It can perform a series of processes such as prediction, transformation, and quantization to achieve compression and compilation efficiency. The encoded data (encoded video / image information) can be output as a bitstream.

[0034] A transmitter can send encoded images / image information or data, output as a bitstream, to a receiver in a receiving device via a digital storage medium or network, either as a file or a stream. Digital storage media can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter can include elements for generating media files according to a predetermined file format and may include elements for transmission over a broadcast / communication network. The receiver can receive / extract the bitstream and send the received bitstream to a decoding device.

[0035] Decoding devices can decode video / images by performing a series of processes, such as dequantization, inverse transform, and prediction, that correspond to the operations of encoding devices.

[0036] The renderer can render decoded video / images. Rendered video / images can then be displayed on a monitor.

[0037] This disclosure relates to video / image compilation. For example, the methods / implementations disclosed in this disclosure can be applied to methods disclosed in Multifunctional Video Compiler (VVC), EVC (Essential Video Compiler) standard, AOMedia Video 1 (AV1) standard, second-generation audio-visual compilation standard (AVS2), or next-generation video / image compilation standards (e.g., H.267, or H.268, etc.).

[0038] This disclosure presents various implementations of video / image compilation, and unless otherwise stated, the implementations can be combined with each other.

[0039] In this disclosure, video can refer to a series of images over time. Generally, an image refers to a unit representing an image in a specific time zone, and sub-images / slices / tiles are units that constitute part of an image in compilation. Sub-images / slices / tiles can include one or more compilation tree units (CTUs). An image can consist of one or more sub-images / slices / tiles. An image can consist of one or more tile groups. A tile group can include one or more tiles. A brick can represent a rectangular area of ​​CTU rows within a tile in an image. A tile can be partitioned into multiple bricks, each brick consisting of one or more CTU rows within the tile. A tile that is not partitioned into multiple bricks can also be referred to as a brick. Brick scanning can sort the CTUs of an image in a specific order, wherein CTUs are sequentially ordered by CTU raster scan within a brick, bricks within a tile are sequentially ordered by raster scan of the tiles of a tile, and tiles in an image are sequentially ordered by raster scan of the tiles of an image. Additionally, a sub-image can represent a rectangular region of one or more slices within an image. That is, a sub-image contains one or more slices that collectively cover a rectangular region of the image. A tile is a rectangular region of a CTU within a specific tile column and a specific tile row in an image. A tile column is a rectangular region of a CTU whose height is equal to the height of the image and whose width is specified by a syntax element in the image parameter set. A tile row is a rectangular region of a CTU whose height is specified by a syntax element in the image parameter set and whose width is equal to the width of the image. A tile scan is a specific ordering of CTUs that partition the image, wherein CTUs can be sequentially ordered by CTU raster scan within a tile, and tiles in the image can be sequentially ordered by raster scan of the image's tiles. A slice comprises an integer number of tiles of an image that can be exclusively contained within a single NAL unit. A slice can consist of multiple complete tiles or only a continuous sequence of complete tiles of a single tile. In this disclosure, tile groups and slices can be used interchangeably. For example, in this disclosure, a tile group / tile group header can be referred to as a slice / slice header.

[0040] A pixel, or cell (pel), can represent the smallest unit that makes up a picture (or image). Additionally, "sample" can be used as the term corresponding to a pixel. A sample can typically represent a pixel or a pixel value, and can represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chrominance component.

[0041] A unit can represent the basic unit of image processing. A unit may include a specific region of an image and at least one of the information associated with that region. A unit may include a luminance block and two chrominance (e.g., cb, cr) blocks. In some cases, the term "unit" may be used interchangeably with terms such as "block" or "region". In general, an M×N block may include a set (or array) of samples (or sample arrays) or transform coefficients in M ​​columns and N rows.

[0042] In this specification, “A or B” can mean “A only”, “B only”, or “A and B”. In other words, in this specification, “A or B” can be interpreted as “A and / or B”. For example, “A, B or C” in this document means “A only”, “B only”, “C only”, or “any one and any combination of A, B and C”.

[0043] The forward slash ( / ) or comma used in this specification can mean "and / or". For example, "A / B" can mean "A and / or B". Therefore, "A / B" can mean "A only", "B only", or "A and B". For example, "A,B,C" can mean "A, B, or C".

[0044] In this specification, "at least one of A and B" can mean "A only", "B only" or "both A and B". Furthermore, in this specification, the expression "at least one of A or B" or "at least one of A and / or B" can be interpreted as the same as "at least one of A and B".

[0045] Additionally, in this specification, "at least one of A, B, and C" means "A only", "B only", "C only" or "any combination of A, B, and C". Furthermore, "at least one of A, B, or C" or "at least one of A, B, and / or C" can mean "at least one of A, B, and C".

[0046] Furthermore, the parentheses used in this specification may refer to "for example". Specifically, when "prediction (intra-frame prediction)" is indicated, "intra-frame prediction" may be given as an example of "prediction". In other words, "prediction" in this specification is not limited to "intra-frame prediction", and "intra-frame prediction" may be given as an example of "prediction". Moreover, even when "prediction (i.e., intra-frame prediction)" is indicated, "intra-frame prediction" may be given as an example of "prediction".

[0047] In this specification, the technical features described individually in a single figure may be implemented individually or simultaneously.

[0048] The following figures are created to illustrate specific examples of this specification. Since the names of specific devices or signals / messages / fields described in the figures are presented by way of example, the technical features of this specification are not limited to the specific names used in the following figures.

[0049] Figure 2 This is a schematic diagram illustrating a configuration of a video / image encoding apparatus to which embodiments of the present disclosure may be applied. In the following, the video encoding apparatus may include an image encoding apparatus.

[0050] Reference Figure 2 The encoding apparatus 200 includes an image segmenter 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-frame predictor 221 and an intra-frame predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may also include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstruction block generator. According to embodiments, the image segmenter 210, predictor 220, residual processor 230, entropy encoder 240, adder 250, and filter 260 may be constituted by at least one hardware component (e.g., an encoder chipset or a processor). Additionally, the memory 270 may include a decoded image buffer (DPB) or may be constituted by a digital storage medium. The hardware component may also include the memory 270 as an internal / external component.

[0051] Image segmenter 210 can segment an input image (or picture or frame) input to encoding device 200 into one or more processors. For example, a processor may be referred to as a compilation unit (CU). In this case, the compilation unit can be recursively segmented from a compilation tree unit (CTU) or a maximum compilation unit (LCU) according to a quadtree-binary-trinary tree (QTBTTT) structure. For example, a compilation unit can be segmented into multiple deeper compilation units based on a quadtree structure, a binary tree structure, and / or a ternary structure. In this case, for example, a quadtree structure can be applied first, followed by a binary tree structure and / or a ternary structure. Alternatively, a binary tree structure can be applied first. The compilation process according to this disclosure can be performed based on the final compilation unit that is no longer segmented. In this case, the maximum compilation unit can be used as the final compilation unit based on compilation efficiency according to image characteristics, or, if necessary, the compilation unit can be recursively segmented into deeper compilation units, and the compilation unit with the optimal size can be used as the final compilation unit. Here, the compilation process may include prediction, transformation, and reconstruction processes, which will be described later. As another example, the processor may also include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit can be separated or divided from the final compilation unit described above. The prediction unit may be a unit for predicting samples, and the transform unit may be a unit for deriving transform coefficients and / or a unit for deriving residual signals from transform coefficients.

[0052] In some cases, a unit can be used interchangeably with terms such as block or region. Generally, an M×N block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can typically represent a pixel or pixel value, and can represent only the pixel / pixel value of the luminance component, or only the pixel / pixel value of the chrominance component. A sample can be used as a term corresponding to a picture (or image) of pixels or cells.

[0053] In the encoding apparatus 200, the predicted signal (prediction block, prediction sample array) output from the inter-frame predictor 221 or the intra-frame predictor 222 is subtracted from the input image signal (original block, original sample array) to generate a residual signal (residual block, residual sample array), and the generated residual signal is sent to the converter 232. In this case, as shown, the unit in the encoder 200 used to subtract the predicted signal (prediction block, prediction sample array) from the input image signal (original block, original sample array) can be called the subtractor 231. The predictor can perform prediction on the block to be processed (hereinafter referred to as the current block) and generate a prediction block including the prediction samples of the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction on a unit of the current block or CU. As described later in the description of each prediction mode, the predictor can generate various information related to the prediction, such as prediction mode information, and send the generated information to the entropy encoder 240. The information about the prediction can be encoded in the entropy encoder 240 and output as a bitstream.

[0054] Intra-predictor 222 can predict the current block by referencing samples in the current image. Depending on the prediction mode, the referenced samples may be located near or far from the current block. In intra-prediction, the prediction mode can include multiple non-directional modes and multiple directional modes. Non-directional modes can include, for example, DC mode and planar mode. Depending on the level of detail in the prediction direction, the directional modes can include, for example, 33 or 65 directional prediction modes. However, this is just an example, and more or fewer directional prediction modes may be used depending on the settings. Intra-predictor 222 can determine the prediction mode to be applied to the current block by using the prediction modes applied to neighboring blocks.

[0055] Inter-frame predictor 221 can deduce the predicted block of the current block based on a reference block (reference sample array) specified by motion vectors on a reference image. Here, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation between motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference image indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, neighboring blocks may include spatially adjacent blocks existing in the current image and temporally adjacent blocks existing in the reference image. The reference image including the reference block and the reference image including the temporally adjacent block may be the same or different. The temporally adjacent block may be called a juxtaposed reference block, a co-located CU (colCU), etc., and the reference image including the temporally adjacent block may be called a juxtaposed image (colPic). For example, inter-frame predictor 221 can configure a motion information candidate list based on neighboring blocks and generate information indicating which candidate to use to deduce the motion vector and / or reference image index of the current block. Inter-frame prediction can be performed based on various prediction modes. For example, in skip mode and merge mode, the inter-frame predictor 221 can use motion information from neighboring blocks as motion information for the current block. In skip mode, unlike merge mode, residual signals may not be transmitted. In motion vector prediction (MVP) mode, motion vectors from neighboring blocks can be used as motion vector predictors, and the motion vector of the current block can be indicated by signaling the motion vector difference.

[0056] Predictor 220 can generate prediction signals based on various prediction methods described below. For example, the predictor can not only apply intra-frame prediction or inter-frame prediction to predict a block, but can also apply both intra-frame prediction and inter-frame prediction simultaneously. This can be referred to as Inter-intra-frame Combined Prediction (CIIP). Alternatively, the predictor can predict blocks based on an Intra-Block Copy (IBC) prediction mode or a palette mode. IBC prediction modes or palette modes can be used for content image / video compilation, such as in games, for example, Screen Content Encoding (SCC). IBC essentially performs prediction within the current image, but can be performed similarly to inter-frame prediction because the reference block is derived from the current image. That is, IBC can use at least one of the inter-frame prediction techniques described in this disclosure. A palette mode can be considered an example of intra-frame compilation or intra-frame prediction. When a palette mode is applied, sample values ​​within the image can be signaled based on information about the palette table and palette index.

[0057] The predicted signal generated by the predictor (including inter-frame predictor 221 and / or intra-frame predictor 222) can be used to generate a reconstructed signal or a residual signal. Transformer 232 can generate transform coefficients by applying transform techniques to the residual signal. For example, the transform technique can include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loève Transform (KLT), Graph-Based Transform (GBT), or Conditional Nonlinear Transform (CNT). Here, GBT refers to a transform obtained from a graph when the relationship information between pixels is represented by a graph. CNT refers to a transform generated based on the predicted signal generated using all previously reconstructed pixels. Furthermore, the transform processing can be applied to square pixel blocks of the same size, or it can be applied to blocks of variable size that are not square.

[0058] Quantizer 233 quantizes the transform coefficients and sends them to entropy encoder 240, which encodes the quantized signal (information about the quantized transform coefficients) and outputs a bitstream. This information about the quantized transform coefficients can be called residual information. Quantizer 233 can rearrange the block-type quantized transform coefficients into a one-dimensional vector form based on the coefficient scan order and generate information about the quantized transform coefficients based on this one-dimensional vector form. Entropy encoder 240 can perform various encoding methods, such as Golomb, Context Adaptive Variable-Length Compilation (CAVLC), and Context Adaptive Binary Arithmetic Compilation (CABAC). Entropy encoder 240 can encode information required for video / image reconstruction other than the quantized transform coefficients (e.g., values ​​of syntax elements) together or separately. Encoded information (e.g., encoded video / image information) can be sent or stored in units of NAL (Network Abstraction Layer) in the form of a bitstream. The video / image information may also include information about various parameter sets such as Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). Additionally, the video / image information may also include general constraint information. In this disclosure, information and / or syntax elements that are sent / signaled from the encoding device to the decoding device may be included in the video / image information. The video / image information may be encoded by the encoding process described above and included in a bitstream. The bitstream may be transmitted over a network or stored in a digital storage medium. The network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitter (not shown) that transmits a signal output from the entropy encoder 240 and / or a storage unit (not shown) that stores the signal may be included as internal / external elements of the encoding device 200; alternatively, the transmitter may be included in the entropy encoder 240.

[0059] The quantized transform coefficients output from quantizer 233 can be used to generate a prediction signal. For example, the residual signal (residual block or residual sample) can be reconstructed by applying dequantization and inverse transform to the quantized transform coefficients using dequantizer 234 and inverse transformer 235. Adder 250 adds the reconstructed residual signal to the prediction signal output from inter-frame predictor 221 or intra-frame predictor 222 to generate a reconstructed signal (reconstructed image, reconstructed block, reconstructed sample array). If the block to be processed has no residual (such as when a skip mode is applied), the prediction block can be used as a reconstructed block. Adder 250 can be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current image, and can be used for inter-frame prediction of the next image by filtering as described below.

[0060] In addition, Luminance Mapping and Chroma Scaling (LMCS) can be applied during image encoding and / or reconstruction.

[0061] Filter 260 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 260 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image and store the modified reconstructed image in memory 270 (specifically, the DPB of memory 270). Various filtering methods may include, for example, deblocking filtering, sample adaptive offsetting, adaptive loop filtering, bilateral filtering, etc. Filter 260 can generate various filtering-related information and send the generated information to entropy encoder 240, as described later in the description of the various filtering methods. The filtering-related information can be encoded by entropy encoder 240 and output as a bitstream.

[0062] The modified reconstructed image sent to memory 270 can be used as a reference image in inter-frame predictor 221. When inter-frame prediction is applied through the encoding device, prediction mismatch between the encoding device 200 and the decoding device can be avoided, and compilation efficiency can be improved.

[0063] The DPB of memory 270 can store a modified reconstructed image used as a reference image in inter-frame predictor 221. Memory 270 can store motion information of blocks from which motion information in the current image is derived (or encoded) and / or motion information of reconstructed blocks in the image. The stored motion information can be sent to inter-frame predictor 221 and used as motion information for spatially adjacent blocks or temporally adjacent blocks. Memory 270 can store reconstructed samples of reconstructed blocks in the current image and can transmit these reconstructed samples to intra-frame predictor 222.

[0064] Figure 3 This is a schematic diagram illustrating the configuration of a video / image decoding apparatus to which embodiments of the present disclosure can be applied.

[0065] Reference Figure 3 The decoding device 300 may include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-frame predictor 332 and an intra-frame predictor 331. The residual processor 320 may include a dequantizer 321 and an inverse transformer 322. According to embodiments, the entropy decoder 310, residual processor 320, predictor 330, adder 340, and filter 350 may be constructed from hardware components (e.g., a decoder chipset or processor). Additionally, the memory 360 may include a decoded image buffer (DPB) or may be constructed from a digital storage medium. The hardware components may also include the memory 360 as an internal / external component.

[0066] When the input includes a bitstream containing video / image information, the decoding device 300 can interact with... Figure 2 The processing of video / image information in the encoding apparatus correspondingly reconstructs the image. For example, the decoding apparatus 300 can derive units / blocks based on block segmentation information obtained from the bitstream. The decoding apparatus 300 can use a processor applied in the encoding apparatus to perform decoding. Therefore, the decoding processor can be, for example, a compilation unit, and the compilation unit can be segmented from a compilation tree unit or a maximum compilation unit according to a quadtree structure, a binary tree structure, and / or a ternary tree structure. One or more transformation units can be derived from the compilation unit. The reconstructed image signal decoded and output by the decoding apparatus 300 can be reproduced by a reproduction apparatus.

[0067] Decoding device 300 can receive data in bitstream form from... Figure 2The signal output by the encoding device can be decoded by the entropy decoder 310. For example, the entropy decoder 310 can parse the bitstream to derive information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets such as adaptive parameter sets (APS), picture parameter sets (PPS), sequence parameter sets (SPS), or video parameter sets (VPS). In addition, the video / image information may also include general constraint information. The decoding device can also decode the picture based on the information about the parameter sets and / or general constraint information. The signaling / receiving information and / or syntax elements described later in this disclosure can be decoded and obtained from the bitstream through the decoding process. For example, the entropy decoder 310 decodes the information in the bitstream based on a compilation method such as exponential Columbus compilation, CAVLC, or CABAC, and outputs the quantized values ​​of the syntax elements and transform coefficients of the residuals required for image reconstruction. More specifically, the CABAC entropy decoding method can receive a bin corresponding to each syntax element in the bitstream, determine the context model using information about the target syntax element, decoding information about the target block, or information about symbols / bins decoded in previous stages, and perform arithmetic decoding on the bin by predicting the occurrence probability of the bin based on the determined context model, generating symbols corresponding to the value of each syntax element. In this case, after determining the context model, the CABAC entropy decoding method can update the context model by using the information of the decoded symbols / bins for the context model of the next symbol / bin. The prediction-related information in the information decoded by the entropy decoder 310 can be provided to the predictors (inter-frame predictor 332 and intra-frame predictor 331), and the residual values ​​(i.e., quantization transform coefficients and related parameter information) from which entropy decoding has been performed in the entropy decoder 310 can be input to the residual processor 320. The residual processor 320 can derive the residual signals (residual blocks, residual samples, residual sample arrays). Additionally, the filtering information in the information decoded by the entropy decoder 310 can be provided to the filter 350. Furthermore, the receiver (not shown) for receiving the signal output from the encoding device can be further configured as an internal / external element of the decoding device 300, or the receiver can be a component of the entropy decoder 310. Additionally, the decoding device according to this disclosure can be referred to as a video / image / picture decoding device, and the decoding device can be classified as an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoder 310, and the sample decoder may include at least one of a dequantizer 321, an inverse transformer 322, an adder 340, a filter 350, a memory 360, an inter-frame predictor 332, and an intra-frame predictor 331.

[0068] The dequantizer 321 can dequantize the quantized transform coefficients and output the transform coefficients. The dequantizer 321 can rearrange the quantized transform coefficients in the form of two-dimensional blocks. In this case, the rearrangement can be performed based on the coefficient scan order executed in the encoding device. The dequantizer 321 can perform dequantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information) and obtain the transform coefficients.

[0069] The inverse transformer 322 performs an inverse transformation on the transformation coefficients to obtain the residual signal (residual block, residual sample array).

[0070] The predictor can perform prediction on the current block and generate a prediction block that includes the prediction samples of the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction to the current block based on the prediction information output from the entropy decoder 310, and can determine the specific intra-frame / inter-frame prediction mode.

[0071] Predictor 320 can generate prediction signals based on various prediction methods described below. For example, the predictor can not only apply intra-frame prediction or inter-frame prediction to predict a block, but can also apply intra-frame prediction and inter-frame prediction simultaneously. This can be referred to as combined intra-frame and inter-frame prediction (CIIP). Alternatively, the predictor can predict blocks based on an intra-frame block copy (IBC) prediction mode or a palette mode. The IBC prediction mode or palette mode can be used for content image / video compilation, such as screen content compilation (SCC), for games, etc. IBC essentially performs prediction in the current image, but can be performed similarly to inter-frame prediction because a reference block is derived in the current image. That is, IBC can use at least one of the inter-frame prediction techniques described in this disclosure. The palette mode can be considered as an example of intra-frame compilation or intra-frame prediction. When a palette mode is applied, sample values ​​within the image can be signaled based on information about the palette table and palette index.

[0072] Intra-predictor 331 can predict the current block by referencing samples in the current image. Depending on the prediction mode, the referenced samples may be located near or far from the current block. In intra-prediction, the prediction mode can include multiple non-directional modes and multiple directional modes. Intra-predictor 331 can determine the prediction mode applied to the current block by using prediction modes applied to neighboring blocks.

[0073] Inter-frame predictor 332 can deduce the predicted block of the current block based on reference blocks (reference sample arrays) specified by motion vectors on a reference image. In this case, to reduce the amount of motion information transmitted in the inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation of motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference image indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, neighboring blocks may include spatially adjacent blocks existing in the current image and temporally adjacent blocks existing in the reference image. For example, inter-frame predictor 332 can configure a motion information candidate list based on neighboring blocks and deduce the motion vector and / or reference image index of the current block based on the received candidate selection information. Inter-frame prediction can be performed based on various prediction modes, and the information about the prediction may include information indicating the mode of inter-frame prediction for the current block.

[0074] Adder 340 can generate a reconstruction signal (reconstructed image, reconstruction block, reconstruction sample array) by adding the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the predictor (including inter-frame predictor 332 and / or intra-frame predictor 331). If the block to be processed has no residual (e.g., when a skip mode is applied), the prediction block can be used as the reconstruction block.

[0075] Adder 340 can be called a reconstructor or reconstruction block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current image, through filtering output as described below, or it can be used for inter-frame prediction of the next image.

[0076] In addition, Luminance Mapping and Chromaticity Scaling (LMCS) can be applied during image decoding.

[0077] Filter 350 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 350 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image and store the modified reconstructed image in memory 360 (specifically, the DPB of memory 360). Various filtering methods may include, for example, deblocking filtering, adaptive sample shifting, adaptive loop filtering, bilateral filtering, etc.

[0078] The (modified) reconstructed image stored in the DPB of memory 360 can be used as a reference image in inter-frame predictor 332. Memory 360 can store motion information of blocks from which motion information in the current image is derived (or decoded) and / or motion information of reconstructed blocks in the image. The stored motion information can be sent to inter-frame predictor 260 to be used as motion information of spatially adjacent blocks or temporally adjacent blocks. Memory 360 can store reconstructed samples of reconstructed blocks in the current image and can transmit the reconstructed samples to intra-frame predictor 331.

[0079] In this disclosure, the embodiments described in the filter 260, inter-frame predictor 221, and intra-frame predictor 222 of the encoding apparatus 200 can be the same as, or applied to, the filter 350, inter-frame predictor 332, and intra-frame predictor 331 of the decoding apparatus 300, respectively. The same content can also be applied to the inter-frame predictor 332 and intra-frame predictor 331.

[0080] In this disclosure, at least one of quantization / inverse quantization and / or transformation / inverse transformation may be omitted. When quantization / inverse quantization is omitted, the transformation coefficients of the quantization may be referred to as transformation coefficients. When transformation / inverse transformation is omitted, the transformation coefficients may be referred to as coefficients or residual coefficients, or for the sake of consistency, they may still be referred to as transformation coefficients.

[0081] In this disclosure, quantization transform coefficients and transform coefficients can be referred to as transform coefficients and scaling transform coefficients, respectively. In this case, residual information can include information about the transform coefficients, and this information can be signaled via residual compilation syntax. Transform coefficients can be derived based on the residual information (or information about the transform coefficients), and scaling transform coefficients can be derived by inverse transforming (scaling) the transform coefficients. Residual samples can be derived based on the inverse transform (scaling) of the scaling transform coefficients. This can also be applied / expressed in other parts of this disclosure.

[0082] Simultaneously, as described above, prediction is performed during video compilation to improve compression efficiency. In this way, a prediction block, including prediction samples of the current block, can be generated as the block to be compiled (i.e., the compilation target block). Here, the prediction block includes prediction samples in the spatial domain (or pixel domain). The prediction block is derived in the same manner in both the encoding and decoding devices, and the encoding device can signal the decoding device with information about the residual between the original block and the prediction block (residual information), rather than the original sample values ​​of the original block, thereby improving image compilation efficiency. The decoding device can derive a residual block including residual samples based on the residual information, add the residual block and the prediction block to generate a reconstruction block including reconstructed samples, and generate a reconstructed image including the reconstruction block.

[0083] Residual information can be generated through transformation and quantization processes. For example, the encoding device can derive a residual block between the original block and the prediction block, perform a transformation process on the residual samples (residual sample array) included in the residual block to derive transform coefficients, perform a quantization process on the transform coefficients to derive quantized transform coefficients, and signal the relevant residual information (via bitstream) to the decoding device. Here, the residual information may include the values ​​of the quantized transform coefficients, their positions, the transform technique, the transform core, and the values ​​of quantization parameters. The decoding device can perform dequantization / inverse transform processes based on the residual information and derive residual samples (or residual blocks). The decoding device can generate a reconstructed image based on the prediction block and the residual block. Furthermore, as a reference for inter-frame prediction of subsequent reference images, the encoding device can dequantize / inverse transform the quantized transform coefficients to derive residual blocks and generate a reconstructed image based on them.

[0084] Intra-frame prediction can refer to generating prediction samples for the current block based on reference samples in the image to which the current block belongs (hereinafter referred to as the current image). When intra-frame prediction is applied to the current block, neighboring reference samples to be used for intra-frame prediction of the current block can be derived. The neighboring reference samples of the current block can include samples adjacent to the left boundary of the current block of size nW x nH and a total of 2 x nH samples adjacent to the lower left of the current block, samples adjacent to the top boundary of the current block and a total of 2 x nW samples adjacent to the upper right of the current block, and a sample adjacent to the upper left of the current block. Alternatively, the neighboring reference samples of the current block can include multiple columns of upper adjacent samples and multiple rows of left adjacent samples. In addition, the neighboring reference samples of the current block can include a total of nH samples adjacent to the right boundary of the current block of size nW x nH, a total of nW samples adjacent to the lower boundary of the current block, and a sample adjacent to the lower right of the current block.

[0085] However, some neighboring reference samples of the current block may not have been decoded or may be unavailable. In this case, the decoder can construct neighboring reference samples to be used for prediction by replacing unavailable samples with available samples. Alternatively, the neighboring reference samples to be used for prediction can be configured by interpolation of available samples.

[0086] When deriving neighboring reference samples, (i) the predicted sample can be derived based on the average or interpolation of the neighboring reference samples of the current block, or (ii) the predicted sample can be derived based on a reference sample existing in a specific (predictive) direction relative to the predicted sample in the neighboring reference samples of the current block. Case (i) can be referred to as non-directional mode or non-angular mode, and case (ii) can be referred to as directional mode or angular mode.

[0087] Alternatively, prediction samples can be generated by interpolating the first neighboring sample in the prediction direction of the intra-prediction mode of the current block and the second neighboring sample in the opposite direction from the prediction sample in the adjacent reference samples. This can be called linear interpolation intra-prediction (LIP). Furthermore, a linear model (LM) can be used to generate chroma prediction samples based on luminance samples. This can be called LM mode or chroma component LM (CCLM) mode.

[0088] Alternatively, a provisional prediction sample for the current block can be derived based on filtered neighboring reference samples, or it can be derived by weighted summing of the provisional prediction sample with at least one reference sample derived from the existing neighboring reference samples (i.e., unfiltered neighboring reference samples) according to the intra-prediction mode. This process can be referred to as position-dependent intra-prediction (PDPC).

[0089] Alternatively, the reference sample line with the highest prediction accuracy among the adjacent reference sample lines of the current block is selected, and the prediction sample is derived using the reference sample in the prediction direction of the selected line. In this case, intra-frame prediction coding can be performed by instructing (signaling) the reference sample line used to the decoding device. The above situation can be referred to as multi-reference line intra-frame prediction or MRL-based intra-frame prediction.

[0090] Furthermore, the current block can be divided into vertical or horizontal sub-partitions and intra-prediction can be performed based on the same intra-prediction mode, but neighboring reference samples can be derived and used on a sub-partition basis. That is, in this case, the intra-prediction mode of the current block also applies to sub-partitions, but in some cases, intra-prediction performance can be improved by deriving and using neighboring reference samples on a sub-partition basis. This prediction method can be called intra-prediction based on intra-partition sub-partitions (ISP).

[0091] The intra-prediction methods described above can be referred to as intra-prediction types to distinguish them from intra-prediction modes. Intra-prediction types can be referred to using various terms, such as intra-prediction techniques or additional intra-prediction modes. For example, an intra-prediction type (or additional intra-prediction mode, etc.) can include at least one of LIP, PDPC, MRL, and ISP. General intra-prediction methods that exclude specific intra-prediction types such as LIP, PDPC, MRL, and ISP can be referred to as normal intra-prediction types. When not applying specific intra-prediction types, normal intra-prediction types can generally be applied, and prediction can be performed based on the intra-prediction modes described above. Additionally, post-processing filtering can be performed on the derived prediction samples if necessary.

[0092] Specifically, the intra-frame prediction process may include an intra-frame prediction mode / type determination step, a neighboring reference sample derivation step, and a prediction sample derivation step based on the intra-frame prediction mode / type. Furthermore, if necessary, a post-filtering step may be performed on the derived prediction samples.

[0093] Figure 4 An example of a video / image coding method based on intra-frame prediction is illustrated.

[0094] Reference Figure 4 The encoding device performs intra-prediction on the current block (S400). The encoding device derives the intra-prediction mode / type for the current block, derives neighboring reference samples for the current block, and generates prediction samples for the current block based on the intra-prediction mode / type and neighboring reference samples. Here, the intra-prediction mode / type determination, neighboring reference sample derivation, and prediction sample generation processes can be performed simultaneously, or one process can be performed before the other. The encoding device can determine the mode / type applicable to the current block from multiple intra-prediction modes / types. The encoding device can compare the RD costs of the intra-prediction modes / types and determine the optimal intra-prediction mode / type for the current block.

[0095] Simultaneously, the encoding device can perform a prediction sample filtering process. Prediction sample filtering can be referred to as post-filtering. Some or all prediction samples can be filtered by the prediction sample filtering process. In some cases, the prediction sample filtering process can be omitted.

[0096] The encoding device generates residual samples for the current block based on the (filtered) prediction samples (S410). The encoding device can derive residual samples by comparing the prediction samples in the original samples of the current block with the phase.

[0097] The encoding device can encode image information including information about intra-frame prediction (prediction information) and residual information about residual samples (S420). The prediction information may include intra-frame prediction mode information and intra-frame prediction type information. The encoding device can output the encoded image information in the form of a bitstream. The output bitstream can be sent to the decoding device via a storage medium or a network.

[0098] The residual information may include the residual compilation syntax described later. The encoding device can transform / quantize the residual samples to derive quantized transform coefficients. The residual information may include information about the quantized transform coefficients.

[0099] Simultaneously, as described above, the encoding device can generate a reconstructed image (including reconstructed samples and reconstructed blocks). To this end, the encoding device can derive (modified) residual samples by performing inverse quantization / inverse transform again on the quantized transform coefficients. The reason for performing inverse quantization / inverse transform again after transforming / quantizing the residual samples in this way is to derive residual samples identical to those derived in the decoding device described above. The encoding device can generate a reconstructed block, including reconstructed samples for the current block, based on the predicted samples and the (modified) residual samples. A reconstructed image for the current image can be generated based on the reconstructed block. As described above, the in-loop filtering process can be further applied to the reconstructed image.

[0100] Figure 5 An example of a video / image coding method based on intra-frame prediction is illustrated.

[0101] Decoding devices can perform operations corresponding to those performed by encoding devices.

[0102] Prediction and residual information can be obtained from the bitstream. Residual samples for the current block can be derived based on the residual information. Specifically, transform coefficients can be derived by performing inverse quantization based on the quantization transform coefficients derived from the residual information, and residual samples for the current block can be derived by performing an inverse transform on the transform coefficients.

[0103] Specifically, the decoding device can derive the intra-prediction mode / type of the current block based on the received prediction information (intra-prediction mode / type information) (S500). The decoding device can derive the neighboring reference samples of the current block (S510). Based on the intra-prediction mode / type and the neighboring reference samples, the decoding device generates prediction samples in the current block (S520). In this case, the decoding device can perform a prediction sample filtering process. Prediction sample filtering can be referred to as post-filtering. Some or all prediction samples can be filtered by the prediction sample filtering process. In some cases, the prediction sample filtering process can be omitted.

[0104] The decoding device generates residual samples for the current block based on the received residual information (S530). The decoding device can generate reconstructed samples for the current block based on the predicted samples and residual samples, and can derive a reconstructed block including the reconstructed samples (S540). A reconstructed image of the current image can be generated based on the reconstructed block. As described above, the in-loop filtering process can be further applied to the reconstructed image.

[0105] Intra-luma_mpm_flag may include, for example, flag information indicating whether the most probable mode (MPM) is applied to the current block or whether the remaining modes are applied. When the MPM is applied to the current block, the prediction mode information may further include index information (e.g., intra_luma_mpm_idx) indicating one of the intra-luma_mpm_candidates. The intra-luma_mpm_candidates may consist of a list of MPM candidates or a list of MPMs. Additionally, when the MPM is not applied to the current block, the intra-luma_mpm_remainder includes remaining mode information (e.g., intra_luma_mpm_remainder) indicating one of the remaining intra-luma_mpm_candidates besides the intra-luma_mpm_candidates. The decoding device can determine the intra-luma_mpm_remainder for the current block based on the intra-luma_mpm_remainder information.

[0106] Furthermore, intra-prediction type information can be implemented in various forms. For example, intra-prediction type information may include intra-prediction type index information indicating one of the intra-prediction types. As another example, intra-prediction type information may include at least one of the following: reference sample line information indicating whether MRL is applied to the current block, and if so, which reference sample line is used (e.g., intra_luma_ref_idx); ISP flag information indicating whether ISP is applied to the current block (e.g., intra_subpartitions_mode_flag); or ISP type information indicating the split type of the subpartitions when ISP is applied (e.g., intra_subpartitions_split_flag). Additionally, intra-prediction type information may include a MIP flag indicating whether matrix-based intra-prediction (MIP) is applied to the current block.

[0107] Intra-prediction mode information and / or intra-prediction type information can be encoded / decoded using the compilation methods described in this disclosure. For example, intra-prediction mode information and / or intra-prediction type information can be encoded / decoded using entropy compilation (e.g., CABAC, CAVLC).

[0108] Figure 6 The intra-frame prediction process is illustrated schematically.

[0109] Reference Figure 6 As described above, the intra-frame prediction process may include steps such as determining the intra-frame prediction mode / type, deriving neighboring reference samples, and performing intra-frame prediction (generating prediction samples). The intra-frame prediction process may be performed by the encoding and decoding devices described above. In this disclosure, the encoding device may include the encoding device and / or the decoding device.

[0110] Reference Figure 6 The compiler determines the intra-frame prediction mode / type S600.

[0111] The encoding device can determine the intra prediction mode / type applicable to the current block from the various intra prediction modes / types described above, and can generate prediction-related information. The prediction-related information may include intra prediction mode information representing the intra prediction mode applied to the current block and / or intra prediction type information representing the intra prediction type applied to the current block. The decoding device can determine the intra prediction mode / type applicable to the current block based on the prediction-related information.

[0112] Intra-luma_mpm_flag may include, for example, flag information indicating whether the most probable mode (MPM) will be applied to the current block or a remaining mode will be applied. When an MPM is applied to the current block, the prediction mode information may further include index information (e.g., intra_luma_mpm_idx) indicating one of the intra-luma_mpm_candidates. The intra-luma_mpm_candidates may consist of a list of MPM candidates or a list of MPMs. Additionally, when no MPM is applied to the current block, the intra-luma_mpm_remainder may further include remaining mode information (e.g., intra_luma_mpm_remainder) indicating one of the remaining intra-luma_prediction modes besides the intra-luma_mpm_candidates. The decoding device can determine the intra-luma_prediction mode for the current block based on the intra-luma_prediction mode information.

[0113] Furthermore, intra-prediction type information can be implemented in various forms. For example, intra-prediction type information may include intra-prediction type index information indicating one of the intra-prediction types. As another example, intra-prediction type information may include at least one of the following: reference sample line information indicating whether MRL is applied to the current block, and if so, which reference sample line is used (e.g., intra_luma_ref_idx); ISP flag information indicating whether ISP is applied to the current block (e.g., intra_subpartitions_mode_flag); or ISP type information indicating the split type of the subpartitions when ISP is applied (e.g., intra_subpartitions_split_flag). Additionally, intra-prediction type information may include a MIP flag indicating whether matrix-based intra-prediction (MIP) is applied to the current block.

[0114] For example, when applying intra-prediction, the intra-prediction modes of neighboring blocks can be used to determine the intra-prediction mode to be applied to the current block. For instance, the compiler can select one of the most probable mode (MPM) candidates derived from the MPM list based on additional candidate modes and / or the intra-prediction modes of the current block's neighboring blocks (e.g., left and / or top neighboring blocks), or one of the remaining intra-prediction modes not included in the MPM candidates (and planar modes) based on MPM residual information (remaining intra-prediction mode information). The MPM list can be configured to include or exclude planar modes as candidates. For example, the MPM list can have 6 candidates when it includes planar modes as candidates, and 5 candidates when it does not. When the MPM list does not include planar modes as candidates, a non-planar flag (e.g., intra_luma_not_planar_flag) indicating whether the intra-prediction mode of the current block is not a planar mode can be signaled. For example, the MPM flag can be signaled first, and when the MPM flag is 1, the MPM index and non-planar flag can be signaled. Furthermore, when the non-planar flag is 1, the MPM index can be signaled. Here, the fact that the MPM list does not include planar patterns as candidates is that planar patterns are always considered MPMs, not that they are not MPMs; therefore, the flag (non-planar flag) is signaled first to check if it is a planar pattern.

[0115] For example, MPM flags (e.g., `intra_luma_mpm_flag`) can indicate whether the intra-prediction mode applied to the current block is among the MPM candidates (and planar modes) or among the remaining modes. An MPM flag of 1 indicates that the intra-prediction mode of the current block is among the MPM candidates (and planar modes), while an MPM flag of 0 indicates that the intra-prediction mode of the current block is not among the MPM candidates (and planar modes). A non-planar flag of 0 (e.g., `intra_luma_not_planar_flag`) indicates that the intra-prediction mode of the current block is planar, while a non-planar flag of 1 indicates that the intra-prediction mode of the current block is not planar. The MPM index can be signaled in the form of `mpm_idx` or `intra_luma_mpm_idx` syntax elements, and the remaining intra-prediction mode information can be signaled in the form of `rem_intra_luma_pred_mode` or `intra_luma_mpm_remainder` syntax elements. For example, the remaining intra-prediction mode information can be indexed sequentially by prediction mode number to indicate one of the remaining intra-prediction modes not included in the MPM candidates (and planar modes) among all intra-prediction modes. The intra-prediction mode can be an intra-prediction mode for the luma component (sample). In the following, the intra-prediction mode information may include at least one of the following: an MPM flag (e.g., intra_luma_mpm_flag), a non-planar flag (e.g., intra_luma_not_planar_flag), an MPM index (e.g., mpm_idx or intra_luma_mpm_idx), or remaining intra-prediction mode information (rem_intra_luma_luma_mpm_mode or intra_luma_mpminder). In this disclosure, the MPM list can be referred to using various terms such as the MPM candidate list and candModeList.

[0116] When applying MIP to the current block, the individual MPM flags used for MIP (e.g., intra_mip_mpm_flag), MPM index (e.g., intra_mip_mpm_idx), and remaining intra-prediction mode information (e.g., intra_mip_mpm_remainder) can be signaled, and the non-plane flag can be not signaled.

[0117] In other words, generally speaking, when performing block segmentation of an image, the current block to be compiled and its neighboring blocks have similar image features. Therefore, there is a high probability that the current block and its neighboring blocks have the same or similar intra-prediction modes. Thus, the encoder can use the intra-prediction modes of neighboring blocks to encode the intra-prediction mode of the current block.

[0118] The compiler can construct a most probable pattern (MPM) list for the current block. This MPM list can be referred to as the MPM candidate list. Here, MPM can refer to a pattern used to improve compilation efficiency during intra-frame prediction pattern compilation by considering the similarity between the current block and neighboring blocks. As mentioned above, the MPM list can be constructed to include planar patterns or to exclude planar patterns. For example, when the MPM list includes planar patterns, the number of candidates in the MPM list can be 6. When the MPM list does not include planar patterns, the number of candidates in the MPM list can be 5.

[0119] The encoding device can perform prediction based on various intra-prediction modes and can determine the optimal intra-prediction mode based on its rate-distortion optimization (RDO). In this case, the encoding device can determine the optimal intra-prediction mode by using only the MPM candidates and plane modes configured in the MPM list, or by further using the remaining intra-prediction modes as well as the MPM candidates and plane modes configured in the MPM list. Specifically, for example, if the intra-prediction type of the current block is a specific type other than the normal intra-prediction type (e.g., LIP, MRL, or ISP), the encoding device can determine the optimal intra-prediction mode by only considering the MPM candidates and plane modes as intra-prediction mode candidates for the current block. That is, in this case, the intra-prediction mode of the current block can be determined only from the MPM candidates and plane modes, and in this case, the encoding / signaling of the MPM flag can be omitted. In this case, the decoding device can infer that the MPM flag is 1 without separately signaling the MPM flag.

[0120] Simultaneously, typically, when the intra-prediction mode of the current block is not a planar mode but one of the MPM candidates in the MPM list, the encoding device generates an MPM index (mpm idx) indicating one of the MPM candidates. When the intra-prediction mode of the current block is not included in the MPM list, the encoding device generates MPM remainder information (remaining intra-prediction mode information) indicating the same mode as the intra-prediction mode of the current block among the remaining intra-prediction modes not included in the MPM list (and planar modes). MPM remainder information may include, for example, the intra_luma_mpm_remainder syntax element.

[0121] The decoding device obtains intra-prediction mode information from the bitstream. As described above, the intra-prediction mode information may include at least one of an MPM flag, a non-planar flag, an MPM index, and remaining MPM information (remaining intra-prediction mode information). The decoding device may construct an MPM list. The construction of the MPM list is the same as that of the MPM list constructed in the encoding device. That is, the MPM list may include intra-prediction modes of adjacent blocks, or may further include specific intra-prediction modes according to a predetermined method.

[0122] The decoding device can determine the intra-prediction mode of the current block based on the MPM list and intra-prediction mode information. For example, when the MPM flag is 1, the decoding device can (based on the non-planar flag) derive the planar mode as the intra-prediction mode of the current block, or derive the candidate indicated by the MPM index from the MPM candidates in the MPM list as the intra-prediction mode of the current block. Here, the MPM candidate may only represent the candidates included in the MPM list, or it may include not only the candidates included in the MPM list, but also the planar mode applicable when the MPM flag is 1.

[0123] For example, when the MPM flag value is 0, the decoding device can derive the intra-prediction mode indicated by the remaining intra-prediction mode information (which may be referred to as MPM remaining information) from the remaining intra-prediction modes not included in the MPM list and planar modes as the intra-prediction mode of the current block. Meanwhile, as another example, when the intra-prediction type of the current block is a specific type (e.g., LIP, MRL, or ISP), the decoding device can derive the candidate indicated by the MPM flag in the planar mode or MPM list as the intra-prediction mode of the current block without parsing / decoding / checking the MPM flag.

[0124] The compiler exports neighboring reference samples for the current block (S610). When intra-frame prediction is applied to the current block, neighboring reference samples to be used for intra-frame prediction of the current block can be exported. The neighboring reference samples for the current block may include samples adjacent to the left boundary of the current block of size nW x nH, a total of 2 x nH samples adjacent to the lower left of the current block, samples adjacent to the top boundary of the current block, a total of 2 x nW samples adjacent to the upper right of the current block, and samples adjacent to the upper left of the current block. Alternatively, the neighboring reference samples for the current block may include multiple columns of upper adjacent samples and multiple rows of left adjacent samples. In addition, the neighboring reference samples for the current block may include a total of nH samples adjacent to the right boundary of the current block of size nW x nH, a total of nW samples adjacent to the lower boundary of the current block, and samples adjacent to the lower right of the current block.

[0125] On the other hand, when MRL is applied (i.e., when the MRL index value is greater than 0), adjacent reference samples may be located on lines 1 to 2 instead of line 0, which is adjacent to the current block on the left / top side. In this case, the number of adjacent reference samples can be further increased. Meanwhile, when ISP is applied, adjacent reference samples can be derived on a sub-partition basis.

[0126] The compilation device derives prediction samples by performing intra-frame prediction on the current block (S620). The compilation device can derive prediction samples based on the intra-frame prediction mode / type and neighboring samples. The compilation device can derive reference samples based on the intra-frame prediction mode of the current block in the neighboring reference samples of the current block, and can derive prediction samples of the current block based on the reference samples.

[0127] Additionally, according to the embodiments, Block Differential Pulse Code Modulation (BDPCM) technology can be used. BDPCM can also be referred to as RDPCM (Incremental Pulse Code Modulation Based on Quantization Residual Blocks).

[0128] When predicting blocks by applying BDPCM, rows or columns of the block can be predicted line by line using reconstructed samples. In this case, the reference samples used can be unfiltered samples. The BDPCM direction can indicate whether to use the vertical or horizontal direction for prediction. That is, when applying BDPCM, the vertical or horizontal direction can be selected as the BDPCM direction, and prediction can be performed in the BDPCM direction. The prediction error can be quantized in the spatial domain, and the sample can be reconstructed by adding the inverse-quantized prediction error to the prediction (i.e., the prediction sample). The prediction error may refer to the residual. As an alternative to this BDPCM, a quantized residual domain BDPCM can be proposed, and the prediction direction or signaling can be the same as that applied to the spatial domain BDPCM. That is, the quantization coefficients themselves can be accumulated by quantized residual domain BDPCM like DPCM (Delta Pulse Code Modulation), and then the residuals can be reconstructed by inverse quantization. Therefore, quantized residual domain BDPCM can be used in the sense of applying DPCM in a residual compilation state. The quantization residual domain used below refers to the domain used to quantize the residuals derived from the prediction without transformation. For example, the quantization residual domain could include quantized residuals (or quantized residual coefficients) to which the transformation was skipped; that is, the transformation was skipped but quantization was applied to the residual samples. Alternatively, the quantization residual domain could include quantized transformation coefficients.

[0129] For a block of size M×N, it can be assumed that the residual derived from the predicted values ​​obtained by performing intra-frame prediction in the horizontal direction (copying the left adjacent sample line to the prediction block line by line) using unfiltered samples from the left or top boundary samples (i.e., left adjacent samples or top adjacent samples), or by performing intra-frame prediction in the vertical direction (copying the top adjacent sample line to the prediction block line by line), is r(i,j) (0≤i≤M-1, 0≤j≤N-1). Here, M can represent the row or height, and N can represent the column or width. Furthermore, it can be assumed that the quantized value of the residual r(i,j) is Q(r(i,j)) (0≤i≤M-1, 0≤j≤N-1). Here, the residual refers to the difference between the original block and the predicted block values.

[0130] Then, if BDPCM is applied to the quantized residual samples, it is possible to derive the following: As a modified M×N array for configuration

[0131] For example, when signaling a vertical BDPCM (i.e., when applying a vertical BDPCM), the following equation can be derived:

[0132] [Equation 1]

[0133]

[0134] That is, for example, when applying vertical BDPCM, the encoding device can perform vertical intra-frame prediction based on the upper neighboring samples, and can derive the quantization residual samples of the current block as in Equation 1 above. Referring to Equation 1 above, the quantization residual samples of rows other than the first row of the current block can be derived as the difference between the quantization value at the corresponding position and the quantization value at the position of the previous row (i.e., the upper neighboring position of the corresponding position).

[0135] Furthermore, when similarly applied to horizontal prediction (i.e., when applying BDPCM in the horizontal direction), residual quantization samples can be derived as shown in the following equation.

[0136] [Equation 2]

[0137]

[0138] That is, for example, when applying horizontal BDPCM, the encoding device can perform horizontal intra-frame prediction based on the left adjacent sample and can derive the quantization residual sample of the current block as in Equation 2 above. Referring to Equation 2 above, the quantization residual sample of columns other than the first column of the current block can be derived as the difference between the quantization value at the corresponding position and the quantization value of the position in the previous column (i.e., the left adjacent position of the corresponding position).

[0139] Quantized residual samples It can be sent to a decoding device.

[0140] In the decoding device, the above operation can be performed in reverse to derive Q(r(i,j))(0≤i≤M-1,0≤j≤N-1).

[0141] The following equation can be applied to vertical prediction.

[0142] [Equation 3]

[0143]

[0144] In addition, the following equation can be applied to horizontal forecasting.

[0145] [Equation 4]

[0146]

[0147] Dequantization residual (Q) -1 (Q(r i，j The reconstructed sample value is derived by adding the predicted value within the block to the predicted value.

[0148] The main advantage of this technique is that inverse BDPCM can be performed by simply adding a predictor during or even after the resolution of the coefficients.

[0149] As described above, BDPCM can be applied to the quantization residual domain, and the quantization residual domain can include the quantization residuals (or quantization residual coefficients), in which case transform skipping is applied to the residuals. That is, when BDPCM is applied, the transform can be skipped and quantization can be applied to the residual samples. Alternatively, the quantization residual domain can include the quantization transform coefficients. A flag indicating whether BDPCM is available can be signaled at the sequence level (SPS), and this flag can only be signaled when transform skipping mode is enabled in the SPS. This flag can be referred to as the BDPCM enable flag or the SPS BDPCM enable flag.

[0150] When applying BDPCM, intra-frame prediction can be performed on the entire block by sample copying, based on a prediction direction similar to the intra-frame prediction direction (e.g., vertical or horizontal prediction). The residual, i.e., the difference between the original block and the predicted block, is quantized by skipping the transform, and an incremental value can be compiled, i.e., the difference between the quantized residual in the horizontal or vertical direction and the predictor. (i.e., quantization residuals in the horizontal or vertical direction).

[0151] If BDPCM is applicable, when the CU size is less than or equal to the MaxTsSize (maximum transform skip block size) of the luma samples, and the CU is compiled using intra-frame prediction, flag information can be sent at the CU level. This flag information can be referred to as the BDPCM flag. Here, MaxTsSize can refer to the maximum block size allowed for transform skip mode. The flag information can indicate whether conventional intra-frame compilation or BDPCM is applied. When BDPCM is applied, a BDPCM prediction direction flag indicating whether the prediction direction is horizontal or vertical can be sent. This BDPCM prediction direction flag can be referred to as the BDPCM direction flag. Subsequently, blocks can be predicted using unfiltered reference samples via conventional horizontal or vertical intra-frame prediction procedures. Additionally, residuals can be quantized, and the difference between each quantized residual and its predictor (e.g., residuals already quantized in the horizontal or vertical direction according to the BDPCM prediction direction) can be compiled.

[0152] Meanwhile, as will be described later, the above-mentioned BDPCM will be described in a standard document format.

[0153] For example, the table below shows the syntax elements used for the BDPCM enable flags mentioned above and the semantics used for the syntax elements.

[0154] [Table 1]

[0155]

[0156] [Table 2]

[0157]

[0158] Table 1 shows the `sps_bdpcm_enabled_flag` signaled in the Sequence Parameter Set (SPS). When the syntax element `sps_bdpcm_enabled_flag` is 1, it can represent the presence of flags indicating whether BDPCM is applied to the compilation unit performing intra-frame prediction, namely, "intra_bdpcm_luma_flag" and "intra_bdpcm_chroma_flag". The syntax element `sps_bdpcm_enabled_flag` can be the same syntax element used for the aforementioned BDPCM enabling flags. Furthermore, if the syntax element "sps_bdpcm_enabled_flag" does not exist, its value can be inferred to be 0.

[0159] Furthermore, for example, the syntax elements used for BDPCM flags and BDPCM direction flags, as well as the semantics of the syntax elements, can be shown in the following table.

[0160] [Table 3]

[0161]

[0162] [Table 4]

[0163]

[0164] The syntax element `bdpcm_flag` in Table 3 indicates whether BDPCM is applied to the current block. The syntax element `bdpcm_flag` can be a syntax element that flags the BDPCM. For example, when `bdpcm_flag` is 1, BDPCM can be applied to the current block, the transformation of the current block can be skipped, and `bdpcm_dir_flag` can exist to indicate the prediction direction of the current block. Furthermore, for example, when `bdpcm_flag` is 0, BDPCM can not be applied to the current block. Furthermore, for example, when `bdpcm_flag` does not exist, its value can be inferred to be 0. The current block can be a compile block. `bdpcm_dir_flag` can indicate the prediction direction of the current block. For example, referring to Table 4, when `bdpcm_dir_flag` is 1, the prediction direction of the current block can be vertical. When `bdpcm_dir_flag` is 0, the prediction direction of the current block can be horizontal. The syntax element bdpcm_flag can be a syntax element used for the aforementioned BDPCM flags, and the syntax element bdpcm_dir_flag can be a syntax element used for the aforementioned BDPCM direction flags.

[0165] Furthermore, for example, signals can be sent to the aforementioned syntax elements used for BDPCM flags and BDPCM direction flags for the luminance and chrominance components, respectively. For example, the semantics of the syntax elements can be shown in the table below.

[0166] [Table 5]

[0167]

[0168] [Table 6]

[0169]

[0170] As described above, the syntax element `intra_bdpcm_luma_flag` in Table 5 can indicate whether BDPCM is applied to the current luma block, and `intra_bdpcm_chroma_flag` can indicate whether BDPCM is applied to the current luma block or the current chroma block. For example, when the value of `intra_bdpcm_luma_flag` or `intra_bdpcm_chroma_flag` is 1, the transformation of the corresponding compile block can be skipped, and the prediction mode for the compile block can be set in the horizontal or vertical direction using `intra_bdpcm_luma_dir_flag` or `intra_bdpcm_chroma_dir_flag`, which indicates the prediction direction. When `intra_bdpcm_luma_flag` or `intra_bdpcm_chroma_flag` does not exist, its value can be inferred to be equal to 0.

[0171] For example, when the value of intra_bdpcm_luma_dir_flag or intra_bdpcm_chroma_dir_flag, which represents the prediction direction, is 0, intra_bdpcm_luma_dir_flag or intra_bdpcm_chroma_dir_flag can indicate that the BDPCM prediction direction is horizontal, and when the value of intra_bdpcm_chroma_dir_flag or intra_bdpcm_chroma_dir_flag is 1, intra_bdpcm_luma_dir_flag or intra_bdpcm_chroma_dir_flag can indicate that the BDPCM prediction direction is vertical.

[0172] In addition, an example of the inverse quantization process when BDPCM is applied is shown in the table below.

[0173] [Table 7]

[0174]

[0175]

[0176] Alternatively, an example of the inverse quantization process when BDPCM is applied is shown in the table below.

[0177] [Table 8]

[0178]

[0179]

[0180]

[0181]

[0182] Referring to Table 7 or Table 8, when the value of bdpcm_flag is 1, the inverse quantization residual value d[x][y] can be derived based on the intermediate variable dz[x][y]. Here, x is the horizontal coordinate increasing from left to right, y is the vertical coordinate increasing from top to bottom, and the position in the two-dimensional block can be represented as (x,y). Additionally, the position in the two-dimensional block indicates the (x,y) position when the top-left position of the block is set to (0,0).

[0183] For example, when the value of bdpcm_dir_flag is 0, that is, when applying level BDPCM, when x is 0, the variable dz[x][y] can be TransCoeffLevel[xTbY][yTbY][cIdx][x][y], while when x is not 0, dz[x][y] can be derived based on dz[x-1][y]+dz[x][y]. That is, when applying level BDPCM (the value of bdpcm_dir_flag is 0), the variable dz[x][y] of the sample in the first column where x is 0 is derived as TransCoeffLevel[xTbY][yTbY][cIdx][x][y] derived based on the sample's residual information, and the variable dz[x][y] of the sample in columns other than the first column where x is not 0 is derived as the sum of the dz[x-1][y] of the sample's left neighbor and the sample's dz[x][y]. Here, the dz[x][y] of the sample added to dz[x-1][y] can be derived based on the residual information of the sample that was signaled.

[0184] Additionally, for example, when bdpcm_dir_flag is 1, i.e., when vertical BDPCM is applied, the variable dz[x][y] is derived based on dz[x][y-1] + dz[x][y]. That is, when vertical BDPCM is applied (bdpcm_dir_flag is 1), the variable dz[x][y] for the sample in the first row where y is 0 is derived as TransCoeffLevel[xTbY][yTbY][cIdx][x][y] derived based on the residual information of that sample, and the variable dz[x][y] for the samples in rows other than the first row where y is not 0 is derived as the sum of dz[x][y-1] of the sample's upper neighbor and the sample's dz[x][y]. Here, dz[x][y] of the sample added to dz[x][y-1] can be derived based on the residual information signaled for that sample.

[0185] As described above, the residual at a specific position can be derived based on the sum of the residual at the previous position (i.e., left or top) in the horizontal or vertical direction and the value received as the residual information for that specific position. This is because, when BDPCM is applied, the difference between the residual sample value at the specific position (x,y) in the horizontal or vertical direction and the residual sample value at the previous position (i.e., (x-1,y) or (x,y-1)) is signaled as residual information.

[0186] Furthermore, this disclosure proposes a method for applying BDPCM between residual signals during the compilation of transform-skipped residual signals. The transformed residual signals can be uniformly distributed within the transform unit (TU). Unlike the coefficients to which the transform was applied, the residual coefficients that skipped the transform have a very high probability of resembling the residual coefficients surrounding that component. Moreover, in the case of intra-frame prediction of transform-skipped blocks, as the block size increases due to the distance from the prediction reference sample, the residual level appearing on the lower right side of the block is very likely to be greater than the residual level appearing on the upper left side of the block. Therefore, according to BDPCM, in order to improve compilation efficiency by utilizing the aforementioned characteristics of the residual distribution, row-by-row inter-residual prediction can be performed in the row or column direction as described above. Furthermore, the residuals of blocks to which BDPCM has been applied can be encoded / decoded using the residual compilation syntax for transform skipping.

[0187] Therefore, it may be reasonable to identify BDPCM as another transform skipping method. Therefore, this disclosure proposes a method for determining whether to apply BDPCM based on transform skipping conditions. For example, in an embodiment, a method can be proposed to signal / parse / compile the BDPCM flag indicating whether to apply BDPCM when `transform_skip_enabled_flag` is true (i.e., when the value of `transform_skip_enabled_flag` is 1). Furthermore, in this embodiment, a method can be proposed to signal / parse / compile the BDPCM flag based on the size of the current block. For example, when the size of the current block is less than or equal to the maximum transform block size (i.e., when the width and height of the current block are less than or equal to the maximum transform block size), the BDPCM flag can be signaled / parsed / compile. Furthermore, in this embodiment, a method can be proposed to signal / parse / compile the BDPCM flag based on `transform_skip_enabled_flag` and / or the size of the current block. For example, when transform_skip_enabled_flag is true (i.e., the value of transform_skip_enabled_flag is 1) and the size of the current block is less than or equal to the maximum transform block size (i.e., when the width and height of the current block are less than or equal to the maximum transform block size), the BDPCM flag can be signaled / parsed / encoded.

[0188] For example, the syntax proposed in this embodiment can be shown in the table below.

[0189] [Table 9]

[0190]

[0191] Referring to Table 9, the syntax element for the BDPCM flag can be `intra_bdpcm_flag`. Alternatively, for example, the syntax element for the BDPCM flag can be `intra_bdpcm_luma_flag`. Referring to Table 9, the BDPCM flag for the current block can be signaled based on `transform_skip_enabled_flag` and / or the size of the current block. For example, the BDPCM flag can be signaled when the width and height of the current block are less than or equal to the maximum transform block size, and it can be left unsigned when the width or height of the current block is greater than the maximum transform block size. Here, for example, the maximum transform block size can be derived based on information representing the maximum transform block size. Furthermore, for example, when the BDPCM flag is absent (i.e., when no signal is given for the BDPCM flag), the BDPCM flag can be inferred to be equal to 0.

[0192] Furthermore, this disclosure proposes a method for applying BDPCM between residual signals during the compilation of transform-skipped residual signals. As described above, the transform-skipped residual signals can be uniformly distributed within the transform unit (TU), unlike the coefficients to which the transform is applied. Moreover, the residual coefficients of the transformed-skipped signals have a very high probability of resembling the residual coefficients around the component. Furthermore, in the case of intra-frame prediction of transform-skipped blocks, as the block size increases due to the distance from the prediction reference sample, the probability that the residual level appearing on the lower right side of the block is greater than the residual level appearing on the upper left side of the block is very high.

[0193] Therefore, this embodiment proposes a method to determine whether to apply BDPCM by treating it as another transform skipping method but as a compiler tool separate from the transform skipping mode. The method proposed in this embodiment is based on inferring the BDPCM mode as a transform skipping mode in the prior art.

[0194] For example, the syntax and semantics proposed in this embodiment can be shown in the table below.

[0195] [Table 10]

[0196]

[0197] [Table 11]

[0198]

[0199] For example, according to this embodiment, the syntax element no_bdpcm_constraint_flag can be signaled. The syntax element no_bdpcm_constraint_flag can indicate whether BDPCM is constrained. For example, no_bdpcm_constraint_flag can be called the BDPCM constraint flag.

[0200] For example, referring to Table 11, `no_bdpcm_constraint_flag` equal to 1 can indicate that the value of the BDPCM enable flag is 0. That is, for example, `no_bdpcm_constraint_flag` equal to 1 can indicate that BDPCM is not enabled (for the entire image). `no_bdpcm_constraint_flag` equal to 0 may not impose constraints on BDPCM. Furthermore, for example, the syntax element for the BDPCM enable flag can be `bdpcm_enabled_flag`. Also, for example, the BDPCM enable flag can be defined in one or more of the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Video Parameter Set (VPS), and Slice Header. That is, for example, the BDPCM enable flag can be notified via the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Video Parameter Set (VPS), and / or Slice Header signal.

[0201] Furthermore, for example, referring to Table 11, the BDPCM flag can be inferred to be equal to 0 when the value of the BDPCM enable flag is 0 or the BDPCM flag does not exist. Therefore, for example, when the value of no_bdpcm_constraint_flag is 1, the BDPCM enable flag can be 0, and the BDPCM flag can also be inferred to be equal to 0. Additionally, the BDPCM flag can indicate whether BDPCM is applied to the current block. For example, a BDPCM flag equal to 0 can indicate that BDPCM is not applied to the current block, while a BDPCM flag equal to 1 can indicate that BDPCM is applied to the current block. That is, a BDPCM flag equal to 1 can indicate that the transform is skipped in the current block and the prediction of the current block is performed in the intra-prediction mode indicated by the BDPCM direction flag. The syntax elements of the BDPCM direction flag can be intra_bdpcm_dir_flag, intra_bdpcm_luma_dir_flag, or intra_bdpcm_chroma_dir_flag.

[0202] Additionally, for example, referring to Table 11, when there is no transform skip flag, the BDPCM enable flag is 1, and the BdcpmFlag is 1, the transform skip flag can be inferred to be equal to 1. Alternatively, for example, when there is no transform skip flag, when the BDPCM enable flag is 0 or the BdcpmFlag is 0, the transform skip flag can be inferred to be equal to 0. Here, the value of BdcpmFlag can be set to the same as the value of the BDPCM flag.

[0203] Furthermore, this disclosure proposes a method for applying BDPCM between residual signals during the compilation of transform-skipped residual signals. As described above, the transform-skipped residual signals can be uniformly distributed within the transform unit (TU), unlike the coefficients to which the transform was applied, and the probability that the transform-skipped residual coefficients are similar to the residual coefficients around the component is very high. Moreover, in the case of intra-frame predictive transform-skipped blocks, as the block size increases due to the distance from the prediction reference sample, the probability that the residual level appearing on the lower right side of the block is greater than the residual level appearing on the upper left side of the block is very high.

[0204] Therefore, this embodiment treats BDPCM as another transform skipping method, but considers it as a compilation tool separate from the transform skipping mode, and proposes another method to determine whether to apply BDPCM. The method proposed in this embodiment is based on the fact that BDPCM is executed completely independently of the transform skipping mode.

[0205] For example, the syntax and semantics proposed in this embodiment can be shown in the table below.

[0206] [Table 12]

[0207]

[0208]

[0209] [Table 13]

[0210]

[0211] For example, according to this embodiment, the syntax element no_bdpcm_constraint_flag can be signaled. The syntax element no_bdpcm_constraint_flag can indicate whether BDPCM is constrained.

[0212] For example, referring to Table 13, `no_bdpcm_constraint_flag` equal to 1 can indicate that the value of the BDPCM enable flag is 0. `no_bdpcm_constraint_flag` equal to 0 means that no constraints are imposed on BDPCM. Furthermore, for example, the syntax element for the BDPCM enable flag can be `bdpcm_enabled_flag`. Also, for example, the BDPCM enable flag can be defined in one or more of the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Video Parameter Set (VPS), and slice header. That is, for example, the BDPCM enable flag can be notified via the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Video Parameter Set (VPS), and / or slice header signal.

[0213] Additionally, for example, referring to Table 13, the BDPCM flag can be inferred to be equal to 0 when the value of the BDPCM enable flag is 0 or the BDPCM flag does not exist. Therefore, for example, when the value of no_bdpcm_constraint_flag is 0, the BDPCM enable flag can be 0, and the BDPCM flag can also be inferred to be equal to 0. Furthermore, the BDPCM flag can indicate whether BDPCM is applied to the current block. For example, a BDPCM flag equal to 0 can indicate that BDPCM is not applied to the current block, while a BDPCM flag equal to 1 can indicate that BDPCM is applied to the current block. That is, a BDPCM flag equal to 1 can indicate that the transform is skipped in the current block and the prediction of the current block is performed in the intra-prediction mode indicated by the BDPCM direction flag. The syntax elements of the BDPCM direction flag can be intra_bdpcm_dir_flag, intra_bdpcm_luma_dir_flag, or intra_bdpcm_chroma_dir_flag.

[0214] Furthermore, for example, referring to Table 13, when the transform skip flag is not present, the transform skip flag can be inferred to be equal to 0. That is, the transform skip mode can be executed independently for the BDPCM.

[0215] Furthermore, this disclosure proposes a method for applying BDPCM between residual signals during the compilation of transform-skipped residual signals. As described above, the transform-skipped residual signals can be uniformly distributed within the transform unit (TU), unlike the coefficients to which the transform was applied, and the probability that the transform-skipped residual coefficients are similar to the residual coefficients around the component is very high. Moreover, in the case of intra-frame predictive transform-skipped blocks, as the block size increases due to the distance from the prediction reference sample, the probability that the residual level appearing on the lower right side of the block is greater than the residual level appearing on the upper left side of the block is very high.

[0216] Therefore, this embodiment treats BDPCM as another transformation skipping method, but considers it as a compilation tool separate from the transformation skipping mode, and proposes another method for determining whether to apply BDPCM. The method proposed in this embodiment is based on the fact that BDPCM is executed completely independently of the transformation skipping mode. Unlike previously disclosed embodiments, it does not require a separate flag, no_bdpcm_constraint_flag, to signal the change using high-level syntax.

[0217] For example, the syntax and semantics proposed in this embodiment can be shown in the table below.

[0218] [Table 14]

[0219]

[0220] [Table 15]

[0221]

[0222] Referring to Table 14, residual compilation can be branched based on the BDPCM flag. That is, based on the value of the BDPCM flag (whether BDPCM is applied), different syntax elements can be used for residual compilation.

[0223] For example, referring to Table 14, when the transform skip flag is 0 and the BDPCM flag is 0 (i.e., when a transform is applied but BDPCM is not), regular residual compilation (RRC) can be applied to the current block, and when the transform skip flag is 1 or the BDPCM flag is 1 (i.e., when a transform is skipped or BDPCM is applied), transform skip residual compilation (TSRC) can be applied to the current block. That is, for example, when the transform skip flag is 0 and the BDPCM flag is 0 (i.e., when a transform is applied but BDPCM is not applied), the syntax element for regular residual compilation (RRC) can be signaled for the current block, and when the transform skip flag is 1 or the BDPCM flag is 1 (i.e., when a transform is skipped or BDPCM is applied), the syntax element for transform skip residual compilation (TSRC) can be signaled for the current block. Regular residual compilation can be referred to as general residual compilation. Furthermore, regular residual compilation can be referred to as a regular residual compilation syntax structure, and transform skip residual compilation can be referred to as a transform skip residual compilation syntax structure.

[0224] Furthermore, for example, referring to Table 15, when the transform skip flag is not present, the transform skip flag can be inferred to be equal to 0. That is, the transform skip mode can be executed independently for BDPCM.

[0225] Figure 7 An image encoding method according to the encoding apparatus of this document is illustrated schematically. Figure 7 The method disclosed in the article can be derived from Figure 2 The encoding device disclosed herein is used for execution. Specifically, for example, Figure 7 S700 to S720 can be executed by the predictor of the encoding device, and S730 can be executed by the entropy encoder of the encoding device. Furthermore, although not shown in the figure, the process for deriving residual samples can be executed by the residual processor of the encoding device, and the process for generating reconstructed samples and a reconstructed image of the current block based on the predicted samples and residual samples of the current block can be executed by the adder of the encoding device.

[0226] The encoding device determines whether the width and height of the current block are less than or equal to the maximum transform block size (S700). The encoding device can determine the maximum transform block size and can generate and encode information representing the maximum transform block size. For example, the information representing the maximum transform block size can be signaled via high-level syntax. For example, the information representing the maximum transform block size can be signaled via a sequence parameter set (SPS). For example, the syntax element representing the maximum transform block size could be sps_log2_transform_skip_max_size_minus2.

[0227] When the width and height are less than or equal to the maximum transform block size, the encoding device generates a BDPCM flag indicating whether to apply block-based incremental pulse code modulation (BDPCM) to the current block (S710).

[0228] For example, when the width and height are less than or equal to the maximum transform block size, the encoding device can determine whether to apply BDPCM to the current block and can generate a BDPCM flag indicating whether BDPCM is applied to the current block. For example, when the width and height are less than or equal to the maximum transform block size, the image information may include the BDPCM flag.

[0229] Additionally, for example, when the width or height is greater than the maximum transform block size—that is, when at least one of the width and height is greater than the maximum transform block size—the encoding device can determine whether to not apply BDPCM to the current block and may not generate a BDPCM flag indicating whether BDPCM is applied to the current block. BDPCM can be inferred to be equal to 0. In other words, for example, when the width or height is greater than the maximum transform block size—that is, when at least one of the width and height is greater than the maximum transform block size—the BDPCM flag indicating whether BDPCM is applied to the current block may not be signaled, and BDPCM may not be applied to the current block. For example, when the width or height is greater than the maximum transform block size—that is, when at least one of the width and height is greater than the maximum transform block size—the image information may not include a BDPCM flag for the current block.

[0230] Furthermore, for example, when the BDPCM flag is 0, it can indicate that BDPCM is not applied to the current block. When the BDPCM flag is 1, it can indicate that BDPCM is applied to the current block and that a BDPCM direction flag is present for the current block. That is, for example, when the BDPCM flag is 0, it can indicate that BDPCM is not applied to the current block, and general intra-frame prediction, IBC prediction, inter-frame prediction, or palette prediction is performed. When the BDPCM flag is 1, it can indicate that BDPCM is applied to the current block and that a BDPCM direction flag is present for the current block. For example, the syntax elements of the BDPCM flag can be `bdpcm_flag`, `intra_bdpcm_luma_flag`, or `intra_bdpcm_chroma_flag`. Furthermore, for example, the BDPCM flag can be signaled on a compilation unit (CU) basis. For example, the current block can be a compilation block.

[0231] The encoding device generates a BDPCM direction flag (S720) indicating the prediction direction for the current block.

[0232] For example, the encoding device can determine whether to apply BDPCM to the current block and can determine the direction in which BDPCM is performed. Furthermore, for example, the encoding device can generate and encode a BDPCM direction flag representing the prediction direction for the current block. Image information may include the BDPCM direction flag.

[0233] For example, the BDPCM direction flag can indicate the prediction direction of the current block. For example, when the value of the BDPCM flag is 1, the encoding device can generate and encode the BDPCM direction flag. For example, the BDPCM direction flag can represent the vertical or horizontal direction as the prediction direction of the current block. For example, when the value of the BDPCM direction flag is 0, the BDPCM direction flag can indicate that the prediction direction of the current block is horizontal, and when the value of the BDPCM direction flag is 1, the BDPCM direction flag can indicate that the prediction direction of the current block is vertical. For example, the syntax elements of the BDPCM direction flag can be bdpcm_dir_flag, intra_bdpcm_luma_dir_flag, or intra_bdpcm_chroma_dir_flag.

[0234] Furthermore, for example, the encoding device can derive prediction samples by performing intra-frame prediction on the current block based on the prediction direction of the BDPCM. For example, the prediction direction can be vertical or horizontal, and prediction samples for the current block can be generated based on the intra-frame prediction mode according to the prediction direction.

[0235] For example, when the BDPCM direction flag is 0, i.e., when the prediction direction of the current block is derived as horizontal, the encoding device can derive the prediction samples of the current block based on the horizontal intra-frame prediction mode. In other words, when the BDPCM direction flag is 0, i.e., when the prediction direction of the current block is derived as horizontal, the encoding device can derive the prediction samples of the current block by performing intra-frame prediction based on the left neighboring samples of the current block. For example, when the prediction direction of the current block is derived as horizontal, the encoding device can derive the sample values ​​of the left neighboring samples in the same row as the prediction samples as the sample values ​​of the prediction samples.

[0236] Furthermore, for example, when the BDPCM direction flag is 1, i.e., when the prediction direction of the current block is derived as vertical, the encoding device can derive the prediction samples of the current block based on the vertical intra-frame prediction mode. In other words, for example, when the BDPCM direction flag is 1, i.e., when the prediction direction of the current block is derived as vertical, the encoding device can derive the prediction samples of the current block by performing intra-frame prediction based on the upper neighbor samples of the current block. For example, when the prediction direction of the current block is derived as vertical, the encoding device can derive the sample values ​​of the upper neighbor samples in the same column as the prediction samples as the sample values ​​of the prediction samples.

[0237] Furthermore, the tree type of the current block can be classified as either a single-tree (SINGLE_TREE) or a dual-tree (DUAL_TREE) based on whether the luma block and the corresponding chroma block have separate partitioning structures. When the chroma block has the same partitioning structure as the luma block, it can be represented as a single-tree, while when the chroma component block has a different partitioning structure than the luma block, it can be represented as a dual-tree. According to the example, BDPCM can be applied separately to either the luma block or the chroma block of the current block.

[0238] When the tree structure of the current block is a two-tree structure, BDPCM can only be applied to one component block, and even when the tree structure of the current block is a single-tree structure, BDPCM may only be applied to one component block.

[0239] At the same time, for example, the encoding device can derive the residual sample of the current block based on the predicted sample. For example, the encoding device can derive the residual sample by subtracting the predicted sample from the original sample of the current block.

[0240] The encoding device encodes image information including a BDPCM flag and a BDPCM direction flag (S730). The encoding device can encode image information including a BDPCM flag and a BDPCM direction flag. That is, for example, the encoding device can encode a BDPCM flag indicating whether BDPCM is applied to the current block and a BDPCM direction flag indicating the prediction direction for the current block.

[0241] Simultaneously, image information may include residual information. For example, the encoding device may derive the residual coefficients of the current block based on residual samples. For example, when BDPCM is applied to the current block, the encoding device may determine not to apply the transform to the current block. In this case, for example, the encoding device may derive the residual coefficients by performing quantization on the residual samples. Here, for example, the block to which the transform is not applied may be called a transform-skipped block. That is, for example, the current block may be a transform-skipped block.

[0242] Then, for example, the encoding device can encode residual information about the residual coefficients. For example, the residual information may include residual information about the residual coefficients of the residual samples.

[0243] For example, residual information can include syntax elements of the residual samples of the current block. Based on the syntax elements of the target residual sample, the difference between the residual coefficient value of the target residual sample and the residual coefficient value of its left-adjacent or top-adjacent residual sample can be derived. For example, when the prediction direction of the current block is horizontal, the difference between the residual coefficient value of the target residual sample and its left-adjacent residual sample can be derived based on the syntax elements of the target residual sample. That is, for example, when the prediction direction of the current block is horizontal, the syntax elements of the target residual sample can represent the difference between the residual coefficient value of the target residual sample and its left-adjacent residual sample. As another example, when the prediction direction of the current block is vertical, the difference between the residual coefficient value of the target residual sample and its top-adjacent residual sample can be derived based on the syntax elements of the target residual sample. That is, for example, when the prediction direction of the current block is vertical, the syntax elements of the target residual sample can represent the difference between the residual coefficient value of the target residual sample and the residual coefficient value of its uppermost adjacent residual sample. Furthermore, for example, when the target residual sample is located in the first row or first column of the current block, the residual coefficient value of the target residual sample can be derived based on the syntax elements of the target residual sample. That is, when the target residual sample is located in the first row or first column of the current block, the syntax elements of the target residual sample can represent the residual coefficient value of the target residual sample.

[0244] For example, residual information may include syntax elements such as transform_skip_flag, last_sig_coeff_x_prefix, last_sig_coeff_y_prefix, last_sig_coeff_x_suffix, last_sig_coeff_y_suffix, coded_sub_block_flag, sig_coeff_flag, par_level_flag, abs_level_gt1_flag, abs_level_gtX_flag, abs_remainder, coeff_sign_flag, dec_abs_level, and / or mts_idx.

[0245] Specifically, for example, residual information may include a transform skip flag for the current block. The transform skip flag can indicate whether a transform is applied to the current block. That is, the transform skip flag can indicate whether a transform is applied to the residual coefficients of the current block. Furthermore, for example, when BDPCM is applied to the current block, the transform skip flag for the current block may not be signaled, and the value of the transform skip flag can be inferred to be equal to 1. That is, when BDPCM is applied to the current block, the residual information may not include the transform skip flag for the current block, the value of the transform skip flag can be inferred to be equal to 1, and the current block can be a transform skip block. The syntax element representing the transform skip flag can be `transform_skip_flag`.

[0246] Furthermore, for example, the residual information may include positional information indicating the position of the last non-zero residual coefficient in the residual coefficient array of the current block. That is, the residual information may include positional information indicating the position of the last non-zero residual coefficient in the scan order of the current block. The positional information may include information indicating the prefix of the column position of the last non-zero residual coefficient, information indicating the prefix of the row position of the last non-zero residual coefficient, information indicating the suffix of the column position of the last non-zero residual coefficient, and information indicating the suffix of the row position of the last non-zero residual coefficient. The syntax elements of the positional information may be last_sig_coeff_x_prefix, last_sig_coeff_y_prefix, last_sig_coeff_x_suffix, and last_sig_coeff_y_suffix. Meanwhile, non-zero residual coefficients may be referred to as valid coefficients. Furthermore, for example, when the current block is a transform skip block, the residual information may not include positional information indicating the position of the last non-zero residual coefficient in the residual coefficient array of the current block.

[0247] Furthermore, for example, residual information may include a valid coefficient flag indicating whether the residual coefficients of the residual samples of the current block are non-zero residual coefficients, a parity level flag for the parity of the coefficient levels of the residual coefficients, a first coefficient level flag indicating whether the coefficient level is greater than a first threshold, and a second coefficient level flag indicating whether the coefficient level is greater than a second threshold. Here, the valid coefficient flag may be sig_coeff_flag, the parity level flag may be par_level_flag, the first coefficient level flag may be abs_level_gt1_flag, and the second coefficient level flag may be abs_level_gt3_flag or abs_level_gtx_flag.

[0248] Furthermore, for example, residual information may include a sign flag representing the sign of the residual coefficients of the residual samples in the current block. The sign flag may be coeff_sign_flag.

[0249] Furthermore, for example, residual information may include coefficient value-related information such as the values ​​of the residual coefficients of the residual samples of the current block. This coefficient value-related information may be abs_remainder and / or dec_abs_level.

[0250] Furthermore, for example, image information may include a BDPCM constraint flag. The encoding device can generate and encode a BDPCM constraint flag indicating whether the BDPCM is constrained. That is, for example, the BDPCM constraint flag can be signaled. The BDPCM constraint flag can indicate whether the BDPCM is constrained. For example, a BDPCM constraint flag equal to 1 can indicate that the value of the BDPCM enable flag is 0. That is, for example, a BDPCM constraint flag equal to 1 can indicate that the BDPCM is unavailable (for the entire image). In other words, for example, when the value of the BDPCM constraint flag is 1, the BDPCM constraint flag can indicate that the BDPCM is unavailable (for the entire image). Therefore, when the value of the BDPCM constraint flag is 1, no signal can be sent to the BDPCM flag, and the BDPCM flag can be inferred to be equal to 0. Furthermore, for example, a BDPCM constraint flag equal to 0 can mean that no constraint is imposed on the BDPCM. In other words, for example, when the value of the BDPCM constraint flag is 0, the BDPCM constraint flag can mean that no constraint is imposed on the BDPCM. The syntax element for the BDPCM constraint flag can be no_bdpcm_constraint_flag.

[0251] Simultaneously, a bitstream including image information can be sent to the decoding device via a network or (digital) storage medium. Here, the network may include broadcast networks and / or communication networks, and the digital storage medium may include various types of storage media, such as USB flash drives, SD cards, CDs, DVDs, Blu-ray discs, HDDs, and SDDs.

[0252] Figure 8 An encoding apparatus for performing an image encoding method according to this document is illustrated schematically. Figure 7 The method disclosed in the article can be derived from Figure 8 The encoding device disclosed herein is used for execution. Specifically, for example, Figure 8 The predictor of the encoding device can perform Figure 7 S700 to S720, Figure 8 The entropy encoder of the encoding device can perform Figure 7 S730. In addition, although not shown in the figure, the process of deriving residual samples can be performed by the residual processor of the encoding device, and the process of generating reconstructed samples and reconstructed images of the current block based on the predicted samples and residual samples of the current block can be performed by the adder of the encoding device.

[0253] Figure 9 An image decoding method according to the decoding apparatus of this article is illustrated schematically. Figure 9 The method disclosed in the article can be derived from Figure 3 The decoding device disclosed herein performs the operation. Specifically, for example, Figure 9 S900 to S920 can be executed by the entropy decoder of the decoding device, S930 can be executed by the predictor of the decoding device, and S940 can be executed by the adder of the decoding device.

[0254] The decoding device determines whether the width and height of the current block are less than or equal to the maximum transform block size (S900). The decoding device can determine whether the width and height of the current block are less than or equal to the maximum transform block size. The maximum transform block size can be derived based on information representing the maximum transform block size. For example, information representing the maximum transform block size can be signaled using high-level syntax. For example, information representing the maximum transform block size can be signaled using the Sequence Parameter Set (SPS). For example, the syntax element representing the maximum transform block size could be sps_log2_transform_skip_max_size_minus2.

[0255] When the width and height are less than or equal to the maximum transform block size, the decoding device obtains a BDPCM flag (S910) indicating whether block-based incremental pulse code modulation (BDPCM) should be applied to the current block. The decoding device can obtain image information via the bitstream. The image information may include the BDPCM flag indicating whether BDPCM should be applied to the current block.

[0256] Furthermore, for example, a BDPCM flag indicating whether block-based incremental pulse code modulation (BDPCM) should be applied to the current block can be obtained based on the size of the current block. When the width and height are less than or equal to the maximum transform block size, the decoding device can obtain the BDPCM flag indicating whether block-based incremental pulse code modulation (BDPCM) should be applied to the current block. That is, for example, when the width and height are less than or equal to the maximum transform block size, the image information may include the BDPCM flag. Alternatively, when the width or height is greater than the maximum transform block size, i.e., when at least one of the width and height is greater than the maximum transform block size, the BDPCM flag indicating whether BDPCM is applied to the current block may not be obtained. For example, when the width or height is greater than the maximum transform block size, i.e., when at least one of the width and height is greater than the maximum transform block size, the BDPCM flag indicating whether BDPCM should be applied to the current block may not be signaled, and BDPCM may not be applied to the current block. In other words, for example, when the width or height is greater than the maximum transform block size, i.e., when at least one of the width and height is greater than the maximum transform block size, the image information may not include the BDPCM flag for the current block.

[0257] Furthermore, for example, when the BDPCM flag is 0, it can indicate that BDPCM is not applied to the current block. When the BDPCM flag is 1, it can indicate that BDPCM is applied to the current block and that a BDPCM direction flag is present for the current block. That is, for example, when the BDPCM flag is 0, it can indicate that BDPCM is not applied to the current block, and general intra-frame prediction, IBC prediction, inter-frame prediction, or palette prediction is performed. When the BDPCM flag is 1, it can indicate that BDPCM is applied to the current block and that a BDPCM direction flag is present for the current block. For example, the syntax elements of the BDPCM flag can be `bdpcm_flag`, `intra_bdpcm_luma_flag`, or `intra_bdpcm_chroma_flag`. Furthermore, for example, the BDPCM flag can be signaled on a compilation unit (CU) basis. For example, the current block can be a compilation block.

[0258] Additionally, image information may include, for example, a BDPCM constraint flag. That is, for example, the BDPCM constraint flag can be signaled. The BDPCM constraint flag can indicate whether BDPCM is constrained. For example, a BDPCM constraint flag equal to 1 can indicate that the BDPCM enable flag is 0. That is, for example, a BDPCM constraint flag equal to 1 can indicate that BDPCM is unavailable (for the entire image). In other words, for example, when the value of the BDPCM constraint flag is 1, the BDPCM constraint flag can indicate that BDPCM is unavailable (for the entire image). Therefore, when the value of the BDPCM constraint flag is 1, no signal may be sent to the BDPCM flag, and the BDPCM flag can be inferred to be equal to 0. Furthermore, for example, a BDPCM constraint flag equal to 0 can mean that no constraint is imposed on the BDPCM. In other words, for example, when the value of the BDPCM constraint flag is 0, the BDPCM constraint flag can mean that no constraint is imposed on the BDPCM. The syntax element for the BDPCM constraint flag can be `no_bdpcm_constraint_flag`.

[0259] The decoding device obtains a BDPCM direction flag (S920) indicating the prediction direction for the current block based on the BDPCM flag.

[0260] The decoding device can obtain the BDPCM direction flag, which indicates the prediction direction for the current block, based on the BDPCM flag. For example, the decoding device can obtain the BDPCM direction flag when the BDPCM flag indicates that BDPCM should be applied to the current block. That is, for example, the decoding device can obtain the BDPCM direction flag when the value of the BDPCM flag is 1. For example, the BDPCM direction flag can represent either a vertical or horizontal direction as the prediction direction for the current block. For example, when the value of the BDPCM direction flag is 0, the BDPCM direction flag can indicate that the prediction direction for the current block is horizontal, and when the value of the BDPCM direction flag is 1, the BDPCM direction flag can indicate that the prediction direction for the current block is vertical. For example, the syntax elements of the BDPCM direction flag can be `bdpcm_dir_flag`, `intra_bdpcm_luma_dir_flag`, or `intra_bdpcm_chroma_dir_flag`.

[0261] The decoding device derives the prediction sample of the current block based on the intra-prediction mode derived from the BDPCM direction flag (S930).

[0262] The decoding device can derive prediction samples for the current block based on an intra-prediction mode derived from the BDPCM direction flag.

[0263] For example, when the BDPCM direction flag is 0, i.e., when the BDPCM direction flag indicates that the prediction direction of the current block is horizontal, the decoding device can derive the predicted sample of the current block based on the horizontal intra-frame prediction mode. In other words, when the BDPCM direction flag is 0, i.e., when the BDPCM direction flag indicates that the prediction direction of the current block is horizontal, the decoding device can derive the predicted sample of the current block by performing intra-frame prediction based on the left adjacent sample of the current block. For example, when the prediction direction of the current block is derived to be horizontal, the decoding device can deduce the sample value of the predicted sample from the sample value of the left adjacent sample in the same row as the predicted sample.

[0264] Furthermore, for example, when the BDPCM direction flag is 1, that is, when the BDPCM direction flag indicates that the prediction direction of the current block is vertical, the encoding device can derive the prediction sample of the current block based on the vertical intra-frame prediction mode. In other words, for example, when the BDPCM direction flag is 1, that is, when the BDPCM direction flag indicates that the prediction direction of the current block is vertical, the decoding device can derive the prediction sample of the current block by performing intra-frame prediction based on the upper neighbor sample of the current block. For example, when the prediction direction of the current block is derived as vertical, the decoding device can deduce the sample value of the upper neighbor sample in the same column as the prediction sample as the sample value of the prediction sample.

[0265] The decoding device derives a reconstructed sample for the current block based on the predicted sample (S940). The decoding device can derive a reconstructed sample or a reconstructed image for the current block based on the predicted sample. For example, the decoding device can derive a reconstructed sample by adding the residual sample of the current block to the predicted sample.

[0266] At the same time, for example, the decoding device can derive the residual sample of the current block based on the residual information.

[0267] For example, when BDPCM is applied to the current block, the residual information may include syntax elements of the residual sample for the current block (i.e., when BDPCM is applied to the current block, the residual information may include syntax elements of the target residual sample for the current block), whereby the syntax elements of the target residual sample may represent the difference between the residual coefficient value of the target residual sample and the residual coefficient value of its left-neighboring or top-neighboring residual sample. In other words, for example, when BDPCM is applied to the current block, the residual information may include syntax elements of the target residual sample for the current block, and based on these syntax elements, the difference between the residual coefficient value of the target residual sample and the residual coefficient value of its left-neighboring or top-neighboring residual sample is derived.

[0268] For example, when BDPCM is applied to the current block and the prediction direction of the current block is horizontal, the syntax elements of the target residual sample can represent the difference between the residual coefficient value of the target residual sample and the residual coefficient value of its left-adjacent residual sample. That is, for example, the difference between the residual coefficient value of the target residual sample and the residual coefficient value of its left-adjacent residual sample is derived based on the syntax elements of the target residual sample. Subsequently, the residual coefficient of the target residual sample can be derived as the sum of the residual coefficient values ​​of the left-adjacent residual sample and this difference. Here, the target residual sample can be a residual sample in a column other than the first column of the current block. For example, the residual coefficient of the target residual sample can be derived based on Equation 4 above. Also, for example, when the target residual sample is a residual sample in the first column of the current block, the residual coefficient of the target residual sample can be derived based on the syntax elements of the target residual sample.

[0269] Furthermore, for example, when BDPCM is applied to the current block and the prediction direction of the current block is vertical, the syntax elements of the target residual sample can represent the difference between the residual coefficient value of the target residual sample and the residual coefficient value of its upper neighbor residual sample. That is, for example, the difference between the residual coefficient value of the target residual sample and the residual coefficient value of its upper neighbor residual sample is derived based on the syntax elements of the target residual sample. Subsequently, the residual coefficient of the target residual sample can be derived as the sum of the residual coefficient values ​​of its upper neighbor residual sample and this difference. Here, the target residual sample can be a residual sample from a row other than the first row of the current block. For example, the residual coefficient of the target residual sample can be derived based on Equation 3 above. Also, for example, when the target residual sample is a residual sample from the first row of the current block, the residual coefficient of the target residual sample can be derived based on the syntax elements of the target residual sample.

[0270] Then, for example, the decoding device can perform inverse quantization on the residual coefficients to derive the target residual sample. That is, for example, the target residual sample can be derived by performing inverse quantization on the residual coefficients.

[0271] Meanwhile, although not shown in the figure, the decoding device can obtain the residual information of the current block based on the BDPCM flag. For example, when the BDPCM flag indicates that BDPCM is applied to the current block, i.e., when BDPCM is applied to the current block, the residual information can include the syntax elements of the residual samples of the current block. The difference between the residual coefficient value of the target residual sample and the residual coefficient value of the left or upper adjacent residual sample of the target residual sample can be derived based on the syntax elements of the target residual sample. For example, when the prediction direction of the current block is horizontal, i.e., when the prediction direction of the current block is derived to be horizontal based on the BDPCM direction flag, the difference between the residual coefficient value of the target residual sample and the residual coefficient value of the left adjacent residual sample of the target residual sample can be derived based on the syntax elements of the target residual sample. For example, when the prediction direction of the current block is vertical—that is, when the prediction direction of the current block is derived as vertical based on the BDPCM direction flag—the difference between the residual coefficient value of the target residual sample and the residual coefficient value of its uppermost adjacent residual sample is derived based on the syntax elements of the target residual sample. Furthermore, when the target residual sample is located in the first row or first column of the current block, the residual coefficient value of the target residual sample can be derived based on the syntax elements of the target residual sample.

[0272] For example, residual information may include syntax elements such as transform_skip_flag, last_sig_coeff_x_prefix, last_sig_coeff_y_prefix, last_sig_coeff_x_suffix, last_sig_coeff_y_suffix, coded_sub_block_flag, sig_coeff_flag, par_level_flag, abs_level_gt1_flag, abs_level_gtX_flag, abs_remainder, coeff_sign_flag, dec_abs_level, and / or mts_idx.

[0273] Specifically, for example, residual information may include a transform skip flag for the current block. The transform skip flag can indicate whether a transform is applied to the current block. That is, the transform skip flag can indicate whether a transform is applied to the residual coefficients of the current block. Furthermore, for example, when the value of the BDPCM flag is 1, that is, when BDPCM is applied to the current block, the transform skip flag for the current block may not be signaled, and the value of the transform skip flag can be inferred to be equal to 1. That is, when the value of the BDPCM flag is 1, that is, when BDPCM is applied to the current block, the residual information may not include the transform skip flag for the current block, the value of the transform skip flag can be inferred to be equal to 1, and the current block can be a transform skip block. The syntax element representing the transform skip flag can be `transform_skip_flag`.

[0274] Furthermore, for example, the residual information may include positional information indicating the position of the last non-zero residual coefficient in the residual coefficient array of the current block. That is, the residual information may include positional information indicating the position of the last non-zero residual coefficient in the scan order of the current block. The positional information may include information indicating the prefix of the column position of the last non-zero residual coefficient, information indicating the prefix of the row position of the last non-zero residual coefficient, information indicating the suffix of the column position of the last non-zero residual coefficient, and information indicating the suffix of the row position of the last non-zero residual coefficient. The syntax elements of the positional information may be last_sig_coeff_x_prefix, last_sig_coeff_y_prefix, last_sig_coeff_x_suffix, and last_sig_coeff_y_suffix. Meanwhile, non-zero residual coefficients may be referred to as valid coefficients. Furthermore, for example, when the current block is a transform skip block, the residual information may not include positional information indicating the position of the last non-zero residual coefficient in the residual coefficient array of the current block.

[0275] Furthermore, for example, residual information may include a valid coefficient flag indicating whether the residual coefficients of the residual samples of the current block are non-zero residual coefficients, a parity level flag for the parity of the coefficient levels of the residual coefficients, a first coefficient level flag indicating whether the coefficient level is greater than a first threshold, and a second coefficient level flag indicating whether the coefficient level is greater than a second threshold. Here, the valid coefficient flag may be sig_coeff_flag, the parity level flag may be par_level_flag, the first coefficient level flag may be abs_level_gt1_flag, and the second coefficient level flag may be abs_level_gt3_flag or abs_level_gtx_flag.

[0276] Furthermore, for example, residual information may include a sign flag representing the sign of the residual coefficients of the residual samples in the current block. The sign flag may be coeff_sign_flag.

[0277] Furthermore, for example, residual information may include coefficient value-related information such as the values ​​of the residual coefficients of the residual samples of the current block. This coefficient value-related information may be abs_remainder and / or dec_abs_level.

[0278] The decoding device can derive reconstructed samples by adding the predicted samples and the residual samples. Then, as needed, to improve the subjective / objective image quality, as described above, in-loop filtering processes such as deblocking filtering and SAO and / or ALF processes can be applied to the reconstructed samples.

[0279] Figure 10 A decoding apparatus for performing an image decoding method according to this document is illustrated schematically. Figure 9 The method disclosed in the article can be derived from Figure 10 The decoding device disclosed herein performs the operation. Specifically, for example, Figure 10 The entropy decoder of the decoding device can perform Figure 9 The S900 to S920, Figure 10 The predictor of the decoding device can perform Figure 9 The S930, and Figure 10 The adder of the decoding device can perform Figure 9 The S940.

[0280] According to this disclosure, by signaling the BDPCM flag based on the current block size and the maximum transform block size, the current block size and the maximum transform block size can be considered in the signaling and determination of whether to apply BDPCM flags, and in this way, the number of bits of BDPCM can be reduced and the overall compilation efficiency can be improved.

[0281] Furthermore, according to this disclosure, a syntax element can be signaled to indicate whether to apply BDPCM constraints to an image, and in this way, a single syntax element can be used to determine whether to perform BDPCM on an image, thereby improving the overall image compilation efficiency.

[0282] In the above embodiments, the method is described based on a flowchart having a series of steps or blocks. This disclosure is not limited to the order of the above steps or blocks. Some steps or blocks may be performed in a different order than the other steps or blocks described above, or may be performed simultaneously. Furthermore, those skilled in the art will understand that the steps shown in the flowchart are not exclusive, and may include other steps, or one or more steps in the flowchart may be deleted without affecting the scope of this disclosure.

[0283] The embodiments described in this specification can be executed by being implemented on a processor, microprocessor, controller, or chip. For example, the functional unit shown in each figure can be executed by being implemented on a computer, processor, microprocessor, controller, or chip. In this case, the information for implementation (e.g., information about instructions) or algorithm can be stored in a digital storage medium.

[0284] Furthermore, the decoding and encoding devices using this disclosure can be included in devices such as: multimedia broadcasting transmitters / receivers, mobile communication terminals, home theater video devices, digital cinema video devices, surveillance cameras, video chat devices, real-time communication devices such as video communication, mobile streaming devices, storage media, portable video cameras, VoD service providers, over-the-top (OTT) video devices, internet streaming service providers, three-dimensional (3D) video devices, teleconferencing video devices, transportation user devices (e.g., vehicle user devices, airplane user devices, and ship user devices), and medical video equipment; and the decoding and encoding devices using this disclosure can be used to process video signals or data signals. For example, over-the-top (OTT) video devices may include game consoles, Blu-ray players, internet-access televisions, home theater systems, smartphones, tablet computers, digital video recorders (DVRs), etc.

[0285] Furthermore, the processing methods of this disclosure can be generated in the form of a computer-executable program and can be stored in a computer-readable recording medium. Multimedia data with data structures according to this disclosure can also be stored in a computer-readable recording medium. A computer-readable recording medium includes all types of storage devices in which computer-readable data is stored. Computer-readable recording media can include, for example, BD, Universal Serial Bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices. Additionally, computer-readable recording media include media implemented in the form of a carrier wave (e.g., transmission via the Internet). Furthermore, bitstreams generated by encoding methods can be stored in a computer-readable recording medium or transmitted via wired / wireless communication networks.

[0286] Furthermore, embodiments of this disclosure can be implemented using computer program products based on program code, and the program code can be executed on a computer using embodiments of this disclosure. The program code can be stored on a computer-readable medium.

[0287] Figure 11 A structural diagram of a content streaming system that applies this disclosure is illustrated.

[0288] The content streaming system using the embodiments described herein may primarily include an encoding server, a streaming server, a network server, a media storage device, a user device, and a multimedia input device.

[0289] An encoding server compresses content input from multimedia input devices such as smartphones, cameras, or camcorders into digital data to generate a bitstream, which is then sent to a streaming server. As another example, when multimedia input devices such as smartphones, cameras, or camcorders generate bitstreams directly, the encoding server can be omitted.

[0290] A bitstream can be generated by an encoding method or bitstream generation method that applies the embodiments of this disclosure, and the stream server can temporarily store the bitstream during the sending or receiving of the bitstream.

[0291] The streaming server sends multimedia data to the user's device via a web server based on user requests, and the web server acts as a medium for notifying the user of services. When a user requests a desired service from the web server, the web server delivers the request to the streaming server, and the streaming server sends the multimedia data to the user. In this scenario, the content streaming system may include a separate control server. In this case, the control server is used to control the commands / responses between devices within the content streaming system.

[0292] A streaming server can receive content from media storage and / or encoding servers. For example, when receiving content from an encoding server, the content can be received in real time. In this case, to provide a smooth streaming service, the streaming server can store the bitstream for a predetermined period of time.

[0293] Examples of user equipment can include mobile phones, smartphones, laptops, digital broadcast terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigators, touchscreen PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, and head-mounted displays), digital TVs, desktop computers, and digital signage. Each server within the content streaming system can operate as a distributed server, in which case data received from each server can be distributed.

[0294] The claims described in this disclosure can be combined in various ways. For example, the technical features of the method claims of this disclosure can be combined to implement an apparatus, and the technical features of the apparatus claims of this disclosure can be combined to implement a method. Furthermore, the technical features of the method claims and the apparatus claims of this disclosure can be combined to implement an apparatus, and the technical features of the method claims and the apparatus claims of this disclosure can be combined to implement a method.

Claims

1. An image decoding method performed by a decoding device, comprising: Obtain the transform skip start flag from the bitstream; Determine whether the width and height of the current block are less than or equal to the maximum block size for transformation skipping, the maximum block size for transformation skipping being derived based on information representing the maximum transformation block size, the information being signaled from the bit stream; Based on the transform skip start flag being equal to 1 and the width and height being less than or equal to the maximum block size, a BDPCM flag indicating whether block-based incremental pulse code modulation (BDPCM) is applied to the current block is obtained; Based on the BDPCM flag, a BDPCM direction flag representing the prediction direction for the current block is obtained; Based on the intra-frame prediction mode derived from the BDPCM direction flag, the prediction sample of the current block is derived. as well as Based on the predicted samples, the reconstructed samples for the current block are derived. Specifically, when the BDPCM is applied to the current block and no signal is sent to notify the transform skip flag for the current block, the value of the transform skip flag is derived to be equal to 1, and the current block is derived as a transform skip block.

2. The method according to claim 1, wherein, When at least one of the width and the height is greater than the maximum block size, no signal is sent to the BDPCM flag, and The BDPCM is not applied to the current block.

3. The method according to claim 1, wherein, When the value of the BDPCM flag is 1, the BDPCM flag indicates that the BDPCM is applied to the current block and the BDPCM direction flag is present.

4. The method according to claim 3, wherein, When the value of the BDPCM direction flag is 0, the BDPCM direction flag indicates that the prediction direction for the current block is horizontal. Wherein, when the value of the BDPCM direction flag is 1, the BDPCM direction flag indicates that the prediction direction for the current block is the vertical direction.

5. The method according to claim 4, wherein, When the value of the BDPCM direction flag is 0, the prediction sample for the current block is derived based on the horizontal intra-frame prediction mode. When the value of the BDPCM direction flag is 1, the prediction sample of the current block is derived based on the vertical intra-frame prediction mode.

6. The method according to claim 1, wherein, A signal is sent to indicate whether the BDPCM constraint flag is constrained.

7. The method according to claim 6, wherein, When the value of the BDPCM constraint flag is 1, the BDPCM constraint flag indicates that BDPCM is not enabled for the image.

8. The method according to claim 4, wherein, When the BDPCM is applied to the current block and the prediction direction for the current block is the vertical direction, the residual information includes syntax elements for the target residual sample of the current block. The syntax element used for the target residual sample represents the difference between the residual coefficient value of the target residual sample and the residual coefficient value of the upper adjacent residual sample of the target residual sample.

9. The method according to claim 8, wherein, The difference is derived based on the syntax elements used for the target residual sample. The residual coefficient of the target residual sample is derived by summing the residual coefficient values ​​of the adjacent residual samples and the differences between them.

10. The method according to claim 4, wherein, When the BDPCM is applied to the current block and the prediction direction for the current block is the horizontal direction, the residual information includes syntax elements for the target residual sample of the current block. The syntax element used for the target residual sample represents the difference between the residual coefficient value of the target residual sample and the residual coefficient value of the left adjacent residual sample of the target residual sample.

11. The method according to claim 10, wherein, The difference is derived based on the syntax elements used for the target residual sample. The residual coefficient of the target residual sample is derived by summing the residual coefficient values ​​of the left adjacent residual samples and the difference between them.

12. An image encoding method performed by an encoding device, comprising: Generate a transformation to skip the start flag; Determine the maximum block size to be used for transformation skipping; Generate information representing the maximum block size used for transformation skipping; Determine whether the width and height of the current block are less than or equal to the maximum block size used for transformation skipping; Based on the transform skip start flag being equal to 1 and the width and height being less than or equal to the maximum block size, a BDPCM flag indicating whether block-based incremental pulse code modulation (BDPCM) is applied to the current block is generated; Generate a BDPCM direction flag representing the prediction direction for the current block; as well as The image information, including the transform skip start flag, the information, the BDPCM flag, and the BDPCM direction flag, is encoded. Specifically, when the BDPCM is applied to the current block and no signal is sent to notify the transform skip flag for the current block, the value of the transform skip flag is derived to be equal to 1, and the current block is derived as a transform skip block.

13. The method according to claim 12, wherein, When at least one of the width and the height is greater than the maximum block size, no signal is sent to the BDPCM flag, and The BDPCM is not applied to the current block.

14. The method according to claim 12, wherein, A signal is sent to indicate whether the BDPCM constraint flag is constrained.

15. A non-transitory computer-readable storage medium for storing a bitstream generated by a method, the method comprising: Generate a transformation to skip the start flag; Determine the maximum block size to be used for transformation skipping; Generate information representing the maximum block size used for transformation skipping; Determine whether the width and height of the current block are less than or equal to the maximum block size used for transformation skipping; Based on the transform skip start flag being equal to 1 and the width and height being less than or equal to the maximum block size, a BDPCM flag indicating whether block-based incremental pulse code modulation (BDPCM) is applied to the current block is generated; Generate a BDPCM direction flag representing the prediction direction for the current block; The image information, including the transform skip start flag, the information, the BDPCM flag, and the BDPCM direction flag, is encoded. as well as Generate the bitstream including the image information. Specifically, when the BDPCM is applied to the current block and no signal is sent to notify the transform skip flag for the current block, the value of the transform skip flag is derived to be equal to 1, and the current block is derived as a transform skip block.

16. A method for transmitting data for an image, comprising: A bitstream of image information is obtained, the image information including a transform skip start flag, a BDPCM flag indicating whether block-based incremental pulse code modulation (BDPCM) is applied to the current block, and a BDPCM direction flag indicating the prediction direction for the current block; as well as The data stream containing the image information is transmitted, including the transform skip start flag, the BDPCM flag, and the BDPCM direction flag. Specifically, it is determined whether the width and height of the current block are less than or equal to the maximum block size used for transformation skipping. Specifically, the BDPCM flag is generated based on the transformation skip start flag being equal to 1 and the width and height being less than or equal to the maximum block size. Specifically, when the BDPCM is applied to the current block and no signal is sent to notify the transform skip flag for the current block, the value of the transform skip flag is derived to be equal to 1, and the current block is derived as a transform skip block.

Images