Image encoding / decoding method and apparatus, and recording medium storing a bitstream.

Abstract

The image decoding method and apparatus according to this disclosure can divide a current block into a plurality of subblocks based on any one of a predefined block division type, and decode the plurality of subblocks based on a predetermined decoding order. Here, the decoding order may be determined based on any one of one or more scan order candidates, and the one or more scan order candidates may include at least one of a horizontal Z scan order or a vertical Z scan order.

Description

Technical Field

[0001] The present invention relates to an image encoding / decoding method and apparatus, and a recording medium storing a bitstream.

Background Art

[0002] In recent years, demand for high-resolution and high-quality images such as HD (High Definition) images and UHD (Ultra High Definition) images has been increasing in various application fields, and accordingly, high-efficiency image compression technologies are being discussed.

[0003] As image compression technologies, there exist various technologies including an inter prediction technology that predicts pixel values included in a current picture from pictures previous or subsequent to the current picture, an intra prediction technology that predicts pixel values included in the current picture using pixel information within the current picture, and an entropy coding technology that assigns short codes to values with high occurrence frequency and long codes to values with low occurrence frequency, and image data can be effectively compressed, transmitted or stored using such image compression technologies.

Summary of Invention

Problems to be Solved by the Invention

[0004] The present disclosure intends to provide a method and apparatus that use an adaptive decoding / encoding order for sub-blocks obtained by tree-based block division.

[0005] The present disclosure intends to provide a method and apparatus for efficiently signaling information / syntax for using the adaptive decoding / encoding order.

[0006] According to the present disclosure, it is intended to provide a method and apparatus that use an adaptive decoding / encoding order for the highest-level block units constituting one picture.

Means for Solving the Problems

[0007] The image decoding method and apparatus relating to this disclosure can divide a current block into a plurality of subblocks based on any one of a predefined type of block division, and can decode the plurality of subblocks based on a predetermined decoding order. Here, the decoding order may be determined based on any one of one or more scan order candidates. The one or more scan order candidates may include (may include; may constitute; may construct; may set; may encompass; may include; may contain; may have; may possess) at least one of a horizontal Z scan order or a vertical Z scan order.

[0008] In the image decoding method and apparatus relating to this disclosure, the decoding order may be determined based on index information indicating any one of the one or more scan order candidates.

[0009] In the image decoding method and apparatus relating to this disclosure, the index information may be signaled based on information indicating whether or not the current block is divided into four subblocks.

[0010] The image decoding method and apparatus relating to this disclosure can obtain from the bitstream a higher-level syntax indicating whether or not an adaptive scan order is used.

[0011] In the image decoding method and apparatus relating to this disclosure, the higher-level syntax may be signaled by at least one of the sequence parameter set, picture parameter set, or picture header.

[0012] In the image decoding method and apparatus relating to this disclosure, the index information may be signaled based on the higher-level syntax.

[0013] In the image decoding method and apparatus relating to this disclosure, the current block, including the current block, may be divided into a plurality of top-level block units, and the plurality of top-level block units may be decoded based on a decoding order that combines the raster scan order and the Z scan order.

[0014] The image encoding method and apparatus according to this disclosure can divide a current block into a plurality of subblocks based on any one of a predefined block division type, and can encode the plurality of subblocks based on a predetermined encoding order. Here, the encoding order may be determined based on any one of one or more scan order candidates. The one or more scan order candidates may include at least one of a horizontal Z scan order or a vertical Z scan order.

[0015] A computer-readable digital storage medium is provided on which encoded video / image information is stored, which is configured to perform an image decoding method using the decoding device relating to this disclosure.

[0016] A computer-readable digital storage medium is provided on which video / image information generated by the image encoding method relating to this disclosure is stored.

[0017] A method and apparatus for transmitting video / image information generated by the image encoding method relating to this disclosure are provided. [Effects of the Invention]

[0018] According to this disclosure, encoding efficiency can be improved by using an adaptive decoding / encoding order for lower blocks obtained through tree-based block partitioning.

[0019] According to the present disclosure, coding efficiency can be improved by efficiently signaling syntax for using an adaptive decoding / encoding order.

[0020] According to the present disclosure, coding efficiency can be improved by using an adaptive decoding / encoding order for the highest-level block units constituting one picture. Brief Description of Drawings

[0021] [Figure 1] This is a diagram illustrating a video / image coding system according to the present disclosure. [Figure 2] This is a schematic block diagram of an encoding apparatus to which the embodiments of the present disclosure are applicable and in which video / image signal encoding is performed. [Figure 3] This is a schematic block diagram of a decoding apparatus to which the embodiments of the present disclosure are applicable and in which video / image signal decoding is performed. [Figure 4] This is an embodiment according to the present disclosure, and is a diagram illustrating an image decoding method performed by a decoding apparatus 300. [Figure 5] This is a diagram illustrating a decoding order according to a raster scan order for a plurality of CTUs in one picture. [Figure 6-7] This is a diagram illustrating an example of using an adaptive scan order for a plurality of CTUs in one picture. [Figure 8] This is a diagram illustrating a schematic configuration of a decoding apparatus 300 that performs the image decoding method according to the present disclosure. [Figure 9] This is an embodiment according to the present disclosure, and is a diagram illustrating an image encoding method performed by an encoding apparatus 200. [Figure 10] This is a diagram illustrating a schematic configuration of an encoding apparatus 200 that performs the image encoding method according to the present disclosure. [Figure 11]This is a diagram illustrating an example of a content streaming system to which the embodiments of the present disclosure can be applied. [Description of Embodiments]

[0022] The present disclosure may be subjected to various modifications and have various embodiments, and specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the present disclosure to the specific embodiments, and it should be understood that the present disclosure covers all modifications, equivalents or alternatives falling within the spirit and technical scope of the present disclosure. In the description of each drawing, similar reference numerals are used for similar components.

[0023] Terms such as first, second, etc. may be used to describe various components, but these components should not be limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first component may be named a second component without departing from the scope of rights of the present disclosure, and similarly, a second component may also be named a first component. The term "and / or" includes any combination of a plurality of associated listed items or any one of a plurality of associated listed items.

[0024] When a component is referred to as being "coupled to" or "connected to" another component, it should be understood that it may be directly coupled or directly connected to the other component, or additional other components may be present in between. On the other hand, when a component is referred to as being "directly coupled to" or "directly connected to" another component, it should be understood that no additional other components are present in between.

[0025] The terminology used in this application is solely for the purpose of describing specific embodiments and is not intended to limit the disclosure. Singular expressions include plural expressions unless otherwise specified in the context. In this application, terms such as “includes” or “having” are intended to specify the existence of features, figures, stages, operations, components, parts, or combinations thereof described in the specification, and should be understood not to preemptively exclude the possibility of the existence or addition of one or more other features, figures, stages, operations, components, parts, or combinations thereof.

[0026] This disclosure relates to video / image coding. For example, the methods / examples disclosed herein may be applied to methods disclosed in the VVC (versatile video coding) standard. Also, the methods / examples disclosed herein may be applied to methods disclosed in the EVC (essential video coding) standard, AV1 (AOMedia Video 1) standard, AVS2 (2nd generation of audio video coding standard), or next-generation video / image coding standards (e.g., H.267 or H.268).

[0027] This specification presents various embodiments relating to video / image coding, and unless otherwise specified, the above embodiments may be combined with each other.

[0028] In this specification, video can mean a collection of images over time. Picture generally means a unit representing a single image at a specific time point, and slice / tile is a unit that constitutes part of a picture in coding. A slice / tile may contain one or more CTUs (coding tree units). A picture may consist of one or more slices / tiles. A tile is a rectangular area consisting of multiple CTUs in a specific tile column and a specific tile row of a picture. A tile column is a rectangular area of ​​CTUs having the same height as the picture height and the width specified by the syntax requirements of the picture parameter set. A tile row is a rectangular area of ​​CTUs having the same height as the picture parameter set and the width as the picture width. CTUs within a tile may be arranged consecutively by a CTU raster scan, while tiles within a picture may be arranged consecutively by a tile raster scan. A slice may contain an integer number of complete tiles or an integer number of consecutive complete CTU rows within a picture tile that can exclusively be contained within a single NAL unit. On the other hand, a picture may be divided into two or more subpictures. A subpicture may be a rectangular region of one or more slices in the picture.

[0029] A pixel, or pel, can refer to the smallest unit that makes up a picture (or image). The term "sample" may also be used as a counterpart to "pixel." A sample can generally represent a pixel or a pixel value, and may represent only the pixel / pixel value for the lumen component, or only the pixel / pixel value for the chroma component.

[0030] A unit can refer to a basic unit of image processing. A unit may contain at least one of a specific region of a picture and information associated with that region. A unit may contain one luma block and two chroma (e.g., cb, cr) blocks. The term unit may, as in some cases, be used interchangeably with terms such as block or area. Generally, an MxN block may contain a sample (or sample array) or a set (or array) of transform coefficients consisting of M columns and N rows.

[0031] In this specification, "A or B" can mean "A only," "B only," or "both A and B." In other words, in this specification, "A or B" may be interpreted as "A and / or B." For example, in this specification, "A, B or C" can mean "A only," "B only," "C only," or "any combination of A, B and C."

[0032] In this specification, a slash ( / ) or comma can mean "and / or". For example, "A / B" can mean "A and / or B". Thus, "A / B" can mean "A only", "B only", or "both A and B". For example, "A, B, C" can mean "A, B or C".

[0033] In this specification, "at least one of A and B" can mean "A only," "B only," or "both A and B." Furthermore, in this specification, the expressions "at least one of A or B" or "at least one of A and / or B" may be interpreted as equivalent to "at least one of A and B."

[0034] Furthermore, in this specification, "at least one of A, B and C" can mean "A only," "B only," "C only," or "any combination of A, B and C." Also, "at least one of A, B or C" or "at least one of A, B and / or C" can mean "at least one of A, B and C."

[0035] Furthermore, parentheses used in this specification can mean "for example." Specifically, when "prediction (intra prediction)" is displayed, "intra prediction" may be proposed as an example of "prediction." In other words, "prediction" in this specification is not limited to "intra prediction," and "intra prediction" may be proposed as an example of "prediction." Also, when "prediction (i.e., intra prediction)" is displayed, "intra prediction" may be proposed as an example of "prediction."

[0036] In this specification, technical features described individually in the same drawings may be embodied individually or simultaneously.

[0037] Figure 1 shows the video / image coding system related to this disclosure.

[0038] Referring to Figure 1, the video / image coding system may include a first device (source device) and a second device (receiving device).

[0039] A source device can transmit encoded video / image information or data to a receiving device in the form of a file or streaming via a digital storage medium or network. The source device may include a video source, an encoding device, and a transmission unit. The receiving device may include a receiving unit, a decoding device, and a renderer. The encoding device may also be called a video / image encoding device, and the decoding device may also be called a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, which may consist of a separate device or external component.

[0040] A video source can acquire video / images through processes such as video / image capture, synthesis, or generation. A video source may include a video / image capture device and / or a video / image generation device. A video / image capture device may include one or more cameras, a video / image archive containing previously captured video / images, etc. A video / image generation device may include a computer, tablet, and smartphone, and can generate video / images (electronically). For example, virtual video / images may be generated through a computer, in which case the video / image capture process may be replaced by the process of generating the associated data.

[0041] An encoding device can encode input video / images. The encoding device can perform a series of steps such as prediction, transformation, and quantization for compression and coding efficiency. The encoded data (encoded video / image information) may be output in the form of a bitstream.

[0042] The transmission unit can transmit encoded video / image information or data output in bitstream format to the receiving unit of a receiving device via a digital storage medium or network in file or streaming format. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray (registered trademark: same hereinafter), HDD, SSD, etc. The transmission unit may include elements for generating media files in a predetermined file format and may include elements for transmission via a broadcast / communication network. The receiving unit can receive / extract the bitstream and transmit it to a decoding device.

[0043] A decoding device can decode video / images by performing a series of steps, such as inverse quantization, inverse transformation, and prediction, corresponding to the operation of an encoding device.

[0044] The renderer can render the decoded video / image. The rendered video / image may be displayed on the display unit.

[0045] Figure 2 is a schematic block diagram of an encoding device to which the embodiments of this disclosure can be applied, in which video / image signals are encoded.

[0046] Referring to Figure 2, the encoding device 200 may include an image partitioner (210), a predictor (220), a residual processor (230), an entropy encoder (240), an adder (250), a filter (260), and a memory (270). The predictor (220) may include an inter-prediction unit (221) and an intra-prediction unit (222). The residual processor (230) may include a transformer (232), a quantizer (233), a dequantizer (234), and an inverse transformer (235). The residual processor (230) may further include a subtractor (231). The addition unit 250 may be called a reconstructor or a reconstructed block generator. The image segmentation unit 210, prediction unit 220, residual processing unit 230, entropy encoding unit 240, addition unit 250, and filtering unit 260 described above may be composed of one or more hardware components (e.g., an encoding device chipset or processor) depending on the embodiment. The memory 270 may also include a DPB (decoded picture buffer) and may be composed of a digital storage medium. The hardware components may further include the memory 270 as an internal / external component.

[0047] The image splitting unit 210 can split an input image (or picture, frame) input to the encoding device 200 into one or more processing units. For example, the processing unit may be called a coding unit (CU). In this case, the coding unit may be recursively split from a coding tree unit (CTU) or the largest coding unit (LCU) using a QTBTTT (Quad-tree binary-tree ternary-tree) structure.

[0048] As an example, a single coding unit may be divided into multiple coding units having deeper depths based on a quad-tree structure, a binary tree structure, and / or a tertiary structure. In this case, for example, the quad-tree structure may be applied first, followed by the binary tree structure and / or tertiary structure. Alternatively, the binary tree structure may be applied before the quad-tree structure. The coding procedure according to this specification may be performed based on a final coding unit that is not further divided. In this case, based on coding efficiency due to image characteristics, the largest coding unit may be immediately used as the final coding unit, or, if necessary, the coding unit may be recursively divided into coding units of lower depths, and the coding unit with the optimal size may be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and reconstruction, which will be described later.

[0049] As another example, the processing unit may further include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit may be separated or partitioned from the final coding unit described above. The prediction unit may be a unit of sample prediction, and the transform unit may be a unit that derives a transform coefficient and / or a unit that derives a residual signal from the transform coefficient.

[0050] The term "unit" may be used interchangeably with terms such as "block" or "area." Generally, an MxN block may represent a set of samples or transform coefficients consisting of M columns and N rows. A sample may generally represent a pixel or a pixel value, and may represent only the pixel / pixel value of the lumen component, or only the pixel / pixel value of the chroma component. The term "sample" may be used in conjunction with a single picture (or image), pixel, or pel.

[0051] The encoding device 200 can generate a residual signal (residual block, residual sample array) by subtracting the prediction signal (prediction block, prediction sample array) output from the inter-prediction unit 221 or intra-prediction unit 222 from the input image signal (original block, original sample array), and the generated residual signal is transmitted to the conversion unit 232. In this case, the unit that subtracts the prediction signal (prediction block, prediction sample array) from the input image signal (original block, original sample array) within the encoding device 200 may be called the subtraction unit 231.

[0052] The prediction unit 220 can make predictions for the block to be processed (hereinafter referred to as the current block) and generate a predicted block containing prediction samples for the current block. The prediction unit 220 can determine whether intra-prediction or inter-prediction is applied to the current block or on a CU basis. As will be described later in the explanation of each prediction mode, the prediction unit 220 can generate various information related to prediction, such as prediction mode information, and transmit it to the entropy encoding unit 240. The information related to prediction may be encoded by the entropy encoding unit 240 and output in the form of a bitstream.

[0053] The intra-prediction unit 222 can predict the current block by referring to a sample in the current picture. The referenced sample may be located in the vicinity (neighbor) of the current block, or it may be located at a certain distance from the current block, depending on the prediction mode. In intra-prediction, the prediction mode may include one or more non-directional modes and multiple directional modes. The non-directional mode may include at least one of DC mode or planar mode. The directional mode may include 33 or 65 directional modes, depending on the degree of fineness of the prediction direction. However, this is an example, and more or fewer directional modes may be used depending on the settings. The intra-prediction unit 222 can also determine the prediction mode to be applied to the current block using the prediction modes applied to the surrounding blocks.

[0054] The interprediction unit 221 can guide a predicted block for the current block based on a reference block (reference sample array) identified by a motion vector on the reference picture. In this case, in order to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between the surrounding block and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of interprediction, the surrounding block may include a spatial neighboring block existing in the current picture and a temporal neighboring block existing in the reference picture. The reference picture containing the reference block and the reference picture containing the temporal neighboring block may be the same or different. The temporal neighboring block may be called a collocated reference block, colCU, etc., and the reference picture containing the temporal neighboring block may be called a collocated picture (colPic). For example, the interpretation unit 221 can construct a motion information candidate list based on surrounding blocks and generate information indicating which candidates are used to derive the motion vector and / or reference picture index of the current block. Interpretation may be performed based on various prediction modes; for example, in skip mode and merge mode, the interpretation unit 221 can use the motion information of surrounding blocks as the motion information of the current block. In skip mode, unlike merge mode, the residual signal does not need to be transmitted.In motion vector prediction (MVP) mode, the motion vectors of surrounding blocks are used as motion vector predictors, and the motion vector difference is signaled to indicate the motion vector of the current block.

[0055] The prediction unit 220 can generate prediction signals based on various prediction methods described later. For example, the prediction unit can apply intra-prediction or inter-prediction for predictions for a single block, and can also apply intra-prediction and inter-prediction simultaneously. This may be called CIIP (combined inter and intra prediction) mode. The prediction unit may also be based on intra-block copy (IBC) prediction mode or palette mode for predictions for blocks. The IBC prediction mode or palette mode may be used for coding content images / videos such as games, as in SCC (screen content coding). IBC basically performs predictions within the current picture, but can be performed similarly to inter-prediction in that it derives reference blocks within the current picture. That is, IBC can use at least one of the inter-prediction methods described herein. Palette mode may be considered an example of intra-coding or intra-prediction. When palette mode is applied, in-picture sample values ​​can be signaled based on information about the palette table and palette index. The prediction signal generated by the prediction unit 220 may be used to generate a restoration signal or to generate a residual signal.

[0056] The transformation unit 232 can generate transformation coefficients by applying a transformation method to the residual signal. For example, the transformation method may include at least one of the following: DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT (Karhunen-Loeve Transform), GBT (Graph-Based Transform), or CNT (Conditionally Non-linear Transform). Here, GBT refers to the transformation obtained from a graph when the relationship information between pixels is represented by a graph. CNT refers to the transformation obtained by generating a prediction signal using all previously restored pixels and obtaining a transformation based on it. Furthermore, the transformation process may be applied to pixel blocks of the same size that are square, or to blocks of variable size that are not square.

[0057] The quantization unit 233 quantizes the conversion coefficients and transmits them to the entropy encoding unit 240, which can encode the quantized signal (information regarding the quantized conversion coefficients) and output it as a bitstream. The information regarding the quantized conversion coefficients may be called residual information. The quantization unit 233 can rearrange the block-shaped quantized conversion coefficients into a one-dimensional vector based on the coefficient scan order, and can generate information regarding the quantized conversion coefficients based on the one-dimensional vector-shaped quantized conversion coefficients.

[0058] The entropy encoding unit 240 can perform various encoding methods such as exponential Golomb, CAVLC (context-adaptive variable length coding), and CABAC (context-adaptive binary arithmetic coding). In addition to the quantized conversion coefficients, the entropy encoding unit 240 can also encode information necessary for video / image restoration (e.g., the values ​​of syntax elements) together with or separately from the quantized conversion coefficients.

[0059] The encoded information (e.g., encoded video / image information) may be transmitted or stored in the form of a bitstream in units of NAL (network abstraction layer) units. The video / image information may further include information about various parameter sets, such as an adaptation parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), or video parameter set (VPS). The video / image information may also further include general constraint information. In this specification, information and / or syntax elements transmitted / signaled from an encoding device to a decoding device may be included in the video / image information. The video / image information may be encoded by the encoding procedure described above and included in the bitstream. The bitstream may be transmitted over a network or stored on a digital storage medium. Here, the network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The signal output from the entropy encoding unit 240 may be transmitted by a transmission unit (not shown) and / or stored by a storage unit (not shown) which are configured as internal / external elements of the encoding device 200, or the transmission unit may be included in the entropy encoding unit 240.

[0060] The quantized conversion coefficients output from the quantization unit 233 may be used to generate a prediction signal. For example, by applying inverse quantization and inverse transformation to the quantized conversion coefficients in the inverse quantization unit 234 and the inverse transformation unit 235, the residual signal (residual block or residual sample) can be reconstructed. The adder unit 250 may add the reconstructed residual signal to the prediction signal output from the inter-prediction unit 221 or the intra-prediction unit 222 to generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array). When there is no residual for the block to be processed, such as when skip mode is applied, the predicted block may be used as the reconstructed block. The adder unit 250 may be called the reconstruction unit or the reconstructed block generation unit. The generated reconstructed signal may be used for intra-prediction of the next block to be processed in the current picture, and may be used for inter-prediction of the next picture after filtering, as described later. On the other hand, LMCS (luma mapping with chroma scaling) may be applied during the picture encoding and / or reconstruction process.

[0061] The filtering unit 260 can apply filtering to the restored signal to improve subjective / objective image quality. For example, the filtering unit 260 can apply various filtering methods to the restored picture to generate a modified restored picture, and the modified restored picture can be stored in the memory 270, specifically in the DPB of the memory 270. The various filtering methods may include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filtering unit 260 can generate various filtering-related information and transmit it to the entropy encoding unit 240. The filtering-related information may be encoded by the entropy encoding unit 240 and output in bitstream format.

[0062] The corrected restored picture transmitted to memory 270 may be used as a reference picture in the interpretation unit 221. This allows the encoding device to avoid prediction mismatches between the encoding device 200 and the decoding device when interpretation is applied, and also improves encoding efficiency.

[0063] The DPB in memory 270 can store the corrected restored picture for use as a reference picture in the inter-prediction unit 221. Memory 270 can store motion information of blocks from which motion information in the current picture has been derived (or encoded), and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit 221 for use as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. Memory 270 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 222.

[0064] Figure 3 is a schematic block diagram of a decoding device to which the embodiments of this disclosure can be applied, in which video / image signals are decoded.

[0065] Referring to Figure 3, the decoding device 300 may be configured to include an entropy decoder (310), a residual processor (320), a predictor (330), an adder (340), a filter (350), and a memory (360). The predictor (330) may include an inter-prediction unit (331) and an intra-prediction unit (332). The residual processor (320) may include a dequantizer (321) and an inverse transformer (321).

[0066] The entropy decoding unit 310, the residual processing unit 320, the prediction unit 330, the addition unit 340, and the filtering unit 350 described above may be configured by a single hardware component (e.g., a decoding device chipset or processor) depending on the embodiment. The memory 360 may include a DPB (decoded picture buffer) and may be configured by a digital storage medium. The hardware component may further include the memory 360 as an internal / external component.

[0067] When a bitstream containing video / image information is input, the decoding device 300 can reconstruct the image in accordance with the process by which the video / image information was processed in the encoding device shown in Figure 2. For example, the decoding device 300 can derive units / blocks based on block division-related information obtained from the bitstream. The decoding device 300 can perform decoding using the processing units applied in the encoding device. Therefore, the decoding processing units may be coding units, which may be divided from a coding tree unit or a maximum coding unit according to a quad-tree structure, a binary tree structure, and / or a tertiary tree structure. One or more conversion units may be derived from the coding unit. The reconstructed image signal decoded and output by the decoding device 300 may then be reproduced by a playback device.

[0068] The decoding device 300 can receive the signal output from the encoding device shown in Figure 2 in the form of a bitstream, and the received signal may be decoded by the entropy decoding unit 310. For example, the entropy decoding unit 310 can parse the bitstream and derive information necessary for image restoration (or picture restoration) (e.g., video / image information). The video / image information may further include information about various parameter sets such as the adaptation parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), or video parameter set (VPS). The video / image information may also further include general constraint information. The decoding device can decode the picture based on the parameter set information and / or the general constraint information. The signal / received information and / or syntax elements described later in this specification may be decoded by the decoding procedure and obtained from the bitstream. For example, the entropy decoding unit 310 can decode information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of syntax elements necessary for image reconstruction and the quantized values ​​of conversion coefficients related to the residual. More specifically, the CABAC entropy decoding method receives bins corresponding to each syntax element in the bitstream, determines a context model using the syntax element information to be decoded and the decoding information of the surrounding and decoded blocks or symbol / bin information decoded in a previous stage, predicts the probability of bin occurrence based on the determined context model, and generates symbols corresponding to the values ​​of each syntax element by performing arithmetic decoding of the bins. At this time, after determining the context model, the CABAC entropy decoding method can update the context model using the symbol / bin information decoded for the context model of the next symbol / bin.Information related to prediction from the information decoded by the entropy decoding unit 310 is provided to the prediction unit (inter-prediction unit 332 and intra-prediction unit 331), and residual values ​​that have been entropy decoded by the entropy decoding unit 310, i.e., quantized conversion coefficients and related parameter information, may be input to the residual processing unit 320. The residual processing unit 320 can derive residual signals (residual blocks, residual samples, residual sample arrays). In addition, information related to filtering from the information decoded by the entropy decoding unit 310 may be provided to the filtering unit 350. On the other hand, a receiving unit (not shown) that receives signals output from the encoding device may be further configured as an internal / external element of the decoding device 300, or the receiving unit may be a component of the entropy decoding unit 310.

[0069] On the other hand, the decoding device according to this specification may be called a video / image / picture decoding device, and the decoding device may be divided into an information decoding device (video / image / picture information decoding device) and a sample decoding device (video / image / picture sample decoding device). The information decoding device may include the entropy decoding unit 310, and the sample decoding device may include at least one of the inverse quantization unit 321, inverse transformation unit 322, addition unit 340, filtering unit 350, memory 360, inter-prediction unit 332, and intra-prediction unit 331.

[0070] The inverse quantization unit 321 can inverse quantize the quantized transformation coefficients and output the transformation coefficients. The inverse quantization unit 321 can rearrange the quantized transformation coefficients in a two-dimensional block form. In this case, the rearrangement can be performed based on the coefficient scan order performed by the encoding device. The inverse quantization unit 321 can perform inverse quantization on the quantized transformation coefficients using quantization parameters (e.g., quantization step size information) and obtain the transformation coefficients.

[0071] The inverse conversion unit 322 performs an inverse conversion on the conversion coefficients to obtain a resistive signal (residual block, resistive sample array).

[0072] The prediction unit 320 can make predictions for the current block and generate a predicted block containing prediction samples for the current block. Based on the prediction information output from the entropy decoding unit 310, the prediction unit 320 can determine whether intra-prediction or inter-prediction is applied to the current block and determine a specific intra / inter-prediction mode.

[0073] The prediction unit 320 can generate prediction signals based on various prediction methods described later. For example, the prediction unit 320 can apply intra-prediction or inter-prediction for prediction of a single block, or it can apply intra-prediction and inter-prediction simultaneously. This may be called CIIP (combined inter and intra prediction) mode. The prediction unit may also be based on intra-block copy (IBC) prediction mode or palette mode for prediction of a block. The IBC prediction mode or palette mode may be used for content image / video coding such as SCC (screen content coding) for games. IBC basically performs prediction within the current picture, but can be performed similarly to inter-prediction in that it derives a reference block within the current picture. That is, IBC can use at least one of the inter-prediction methods described herein. Palette mode can be considered an example of intra-coding or intra-prediction. When palette mode is applied, information about the palette table and palette index may be included in the video / image information and signaled.

[0074] The intra-prediction unit 331 can predict the current block by referring to a sample in the current picture. The referenced sample may be located in the vicinity (neighbor) of the current block, or it may be located at a certain distance from the current block, depending on the prediction mode. In intra-prediction, the prediction mode may include one or more non-directional modes and multiple directional modes. The intra-prediction unit 331 can determine the prediction mode to be applied to the current block using the prediction modes applied to the surrounding blocks.

[0075] The interprediction unit 332 can derive a predicted block for the current block based on a reference block (reference sample array) identified by motion vectors on the reference picture. In this case, in order to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between surrounding blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of interprediction, surrounding blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the interprediction unit 332 can construct a motion information candidate list based on surrounding blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Interprediction may be performed based on various prediction modes, and the prediction information may include information indicating the interprediction mode for the current block.

[0076] The adder 340 can generate a restored signal (restored picture, restored block, restored sample array) by adding the acquired residual signal to the predicted signal (predicted block, predicted sample array) output from the prediction unit (including the inter-prediction unit 332 and / or intra-prediction unit 331). When there is no residual for the block to be processed, such as when skip mode is applied, the predicted block may be used as the restored block.

[0077] The summing unit 340 may be called the restoration unit or the restoration block generation unit. The generated restoration signal may be used for intra-prediction of the next block to be processed in the current picture, and may be output after filtering as described later, or may be used for intra-prediction of the next picture. On the other hand, LMCS (luma mapping with chroma scaling) may be applied during the picture decoding process.

[0078] The filtering unit 350 can apply filtering to the restored signal to improve subjective / objective image quality. For example, the filtering unit 350 can apply various filtering methods to the restored picture to generate a modified restored picture, and transmit the modified restored picture to the memory 360, specifically to the DPB of the memory 360. The various filtering methods may include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, and the like.

[0079] The restored picture stored (modified) in the DPB of memory 360 may be used as a reference picture in the inter-prediction unit 332. Memory 360 can store motion information of blocks from which motion information in the current picture has been derived (or decoded), and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit 260 for use as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. Memory 360 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 331.

[0080] In this specification, the embodiments described for the filtering unit 260, the inter-prediction unit 221, and the intra-prediction unit 222 of the encoding device 200 may also be applied identically or in a corresponding manner to the filtering unit 350, the inter-prediction unit 332, and the intra-prediction unit 331 of the decoding device 300, respectively.

[0081] Figure 4 is a diagram illustrating an embodiment of the present disclosure, which shows an image decoding method performed by a decoding device 300.

[0082] Referring to Figure 4, the current block can be divided into multiple subblocks (S400).

[0083] The current block in this disclosure may be a coding block or a transformation block. Each of the subblocks may be a coding block or a transformation block in which at least one of the widths or heights is smaller than that of the current block. The partition depth of each subblock may be greater than the partition depth of the current block. For example, if the partition depth of the current block is N, the partition depth of each subblock may be (N+1). The subblocks may be square or rectangular. At least one of the subblocks may have a geometric shape. The terms subblock in this disclosure may be replaced by terms such as subblock or partition.

[0084] The current block may be divided based on one or more dividing lines. The dividing lines may include at least one of vertical lines, horizontal lines, or dividing lines at a predetermined angle. At least one of the dividing lines may pass through the center of the current block, or it may not pass through the center of the current block.

[0085] The division may be performed based on any one of the predefined types of block division. The type of block division may include at least one of quad split, binary split, or ternary split. The quad split may be a type that divides one coding block into four subblocks, where the width and height of the four subblocks may each be half the width and height of the coding block. The binary split may be a type that divides one coding block into two subblocks. The ternary split may be a type that divides one coding block into three subblocks. The image decoding method according to this disclosure can use tree-based block division.

[0086] The division information for the aforementioned block division may be signaled. The division information may include at least one of the following: information indicating whether or not a block will be divided, information indicating whether or not one block will be divided into four subblocks, information indicating whether or not one block will be divided into two subblocks, or information indicating the direction of division.

[0087] As an example, information (split_cu_flag) regarding whether the current block will be split may be signaled. If split_cu_flag is false (or 0), this may indicate that the current block will not be split. In this case, decoding may be performed by the form and / or size of the current block without further block splitting of the current block. On the other hand, if split_cu_flag is true (or 1), this may indicate that the current block will be split into lower blocks using further block splitting. The further block splitting may be based on quad splitting, binary splitting, or ternary splitting. Alternatively, the further block splitting may be based on asymmetric binary splitting.

[0088] If split_cu_flag is true, split_qt_flag may be signaled during the signaling process for the type of further block splitting of the current block. split_qt_flag can indicate whether the current block will be split based on a quad split. If split_qt_flag is true (or 1), the current block may be split into four subblocks. On the other hand, if split_qt_flag is false (or 0), the current block does not have to be split into four subblocks, and the current block may be split into subblocks using other block splitting methods. However, split_qt_flag may be signaled only if the current block satisfies all the conditions for a quad split; otherwise, split_qt_flag may be set to false.

[0089] If `split_qt_flag` is false, information for a block split other than a quad split may be signaled for the current block. At least one of `mtt_split_cu_vertical_flag` or `mtt_split_cu_binary_flag` may be signaled. `mtt_split_cu_vertical_flag` can indicate the splitting direction of the current block. If `mtt_split_cu_vertical_flag` is true (or 1), the current block may be split vertically, and if `mtt_split_cu_vertical_flag` is false (or 0), the current block may be split horizontally. `mtt_split_cu_binary_flag` can indicate whether the current block is split based on a binary split. If mtt_split_cu_binary_flag is true (or 1), the current block may be split into two subblocks; if mtt_split_cu_binary_flag is false (or 0), the current block may be split into three subblocks.

[0090] Therefore, if both mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag are true, the current block may be split vertically into two subblocks. If mtt_split_cu_vertical_flag is true and mtt_split_cu_binary_flag is false, the current block may be split vertically into three subblocks. If mtt_split_cu_vertical_flag is false and mtt_split_cu_binary_flag is true, the current block may be split horizontally into two subblocks. If both mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag are false, the current block may be split horizontally into three subblocks.

[0091] The aforementioned signaling method for partitioning information may be applied to a single coding block, and this may be applied recursively to multiple subblocks partitioned from that single coding block.

[0092] Referring to Figure 4, multiple subblocks can be decoded based on a predetermined decoding order (S410).

[0093] The decoding order of subblocks within a current block may be determined based on one or more predefined scan order candidates. These one or more scan order candidates may include at least one of either a horizontal Z-scan order or a vertical Z-scan order.

[0094] Let's assume that the current block is divided into four subblocks based on a quad partition. In this case, the four subblocks may be decoded sequentially according to a horizontal Z-scan order. Here, the horizontal Z-scan order can mean the order in which the four subblocks obtained through the quad partition are decoded in the order of the upper left subblock, the upper right subblock, the lower left subblock, and the lower right subblock. Alternatively, the four subblocks may be decoded sequentially according to a vertical Z-scan order. Here, the vertical Z-scan order can mean the order in which the four subblocks obtained through the quad partition are decoded in the order of the upper left subblock, the lower left subblock, the upper right subblock, and the lower right subblock.

[0095] One or more scan order candidates for which blocks are currently available may be selectively used.

[0096] The number and / or types of one or more scan order candidates currently available for a block may be derived based on the block partitioning type of the current block.

[0097] For example, if the current block is divided based on a quad partition, the number of available scan order candidates for the current block may be two or more. On the other hand, if the current block is not divided based on a quad partition, the number of available scan order candidates for the current block may be one. Alternatively, if the current block is divided based on a quad partition, the one or more available scan order candidates for the current block may include a horizontal Z scan order and a vertical Z scan order. On the other hand, if the current block is not divided based on a quad partition, the one or more available scan order candidates for the current block may include a horizontal Z scan order but not a vertical Z scan order.

[0098] Index information indicating one or more scan order candidates may be explicitly signaled.

[0099] For example, the current block may be divided into four subblocks based on a quad partition, and index information indicating either a horizontal Z-scan order or a vertical Z-scan order may be signaled to the four subblocks.

[0100] Specifically, if the current block is to be split into four subblocks (i.e., split_cu_flag is true and split_qt_flag is true), then qt_scan_flag may be signaled. That is, qt_scan_flag may be index information for selecting either the horizontal Z scan order or the vertical Z scan order when the current block is to be split into four subblocks based on a quad split.

[0101] If qt_scan_flag is true (or 1), the four lower blocks may be decoded sequentially according to the horizontal Z scan order. On the other hand, if qt_scan_flag is false (or 0), the four lower blocks may be decoded sequentially according to the vertical Z scan order. Conversely, if qt_scan_flag is true, the four lower blocks may be decoded sequentially according to the vertical Z scan order, and if qt_scan_flag is false, the four lower blocks may be decoded sequentially according to the horizontal Z scan order.

[0102] Tables 1 and 2 below show the syntax and semantics corresponding to the examples described above, respectively. As shown in Table 1, if split_qt_flag indicates that the current block will be split into four subblocks (i.e., split_qt_flag is true), then qt_scan_flag may be signaled to the current block.

[0103] [Table 1] TIFF2026534054000003.tif254170TIFF2026534054000004.tif129170

[0104] [Table 2]

[0105] The subblocks within the current block can be decoded in the decoding order determined by the selected scan order candidate.

[0106] As an example, the decoding process for each subblock may further include a block partitioning process for that subblock. Partitioning information for the block partitioning of the subblock may be further signaled, which is the same as the signaling method for partitioning information described above.

[0107] The use / availability of the aforementioned adaptive scan order may be signaled using high-level syntax.

[0108] As an example, the use / availability of adaptive scan order may be signaled by the sequence parameter set (SPS). Table 3 is a syntax table for signaling the use / availability of adaptive scan order through the sequence parameter set. In Table 3, sps_adaptive_scan_order_flag can indicate whether adaptive scan order is used / available in the sequence currently using sps_seq_parameter_set_id among multiple sequences belonging to the bitstream. For example, if sps_adaptive_scan_order_flag is true (or 1), adaptive scan order may be used or available in the sequence currently using sps_seq_parameter_set_id among multiple sequences belonging to the bitstream. On the other hand, if sps_adaptive_scan_order_flag is false (or 0), adaptive scan order may not be used or available in the sequence currently using sps_seq_parameter_set_id among multiple sequences belonging to the bitstream.

[0109] [Table 3]

[0110] As an example, the use / availability of adaptive scan order can be signaled by the picture parameter set (PPS). Table 4 is a syntax table for signaling the use / availability of adaptive scan order using the picture parameter set. In Table 4, pps_adaptive_scan_order_flag can indicate whether adaptive scan order is used / available in the picture currently using pps_seq_parameter_set_id among the multiple pictures belonging to the bitstream. For example, if pps_adaptive_scan_order_flag is true (or 1), adaptive scan order may be used or available in the picture currently using pps_seq_parameter_set_id among the multiple pictures belonging to the bitstream. On the other hand, if pps_adaptive_scan_order_flag is false (or 0), adaptive scan order may not be used or available in the picture currently using pps_seq_parameter_set_id among the multiple pictures belonging to the bitstream.

[0111] [Table 4]

[0112] As an example, the use / availability of adaptive scan order may be signaled in the picture header (PH). Table 5 is a syntax table for signaling the use / availability of adaptive scan order using the picture header. In Table 5, ph_adaptive_scan_order_flag can indicate whether adaptive scan order is used / available in the current picture. For example, if ph_adaptive_scan_order_flag is true (or 1), adaptive scan order may be used or available in the current picture. On the other hand, if ph_adaptive_scan_order_flag is false (or 0), adaptive scan order may not be used or available in the current picture.

[0113] The ph_adaptive_scan_order_flag may be signaled if sps_adaptive_scan_order_flag is true. If sps_adaptive_scan_order is false, the ph_adaptive_scan_order_flag may not be signaled and may be led to false.

[0114] [Table 5]

[0115] The higher-level syntax for signaling the use / availability of the adaptive scan sequence is not limited to SPS, PPS, or PH. For example, the higher-level syntax may be signaled at at least one level of tile, slice, or CTU row / column.

[0116] The index information (qt_scan_flag) indicating one of the aforementioned one or more scan order candidates may be signaled based on at least one of the above-level syntax.

[0117] For example, if sps_adaptive_scan_order_flag is true, qt_scan_flag may be signaled in the bitstream; otherwise, qt_scan_flag does not need to be signaled in the bitstream. If pps_adaptive_scan_order_flag is true, qt_scan_flag may be signaled in the bitstream; otherwise, qt_scan_flag does not need to be signaled in the bitstream. If ph_adaptive_scan_order_flag is true, qt_scan_flag may be signaled in the bitstream; otherwise, qt_scan_flag does not need to be signaled in the bitstream.

[0118] The qt_scan_flag may be signaled based on the split_qt_flag, which indicates whether the block is currently being split based on a quad partition.

[0119] For example, if split_qt_flag is true, qt_scan_flag may be signaled in the bitstream; otherwise, qt_scan_flag does not need to be signaled in the bitstream.

[0120] The qt_scan_flag may be signaled based on at least one of the higher-level syntaxes mentioned above, and the split_qt_flag for the current block.

[0121] For example, if split_qt_flag is true and sps_adaptive_scan_order_flag is true, then qt_scan_flag may be signaled in the bitstream. On the other hand, if at least one of split_qt_flag and sps_adaptive_scan_order_flag is false, then qt_scan_flag does not need to be signaled in the bitstream.

[0122] Alternatively, if split_qt_flag is true and pps_adaptive_scan_order_flag is true, qt_scan_flag may be signaled in the bitstream. On the other hand, if at least one of split_qt_flag and pps_adaptive_scan_order_flag is false, qt_scan_flag does not need to be signaled in the bitstream.

[0123] Alternatively, as shown in Table 6, if split_qt_flag is true and ph_adaptive_scan_order_flag is true, qt_scan_flag may be signaled in the bitstream. On the other hand, if at least one of split_qt_flag and ph_adaptive_scan_order_flag is false, qt_scan_flag does not need to be signaled in the bitstream.

[0124] [Table 6]

[0125] A syntax indicating whether or not adaptive scan order is used in the coding unit may be further signaled. This syntax may be represented by cu_adaptive_scan_order_flag.

[0126] For example, if cu_adaptive_scan_order_flag is true (or 1), an adaptive scan order may be used for the current block. In this case, index information indicating one or more of the aforementioned scan order candidates may be signaled. On the other hand, if cu_adaptive_scan_order_flag is false (or 0), an adaptive scan order may not be used for the current block. In this case, the default scan order (e.g., horizontal Z scan order) may be used for the current block. If cu_adaptive_scan_order_flag is not signaled, cu_adaptive_scan_order_flag may be induced to false.

[0127] The cu_adaptive_scan_order_flag may be signaled based on at least one of the higher-level syntaxes mentioned above.

[0128] For example, if sps_adaptive_scan_order_flag is true, cu_adaptive_scan_order_flag may be signaled in the bitstream; otherwise, cu_adaptive_scan_order_flag does not need to be signaled in the bitstream. If pps_adaptive_scan_order_flag is true, cu_adaptive_scan_order_flag may be signaled in the bitstream; otherwise, cu_adaptive_scan_order_flag does not need to be signaled in the bitstream. If ph_adaptive_scan_order_flag is true, cu_adaptive_scan_order_flag may be signaled in the bitstream; otherwise, cu_adaptive_scan_order_flag does not need to be signaled in the bitstream.

[0129] The cu_adaptive_scan_order_flag may be signaled based on the split_qt_flag, which indicates whether the block is currently being split based on a quad partition.

[0130] For example, if split_qt_flag is true, cu_adaptive_scan_order_flag may be signaled in the bitstream; otherwise, cu_adaptive_scan_order_flag does not need to be signaled in the bitstream.

[0131] The cu_adaptive_scan_order_flag may be signaled based on at least one of the higher-level syntaxes mentioned above, and the split_qt_flag for the current block.

[0132] For example, if split_qt_flag is true and sps_adaptive_scan_order_flag is true, then cu_adaptive_scan_order_flag may be signaled in the bitstream. On the other hand, if at least one of split_qt_flag and sps_adaptive_scan_order_flag is false, then cu_adaptive_scan_order_flag does not need to be signaled in the bitstream. If split_qt_flag is true and pps_adaptive_scan_order_flag is true, then cu_adaptive_scan_order_flag may be signaled in the bitstream. On the other hand, if at least one of split_qt_flag and pps_adaptive_scan_order_flag is false, then cu_adaptive_scan_order_flag does not need to be signaled in the bitstream. If split_qt_flag is true and ph_adaptive_scan_order_flag is true, then cu_adaptive_scan_order_flag may be signaled in the bitstream. On the other hand, if at least one of split_qt_flag and ph_adaptive_scan_order_flag is false, then cu_adaptive_scan_order_flag does not need to be signaled in the bitstream.

[0133] As mentioned above, the decoding process for each subblock may include a block partitioning process for that subblock. In this case, at least one of qt_scan_flag or cu_adaptive_scan_order_flag may be signaled at the node of the subblock along with the partitioning information for block partitioning. Therefore, if the current block is partitioned into multiple first subblocks, and the first subblocks are further partitioned into multiple second subblocks, the decoding order of the first subblocks in the current block may differ from the decoding order of the second subblocks in the first subblock. Here, if the partitioning depth of the current block is N, the partitioning depth of the first subblock may be (N+1), and the partitioning depth of the second subblock may be (N+2).

[0134] Currently, a picture may be divided into multiple top-level block units (or coding tree units, CTUs), and an adaptive decoding order may be applied to these multiple top-level block units. This will be explained in detail below with reference to Figures 5 to 7.

[0135] Figure 5 shows the decoding order based on the raster scan order for multiple CTUs in a single picture.

[0136] As shown in Figure 5, decoding can be performed within a single picture in the order of the first CTU column to the last CTU column, and within each CTU column, decoding can be performed in the order of the leftmost CTU to the rightmost CTU. This can be defined as the decoding order according to the raster scan order. That is, it can mean a scan order in which all CTUs contained in the first CTU column are decoded, and then the CTUs contained in the second CTU column are decoded sequentially.

[0137] Figures 6 and 7 illustrate an example of using adaptive scan order for multiple CTUs in a single picture. A decoding order combining Z-scan order and raster scan order may be used within a predefined number of CTU columns.

[0138] Figure 6 shows the case where the number of predefined CTU columns is 2. Referring to Figure 6, the first and second CTUs of the first CTU column can be decoded sequentially, and then the first and second CTUs of the second CTU column can be decoded sequentially according to the Z-scan order. Subsequently, the third and fourth CTUs of the first CTU column can be decoded sequentially according to the raster scan order, and then the third and fourth CTUs of the second CTU column can be decoded sequentially according to the Z-scan order.

[0139] Figure 7 shows the case where the number of predefined CTU columns is 4. Referring to Figure 7, the picture can be decoded in units of 4 CTU columns according to the Z-scan order, and then the CTU columns located in the raster scan order can be decoded again in units of 4 CTU columns according to the Z-scan order.

[0140] In this way, encoding efficiency can be improved by using a decoding order that combines the Z-scan order and the raster scan order within a predefined number of CTU columns.

[0141] The number of predefined CTU columns may be an integer that is a multiple of 2. Alternatively, the number of predefined CTU columns may be a fixed value predefined in the encoding and decoding devices. Or, information for identifying the number of CTU columns may be separately signaled. This information may be signaled at at least one level of the sequence parameter set, picture parameter set, picture header, or slice header. As an example, the information signaled for using a decoding order that combines the Z scan order and the raster scan order may be as shown in Table 7 below.

[0142] [Table 7]

[0143] According to Table 7, a flag may be signaled indicating whether or not a CTU-based adaptive scan order is used in the sequence currently using sps_seq_parameter_set_id among multiple sequences in the bitstream. This flag may be expressed as sps_adaptive_ctu_scan_order_flag.

[0144] If sps_adpative_ctu_scan_order_flag is true, the CTU-based adaptive scan order may be used for the sequence currently using sps_seq_parameter_id among multiple sequences in the bitstream. On the other hand, if sps_adaptive_ctu_scan_order_flag is false, the CTU-based adaptive scan order may not be used for the sequence currently using sps_seq_parameter_id among multiple sequences in the bitstream.

[0145] Only when sps_adaptive_ctu_scan_order_flag is true, information regarding the number of CTU columns that use an adaptive scan order on a CTU-by-CTU basis in the sequence (sps_num_of_scan_lines) may be explicitly signaled. The number of CTU columns that use an adaptive scan order may be expressed as an integer value that is a multiple of 2. For example, if the value of sps_num_of_scan_lines is N, the number of CTU columns that use an adaptive scan order may be induced to be (2*N). Alternatively, if the value of sps_num_of_scan_lines is N, the number of CTU columns that use an adaptive scan order may be induced to be 2N.

[0146] Figure 8 shows a schematic configuration of a decoding device 300 that performs the image decoding method according to this disclosure.

[0147] Referring to Figure 8, the decoding device 300 may include a memory 360 and at least one processor 800.

[0148] The processor 800 can currently divide a block into multiple subblocks. The division may be based on one of a predefined type of block division. The type of block division may include at least one of quad division, binary division, or ternary division. The binary division may include at least one of symmetric binary division or asymmetric binary division. For this purpose, the processor 800 can obtain division information from the bitstream indicating one of the types of block division, as illustrated with reference to Figure 4. The processor 800 can also use tree-based block division.

[0149] The processor 800 can decode multiple lower blocks based on a predetermined decoding order. The decoding order of the lower blocks within the current block may be determined based on one or more predefined scan order candidates, as explained with reference to Figure 4.

[0150] The processor 800 can determine whether or not to use adaptive scan order based on high-level syntax. For example, the high-level syntax may be signaled at at least one level of SPS, PPS, or PH, as described with reference to Figure 4. However, the high-level syntax may also be signaled at at least one level of Tile, Slice, or CTU row / column.

[0151] Furthermore, as explained with reference to Figure 4, the processor 800 can obtain index information (qt_scan_flag) from the bitstream that indicates one of the aforementioned one or more scan order candidates based on at least one of the above-mentioned higher-level syntaxes. Alternatively, the processor 800 can obtain qt_scan_flag from the bitstream based on split_qt_flag, which indicates whether the current block is to be split based on quad partitioning. Alternatively, the processor 800 can obtain qt_scan_flag from the bitstream based on at least one of the above-mentioned higher-level syntaxes and split_qt_flag for the current block.

[0152] The processor 800 can obtain a syntax (cu_adaptive_scan_order_flag) from the bitstream indicating whether or not an adaptive scan order is used in the coding unit, as explained with reference to Figure 4.

[0153] The processor 800 can now divide the picture into multiple top-level block units (or coding tree units, CTUs) and decode these multiple top-level block units based on an adaptive decoding order. The adaptive decoding order for these multiple top-level block units is described with reference to Figures 5 to 7.

[0154] Figure 9 is a diagram illustrating an embodiment of the present disclosure, which shows an image encoding method performed by an encoding device 200.

[0155] Referring to Figure 9, the current block can be divided into multiple subblocks (S900).

[0156] The partitioning may be performed based on any one of a predefined type of block partitioning. The type of block partitioning may include at least one of quad partitioning, binary partitioning, or ternary partitioning. Here, binary partitioning may include at least one of symmetric binary partitioning or asymmetric binary partitioning. The image encoding method according to this disclosure may use tree-based block partitioning.

[0157] The division information for indicating any one of the aforementioned block division types may be encoded and signaled, and the signaling method for the division information is as described with reference to Figure 4.

[0158] Referring to Figure 9, multiple lower blocks can be encoded based on a predetermined encoding order (S910).

[0159] The encoding order of subblocks within a current block may be determined based on one or more predefined scan order candidates, where one or more scan order candidates may include at least one of either a horizontal Z-scan order or a vertical Z-scan order.

[0160] One or more scan order candidates currently available for the block may be selectively used. The number and / or types of one or more scan order candidates currently available for the block may be determined based on the block partitioning type of the current block.

[0161] Index information indicating one of the scan order candidates selected from one or more scan order candidates may be encoded and signaled. The method for signaling the index information is as described with reference to Figure 4.

[0162] The subblocks within the current block can be encoded in the encoding order determined by the selected scan order candidate. As an example, the encoding process for each subblock may further include a block partitioning process for that subblock. Partitioning information for the block partitioning of the subblock may be further encoded and signaled, which is the same as the signaling method for partitioning information described above.

[0163] The high-level syntax regarding the use / availability of the adaptive scan order described above may be encoded and signaled. The high-level syntax is as explained with reference to Figure 4.

[0164] The index information (qt_scan_flag) indicating any one of the aforementioned one or more scan order candidates may be encoded and signaled based on at least one of the above-level syntaxes. Alternatively, qt_scan_flag may be encoded and signaled based on split_qt_flag indicating whether the current block is to be split based on a quad partition. Alternatively, qt_scan_flag may be encoded and signaled based on at least one of the above-level syntaxes and split_qt_flag for the current block.

[0165] The coding unit may determine whether or not an adaptive scan order is used, and a syntax (cu_adaptive_scan_order_flag) indicating this may be further encoded and signaled.

[0166] For example, if it is determined that an adaptive scan order should be used for the current block, cu_adaptive_scan_order_flag may be encoded to true (or 1). If cu_adaptive_scan_order_flag is true, index information indicating one or more candidate scan orders may be encoded and signaled. On the other hand, if it is determined that an adaptive scan order should not be used for the current block, cu_adaptive_scan_order_flag may be encoded to false (or 0). In this case, the encoding order for the current block may be induced to the default scan order (e.g., horizontal Z scan order). If cu_adaptive_scan_order_flag is not encoded, cu_adaptive_scan_order_flag may be induced to false.

[0167] The cu_adaptive_scan_order_flag may be encoded into a bitstream based on at least one of the higher-level syntaxes described above.

[0168] For example, if sps_adaptive_scan_order_flag is true, cu_adaptive_scan_order_flag may be encoded into a bitstream; otherwise, cu_adaptive_scan_order_flag does not need to be encoded into a bitstream. If pps_adaptive_scan_order_flag is true, cu_adaptive_scan_order_flag may be encoded into a bitstream; otherwise, cu_adaptive_scan_order_flag does not need to be encoded into a bitstream. If ph_adaptive_scan_order_flag is true, cu_adaptive_scan_order_flag may be encoded into a bitstream; otherwise, cu_adaptive_scan_order_flag does not need to be encoded into a bitstream.

[0169] The cu_adaptive_scan_order_flag may be encoded based on the split_qt_flag, which indicates whether the block is currently being split based on a quad partition.

[0170] For example, if split_qt_flag is true, cu_adaptive_scan_order_flag may be encoded into a bitstream; otherwise, cu_adaptive_scan_order_flag does not need to be encoded into a bitstream.

[0171] cu_adaptive_scan_order_flag may be encoded based on at least one of the above-mentioned higher-level syntaxes and the split_qt_flag for the current block.

[0172] For example, if split_qt_flag is true and sps_adaptive_scan_order_flag is true, then cu_adaptive_scan_order_flag may be encoded into a bitstream. On the other hand, if at least one of split_qt_flag and sps_adaptive_scan_order_flag is false, then cu_adaptive_scan_order_flag does not need to be encoded into a bitstream. If split_qt_flag is true and pps_adaptive_scan_order_flag is true, then cu_adaptive_scan_order_flag may be encoded into a bitstream. On the other hand, if at least one of split_qt_flag and pps_adaptive_scan_order_flag is false, then cu_adaptive_scan_order_flag does not need to be encoded into a bitstream. If split_qt_flag is true and ph_adaptive_scan_order_flag is true, then cu_adaptive_scan_order_flag may be encoded into a bitstream. On the other hand, if at least one of split_qt_flag and ph_adaptive_scan_order_flag is false, then cu_adaptive_scan_order_flag does not need to be encoded into a bitstream.

[0173] As mentioned above, the encoding process for each subblock may include a block partitioning process for that subblock. In this case, at least one of qt_scan_flag or cu_adaptive_scan_order_flag may be encoded at the node of the subblock along with the partitioning information for block partitioning. Therefore, if the current block is partitioned into multiple first subblocks, and the first subblocks are then partitioned into multiple second subblocks, the encoding order of the first subblocks within the current block may differ from the encoding order of the second subblocks within the first subblock. Here, if the partitioning depth of the current block is N, the partitioning depth of the first subblock may be (N+1), and the partitioning depth of the second subblock may be (N+2).

[0174] Currently, a picture may be divided into multiple top-level block units (or coding tree units, CTUs), and an adaptive encoding order may be applied to these multiple top-level block units. The adaptive decoding order described with reference to Figures 5 to 7 may also be applied to the adaptive encoding order, and redundant explanations are omitted here.

[0175] Figure 10 shows a schematic configuration of an encoding device 200 that performs the image encoding method according to this disclosure.

[0176] Referring to Figure 10, the encoding device 200 may include a memory 270 and at least one processor 1000.

[0177] The processor 1000 can divide the current block into multiple subblocks. The division may be based on any one of a predefined type of block division. The type of block division may include at least one of quad division, binary division, or ternary division, and binary division may include at least one of symmetric binary division or asymmetric binary division. The processor 1000 can encode division information to indicate any one of the types of block division. The processor 1000 can use tree-based block division.

[0178] The processor 1000 can encode multiple lower blocks based on a predetermined encoding order. The encoding order of the lower blocks within the current block may be determined based on one or more predefined scan order candidates, as explained with reference to Figure 9.

[0179] The processor 1000 can determine whether or not to use adaptive scan order and, based on that decision, can encode high-level syntax. For example, the high-level syntax may be encoded at at least one level of SPS, PPS, or PH, as illustrated with reference to Figure 4. However, it is not limited to this, and the high-level syntax may also be encoded at at least one level of Tile, Slice, or CTU row / column.

[0180] The processor 1000 can encode index information (qt_scan_flag) indicating one of the aforementioned one or more scan order candidates into a bitstream based on at least one of the above-mentioned higher-level syntaxes. Alternatively, the processor 1000 can encode qt_scan_flag into a bitstream based on split_qt_flag, which indicates whether the current block is to be split based on quad partitioning. Alternatively, the processor 1000 can encode qt_scan_flag into a bitstream based on at least one of the above-mentioned higher-level syntaxes and split_qt_flag for the current block.

[0181] The processor 1000 can further encode a syntax (cu_adaptive_scan_order_flag) into the bitstream indicating whether or not an adaptive scan order is used in the coding unit, as explained with reference to Figure 9.

[0182] The processor 1000 can now divide the picture into multiple top-level block units (or coding tree units, CTUs) and encode these multiple top-level block units based on an adaptive encoding order. The adaptive decoding order described with reference to Figures 5 to 7 may also be applied to the adaptive encoding order, and redundant explanations are omitted here.

[0183] In the embodiments described above, the method is explained based on a sequence diagram in a series of steps or blocks, but the embodiments are not limited to the order of the steps, and some steps may occur in a different order or simultaneously with other steps than those described above. Furthermore, those skilled in the art will understand that the steps shown in the sequence diagram are not exclusive, and other steps may be included, or one or more steps in the sequence diagram may be omitted without affecting the scope of the embodiments described herein.

[0184] The methods relating to the embodiments of this document described above may be implemented in software form, and the encoding and / or decoding devices relating to this document may be included in image processing devices such as TVs, computers, smartphones, set-top boxes, and display devices.

[0185] When the embodiments described in this document are implemented as software, the methods described above may be implemented as modules (processes, functions, etc.) that perform the functions described above. The modules may be stored in memory and executed by a processor. The memory may be located inside or outside the processor and may be connected to the processor by various well-known means. The processor may include an ASIC (application-specific integrated circuit), other chipsets, logic circuits, and / or data processing devices. The memory may include ROM (read-only memory), RAM (random access memory), flash memory, memory cards, storage media, and / or other storage devices. In other words, the embodiments described in this document may be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional units shown in each figure may be implemented and executed on a computer, processor, microprocessor, controller, or chip. In this case, information on instructions or algorithms for implementation may be stored on a digital storage medium.

[0186] Furthermore, the decoding and encoding devices to which the embodiments of this specification apply may include multimedia broadcasting transceivers, mobile communication terminals, home cinema video equipment, digital cinema video equipment, surveillance cameras, video conferencing equipment, real-time communication equipment such as video communication, mobile streaming equipment, storage media, camcorders, video-on-demand (VoD) service providers, over-the-top (OTT) video equipment, internet streaming service providers, 3D video equipment, virtual reality (VR) equipment, argumentative reality (AR) equipment, image-phone video equipment, transportation terminals (e.g., vehicle terminals (including autonomous vehicles), airplane terminals, ship terminals, etc.), and medical video equipment, and may be used to process video signals or data signals. For example, over-the-top (OTT) video equipment may include game consoles, Blu-ray players, internet-connected TVs, home theater systems, smartphones, tablet PCs, and digital video recorders (DVRs).

[0187] Furthermore, the processing methods to which the embodiments of this specification apply may be produced in the form of programs executed on a computer and stored on a computer-readable recording medium. Similarly, multimedia data having the data structures according to the embodiments of this specification may also be stored on a computer-readable recording medium. The computer-readable recording medium includes all kinds of storage devices and distributed storage devices on which computer-readable data is stored. The computer-readable recording medium may include, for example, Blu-ray discs (BDs), general-purpose serial buses (USBs), ROMs, PROMs, EPROMs, EEPROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. The computer-readable recording medium also includes media embodied in the form of carrier waves (e.g., transmission over the Internet). Furthermore, a bitstream generated by an encoding method may be stored on a computer-readable recording medium or transmitted over a wireless communication network.

[0188] Furthermore, the embodiments of this specification may be embodied as computer program products in the form of program code, and the program code may be executed on a computer according to the embodiments of this specification. The program code may be stored on a computer-readable carrier.

[0189] Figure 11 shows an example of a content streaming system to which the embodiments of this disclosure can be applied.

[0190] Referring to Figure 11, the content streaming system to which the embodiments of this specification apply may broadly include an encoding server, a streaming server, a web server, media storage, user equipment, and multimedia input devices.

[0191] The encoding server is responsible for compressing content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data to generate a bitstream, and transmitting this bitstream to the streaming server. As another example, if a multimedia input device such as a smartphone, camera, or camcorder directly generates the bitstream, the encoding server may be omitted.

[0192] The bitstream may be generated by an encoding method or bitstream generation method to which any embodiment of this specification is applied, and the streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0193] The streaming server transmits multimedia data to the user's device based on a user request via a web server, and the web server acts as an intermediary to inform the user about available services. When a user requests a desired service from the web server, the web server transmits it to the streaming server, and the streaming server transmits the multimedia data to the user. In this case, the content streaming system may include a separate control server, in which case the control server is responsible for controlling the commands and responses between the devices in the content streaming system.

[0194] The streaming server can receive content from media storage and / or encoding servers. For example, when receiving content from the encoding server, the content can be received in real time. In this case, in order to provide a smooth streaming service, the streaming server can store the bitstream for a certain period of time.

[0195] Examples of user devices include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (such as smartwatches, smart glasses, and HMDs), digital TVs, desktop computers, and digital signage.

[0196] Each server in the aforementioned content streaming system may be operated as a distributed server, in which case the data received by each server may be processed in a distributed manner.

[0197] The claims described herein may be combined in various ways. For example, the technical features of the method claims herein may be combined to be embodied as an apparatus, and the technical features of the apparatus claims herein may be combined to be embodied as a method. Furthermore, the technical features of the method claims and the technical features of the apparatus claims herein may be combined to be embodied as an apparatus, and the technical features of the method claims and the technical features of the apparatus claims herein may be combined to be embodied as a method.

[0198] [Claims when filing an international application] [Claim 1] It is a method, The process involves dividing the current block into multiple subblocks based on one of the predefined block partitioning types; The process includes the step of decoding the plurality of subblocks based on a predetermined decoding order; The decoding order is determined based on one or more candidate scan order options. A method wherein the one or more candidate scan sequence includes at least one of a horizontal Z scan sequence or a vertical Z scan sequence. [Claim 2] The method according to claim 1, wherein the decoding order is determined based on index information indicating any one of the one or more scan order candidates. [Claim 3] The method according to claim 2, wherein the index information is signaled based on information indicating whether or not the current block is divided into four subblocks. [Claim 4] The aforementioned method, The method according to claim 3, further comprising the step of obtaining from the bitstream a higher-level syntax indicating whether or not an adaptive scan order is used. [Claim 5] The method according to claim 4, wherein the higher-level syntax is signaled by at least one of a sequence parameter set, a picture parameter set, or a picture header. [Claim 6] The method according to claim 5, wherein the index information is signaled based on the higher-level syntax. [Claim 7] The current block, including the aforementioned current block, is divided into multiple top-level block units. The method according to claim 1, wherein the plurality of top-level block units are decoded based on a decoding order which is a combination of the raster scan order and the Z scan order. [Claim 8] It is a method, The process involves dividing the current block into multiple subblocks based on one of the predefined block partitioning types; The process includes the step of encoding the plurality of lower blocks based on a predetermined encoding order; The encoding order is determined based on one or more candidate scan order options. A method wherein the one or more candidate scan sequence includes at least one of a horizontal Z scan sequence or a vertical Z scan sequence. [Claim 9] A computer-readable storage medium, A computer-readable storage medium for storing a bitstream generated by the method described in claim 8. [Claim 10] It is a method, The steps of obtaining a bitstream for image information; and The aforementioned bitstream is The current block is divided into multiple subblocks based on one of the predefined block partitioning types, and It is generated based on encoding the plurality of lower blocks according to a predetermined encoding order, The step of transmitting data including the bitstream; The encoding order is determined based on one or more candidate scan order options. A method wherein the one or more candidate scan sequence includes at least one of a horizontal Z scan sequence or a vertical Z scan sequence.

Claims

1. It is a method, The process involves dividing the current block into multiple subblocks based on one of the predefined block partitioning types; The process includes: a step of decoding the plurality of subblocks based on a predetermined decoding order; The decoding order is determined based on one or more candidate scan order options. The method wherein the one or more candidate scan sequence includes at least one of a horizontal Z scan sequence or a vertical Z scan sequence.

2. The method according to claim 1, wherein the decoding order is determined based on index information indicating any one of the one or more scan order candidates.

3. The method according to claim 2, wherein the index information is signaled based on information indicating whether or not the current block is divided into four subblocks.

4. The aforementioned method, The method according to claim 3, further comprising the step of obtaining from the bitstream a higher-level syntax indicating whether or not an adaptive scan order is used.

5. The method according to claim 4, wherein the higher-level syntax is signaled by at least one of the sequence parameter set, picture parameter set, or picture header.

6. The method according to claim 5, wherein the index information is signaled based on the higher-level syntax.

7. The current block, including the aforementioned current block, is divided into multiple top-level block units. The method according to claim 1, wherein the plurality of top-level block units are decoded based on a decoding order which is a combination of the raster scan order and the Z scan order.

8. It is a method, The process involves dividing the current block into multiple subblocks based on one of the predefined block partitioning types; The process includes: a step of encoding the plurality of lower blocks based on a predetermined encoding order; The encoding order is determined based on one or more candidate scan order options. The method wherein the one or more candidate scan sequence includes at least one of a horizontal Z scan sequence or a vertical Z scan sequence.

9. A computer-readable storage medium, A computer-readable storage medium for storing a bitstream generated by the method described in claim 8.

10. It is a method, The steps of obtaining a bitstream for image information; and The aforementioned bitstream is The current block is divided into multiple subblocks based on one of the predefined block partitioning types, and It is generated by encoding the plurality of lower blocks based on a predetermined encoding order, The step of transmitting data including the bitstream is included; The encoding order is determined based on one or more candidate scan order options. The method wherein the one or more candidate scan sequence includes at least one of a horizontal Z scan sequence or a vertical Z scan sequence.

Images