Image coding method and apparatus therefor based on block partitioning

Abstract

An image decoding method according to an embodiment of the present disclosure comprises the steps of: acquiring partition information through a bitstream; deriving a partitioning structure of a coding block on the basis of the partition information; deriving a current coding block on the basis of the partitioning structure; and performing a decoding procedure for the current coding block, wherein the partition information indicates one of a plurality of partitioning candidates on the basis of a block size.

Description

Block partitioning-based image coding method and apparatus

[0001] The present disclosure relates to an image / video coding method and an apparatus thereof.

[0002] As the utilization of multimedia data increases, the need for efficient video compression technology is growing. Video compression technology is essential for the efficient transmission of high-quality video data within limited network bandwidth, and to this end, various video codec technologies have been developed.

[0003] Video codec technologies include MPEG-2, H.264 / AVC, H.265 / HEVC, H.266 / VVC, AV1 (AOMedia video 1), and are expected to be widely used in various application fields such as internet-based video streaming, video calls, virtual reality (VR), and augmented reality (AR).

[0004] As video resolution and quality increase, the data size grows, and consequently, the amount of information or bits transmitted rises. Consequently, transmitting video data using existing wired or wireless broadband lines or storing it on conventional storage media leads to increased transmission and storage costs. Therefore, there is a growing need for subsequent codec technologies, such as H.267 and AV2, to provide improved compression efficiency and video quality. In other words, high-efficiency video compression technology is required to effectively compress, transmit, store, and play back high-resolution, high-quality video information.

[0005] According to one embodiment of the present disclosure, a method and apparatus for increasing video / image coding efficiency are provided.

[0006] According to one embodiment of the present disclosure, a method and apparatus for providing next-generation high-definition video services are provided.

[0007] According to one embodiment of the present disclosure, a method and apparatus for providing improved performance in a real-time streaming environment are provided.

[0008] According to one embodiment of the present disclosure, a block partitioning method and apparatus for video / image coding are provided.

[0009] According to one embodiment of the present disclosure, an image decoding method is provided by a decoding device. The method comprises the steps of obtaining partition information through a bitstream, deriving a partitioning structure of a coding block based on the partition information, deriving a current coding block based on the partitioning structure, and performing a decoding procedure for the current coding block, wherein the partition information indicates one of a plurality of partitioning candidates based on a block size.

[0010] According to one embodiment of the present disclosure, an image encoding method is provided by an encoding device. The method comprises the steps of deriving a partitioning structure of a coding block, deriving a current coding block based on the partitioning structure, generating partition information indicating the partitioning structure, and encoding image information including the partition information to generate a bitstream, wherein the partition information indicates one of a plurality of partitioning candidates based on a block size.

[0011] According to one embodiment of the present disclosure, a decoding device for image decoding is provided. The decoding device comprises a memory and at least one processor connected to the memory, wherein the at least one processor is configured to perform the steps of acquiring partition information through a bitstream, deriving a partitioning structure of a coding block based on the partition information, deriving a current coding block based on the partitioning structure, and performing a decoding procedure for the current coding block, wherein the partition information indicates one of a plurality of partitioning candidates based on a block size.

[0012] According to one embodiment of the present disclosure, an encoding device for video encoding is provided. The encoding device comprises a memory and at least one processor connected to the memory, wherein the at least one processor is configured to perform the steps of deriving a partitioning structure of a coding block, deriving a current coding block based on the partitioning structure, generating partition information indicating the partitioning structure, and encoding video information including the partition information to generate a bitstream, wherein the partition information indicates one of a plurality of partitioning candidates based on a block size.

[0013] According to one embodiment of the present disclosure, a method for storing or transmitting video / image data including a bitstream generated according to a video / image encoding method according to at least one of the embodiments of the present disclosure is provided.

[0014] According to one embodiment of the present disclosure, an apparatus for storing or transmitting video / image data including a bitstream generated according to a video / image encoding method according to at least one of the embodiments of the present disclosure is provided.

[0015] According to one embodiment of the present disclosure, a computer-readable storage medium may be provided that stores a program for performing a method according to at least one of the embodiments of the present disclosure.

[0016] According to one embodiment of the present disclosure, a computer-readable digital storage medium is provided that stores encoded video / image information generated according to a video / image encoding method according to at least one of the embodiments of the present disclosure.

[0017] According to one embodiment of the present disclosure, a computer-readable digital storage medium is provided that stores encoded information or encoded video / image information, which causes a video / image decoding method according to at least one of the embodiments of the present disclosure to be performed by a decoding device.

[0018] According to one embodiment of the present disclosure, overall video / image compression efficiency can be increased.

[0019] According to one embodiment of the present disclosure, various partitioning structures of coding blocks can be efficiently signaled.

[0020] According to one embodiment of the present disclosure, partitioning information can be efficiently signaled by considering block size and image characteristics.

[0021] According to one embodiment of the present disclosure, a partition structure can be efficiently derived for coding blocks that extend beyond the boundaries of a frame without signaling partition information.

[0022] According to one embodiment of the present disclosure, an optimal coding block size according to frame characteristics can be efficiently signaled through hierarchical semi-independent partition structure signaling considering the frame type.

[0023] FIG. 1 schematically illustrates an example of a video / image coding system to which embodiments of the present disclosure may be applied.

[0024] FIG. 2 is a diagram schematically illustrating the configuration of a video / image encoding device to which embodiments of the present disclosure can be applied.

[0025] FIG. 3 is a diagram schematically illustrating the configuration of a video / image decoding device to which embodiments of the present disclosure can be applied.

[0026] FIG. 4 shows an example of a partitioning structure according to one embodiment of the present disclosure.

[0027] FIG. 5 shows an example of a partitioning structure considering recursive partitioning according to one embodiment of the present disclosure.

[0028] FIG. 6 shows examples of partitioning candidates proposed in the present disclosure.

[0029] FIG. 7 shows examples of partitioning candidates proposed in the present disclosure.

[0030] FIG. 8 shows examples of partitioning candidates proposed in the present disclosure.

[0031] FIG. 9 shows an example of a partitioning structure according to another embodiment of the present disclosure.

[0032] Figure 10 illustrates an exemplary intra-prediction procedure.

[0033] Figure 11 shows an example of peripheral reference samples for intra-prediction.

[0034] FIG. 12 illustrates the directional intra prediction modes described in the present disclosure as an example.

[0035] Figure 13 exemplarily illustrates directional intra prediction modes extended based on angle delta values.

[0036] Figure 14 shows an example of peripheral reference samples of multiple reference lines for intra prediction.

[0037] Figure 15 shows an example of multiple line surrounding reference samples for intra prediction.

[0038] Figure 16 shows an example of the configuration of surrounding reference samples when the left line number and the upper line number are determined differently.

[0039] Figure 17 shows an example of generating a final prediction block based on the weighted sum of two prediction blocks based on multiple reference lines according to an intra prediction mode.

[0040] Figure 18 shows an example of intra-predicted sample generation through 2D interpolation.

[0041] Figure 19 shows an example of cross-line filtering for reference samples.

[0042] Figure 20 shows an example where a coding block extends beyond the boundaries of a frame.

[0043] Figures 21 and 22 show examples of division where the coding block extends beyond the boundaries of the frame.

[0044] Figure 23 shows an example of a semi-independent partitioning structure indication.

[0045] FIG. 24 schematically illustrates a video / image encoding method according to the embodiment(s) of the present disclosure.

[0046] FIG. 25 schematically illustrates a video / image decoding method according to the embodiment(s) of the present disclosure.

[0047] As the present disclosure is subject to various modifications and may have various embodiments, specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the embodiments of the present disclosure to specific embodiments. Terms used in the present disclosure are used merely to describe specific embodiments and are not intended to limit the technical scope of the present disclosure. The singular forms used in the present disclosure are intended to include the plural forms unless the context clearly indicates otherwise. The term "and / or" used in the present disclosure includes any one or more combinations of the related listing items. The terms "comprising," "composing," and "holding" used in this specification specify the presence of the stated features, numbers, actions, elements, components, and / or combinations thereof, but do not exclude the presence or addition of one or more other features, numbers, actions, elements, components, and / or combinations thereof. In this disclosure, the use of the term “may” in relation to examples or embodiments (e.g., what an example or embodiment may include or implement) means that there exists at least one example or embodiment in which such feature is included or implemented, but not all examples are limited thereto, and such feature or configuration may be omitted.

[0048] Meanwhile, each component in the drawings described in this disclosure is depicted independently for the convenience of explaining different characteristic functions and does not imply that each component is implemented in separate hardware or separate software. For example, two or more components may be combined to form a single component, or a single component may be divided into multiple components. Embodiments in which each component is integrated and / or separated are also included within the scope of this disclosure as long as they do not deviate from the essence of this disclosure.

[0049] In the present disclosure, "A or B" may mean "only A," "only B," or "both A and B." Alternatively, in the present disclosure, "A or B" may be interpreted as "A and / or B." For example, in the present disclosure, "A, B or C" may mean "only A," "only B," "only C," or "any combination of A, B and C."

[0050] A slash ( / ) or a comma used in the present disclosure may mean "and / or." For example, "A / B" may mean "A and / or B." Accordingly, "A / B" may mean "only A," "only B," or "both A and B." For example, "A, B, C" may mean "A, B or C."

[0051] In the present disclosure, "at least one of A and B" may mean "only A," "only B," or "both A and B." Additionally, in the present disclosure, the expressions "at least one of A or B" or "at least one of A and / or B" may be interpreted as synonymous with "at least one of A and B."

[0052] Additionally, in the present disclosure, "at least one of A, B and C" may mean "only A," "only B," "only C," or "any combination of A, B and C." Additionally, "at least one of A, B or C" or "at least one of A, B and / or C" may mean "at least one of A, B and C."

[0053] Additionally, parentheses used in the present disclosure may mean "for example." Specifically, when indicated as "prediction (intra-prediction)," "intra-prediction" may be proposed as an example of "prediction." In other words, the "prediction" of the present disclosure is not limited to "intra-prediction," and "intra-prediction" may be proposed as an example of "prediction." Furthermore, even when indicated as "prediction (i.e., intra-prediction)," "intra-prediction" may be proposed as an example of "prediction."

[0054] Technical features described individually within one drawing in this disclosure may be implemented individually or simultaneously.

[0055] The present disclosure relates to video / image coding. For example, the methods / exemplars described in the present disclosure may be applied to methods disclosed in the AV2 (AOMedia Video 2) standard. Additionally, the methods / exemplars disclosed in this disclosure may be applied to methods disclosed in the ECM (enhanced compression model) or H.267 standard, or next-generation video / image coding standards (e.g., H.268, H.269, etc.).

[0056] In the present disclosure, coding may include encoding and / or decoding. In the present disclosure, image coding may be used interchangeably with video coding.

[0057] In the present disclosure, "video" may refer to a set of a series of images over time. "Frame" generally refers to a unit representing a single image at a specific time, and "slice" or "tile" is a unit that constitutes a part of a frame in coding. A slice or tile may include one or more superblocks. A single frame may be composed of one or more slices or tiles. A tile may represent a rectangular area of ​​superblocks within a specific tile row or a specific tile column within a frame. Meanwhile, a single frame may be divided into two or more subframes.

[0058] A pixel or pel can refer to the smallest unit that constitutes a single frame (or image). Additionally, the term 'sample' may be used as a counterpart to pixel. Generally, a sample can represent a pixel or its value, and it may represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chroma component.

[0059] A unit may represent a basic unit of image processing. A unit may include at least one of a specific region of a frame and information related to that region. A unit may include one luminance block and two chroma (e.g., cb, cr) blocks. Depending on the case, the term unit may be used interchangeably with terms such as block or area. In general, an MxN block may include samples (or sample arrays) or a set (or array) of transform coefficients consisting of M columns and N rows.

[0060] Hereinafter, embodiments of the present disclosure will be described in more detail with reference to the attached drawings. Hereinafter, the same reference numerals may be used for identical components in the drawings, and redundant descriptions of identical components may be omitted.

[0061] FIG. 1 schematically illustrates an example of a video / image coding system to which embodiments of the present disclosure may be applied.

[0062] Referring to FIG. 1, a video / image coding system may include a first device (encoding device) and a second device (decoding device). The first device may transmit encoded video / image information or data to the second device in the form of a file or streaming via a digital storage medium or a network.

[0063] The video / image coding system may further include a video / image acquisition device and a video / image renderer. The video / image acquisition device may be included in the encoding device, or it may be configured as a separate device or an external component. The video / image renderer may be included in the decoding device, or it may be configured as a separate device or an external component.

[0064] The first device may include a transmission unit as an internal component, or it may include a separate device or an external component.

[0065] The second device may include a receiver as an internal component, or it may include a separate device or an external component.

[0066] The encoder may be called an encoding device, and the decoder may be called a decoding device. The transmission unit may be included in the encoding device. The reception unit may be included in the decoding device. The renderer may include a display unit, and the display unit may be composed of a separate device or an external component.

[0067] The decoding device and encoding device to which the embodiment(s) of the present disclosure are applied may be included in multimedia broadcasting transmission and reception devices, mobile communication terminals, home cinema video devices, digital cinema video devices, surveillance cameras, video conversation devices, real-time communication devices such as video communication, mobile streaming devices, storage media, camcorders, Video on Demand (VoD) service providers, Over-the-top video (OTT) devices, internet streaming service providers, 3D video devices, virtual reality (VR) devices, augmented reality (AR) devices, video phone video devices, transportation terminals (e.g., vehicle terminals (including autonomous vehicles), airplane terminals, ship terminals, etc.), and medical video devices, and may be used to process video signals or data signals. For example, Over-the-top video (OTT) devices may include game consoles, Blu-ray players, internet-connected TVs, home theater systems, smartphones, tablet PCs, Digital Video Recorders (DVRs), etc.

[0068] A video / image acquisition device can acquire a video / image source. The video / image acquisition device can acquire video / image through processes such as video / image capture, synthesis, or generation. The video / image acquisition device may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, a video / image archive containing previously captured video / image, etc. The video / image generation device may include, for example, a camcorder, a computer, a tablet, and a smartphone, etc., and can generate video / image (electronically). For example, a virtual video / image may be generated through a computer, etc., in which case the video / image capture process may be replaced by a process of generating related data. The video / image source may perform a video / image preprocessing process to input the optimized video / image into an encoder.

[0069] An encoding device can encode an input video / image. The encoding device can encode the input video / image through the encoding method presented in this disclosure. The encoding device can perform a series of procedures, such as prediction, transformation, and quantization, for compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0070] The transmission unit can transmit encoded video / image information or data output in the form of a bitstream to the receiving unit of a receiving device via a network in the form of a file or streaming. The encoded video / image information or data output in the form of a bitstream may also be transmitted to the receiving unit through a streaming server. Digital storage media may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmission unit may include elements for creating a media file through a predetermined file format and elements for transmission through a broadcasting / communication network. The receiving unit may receive / extract the bitstream and transmit it to a decoding device. In this disclosure, the transmission unit may be referred to as a transmission device, and the receiving unit may be referred to as a receiving device. As video becomes high-resolution and high-quality, the size of the original video / image data increases, and by acquiring and storing / transmitting a bitstream (or data including a bitstream) generated through an efficient encoding method according to this disclosure, storage / transmission efficiency can be increased and low-latency / real-time transmission can be supported.

[0071] The streaming server may temporarily store the bitstream during the process of transmitting or receiving the bitstream. The streaming server transmits multimedia data to a user device based on a user request through a web server, and the web server acts as a medium to inform the user of available services. When a user requests a desired service from the web server, the web server forwards this request to the streaming server, and the streaming server transmits the multimedia data to the user. At this time, the content streaming system may include a separate control server, and in this case, the control server plays the role of controlling commands and responses between each device within the content streaming system.

[0072] The streaming server can receive content from a media storage and / or encoding device. For example, when receiving content from the encoding device, the content can be received in real time. In this case, to provide a seamless streaming service, the streaming server can store the bitstream for a certain period of time.

[0073] A decoding device can decode a video / image by performing a series of procedures, such as inverse quantization, inverse transform, and prediction, corresponding to the operation of an encoding device. The decoding device can decode a video / image through the decoding method presented in this disclosure.

[0074] The renderer can render the decoded video / image. The rendered video / image can be displayed through the display unit.

[0075] FIG. 2 is a diagram schematically illustrating the configuration of a video / image encoding device to which embodiments of the present disclosure may be applied. The term "encoding device" below may include an image encoding device and / or a video encoding device.

[0076] Referring to FIG. 2, the encoding device (200) may be configured to include an image partitioner (210), a predictor (220), a residual processor (230), an entropy encoder (240), an adder (250), a filter (260), and a memory (270). The predictor (220) may include an inter-predictor and an intra-predictor. The residual processor (230) may include a transformer (232), a quantizer (233), a dequantizer (234), and an inverse transformer (235). The residual processor (230) may further include a subtractor (231). The addition unit (250) may be referred to as a reconstructor or a reconstructed block generator. The above-described image segmentation unit (210), prediction unit (220), residual processing unit (230), entropy encoding unit (240), addition unit (250), and filtering unit (260) may be configured by one or more hardware components (e.g., an encoder chipset or processor) according to the embodiment. Additionally, the memory (270) may include a frame buffer and may be configured by a digital storage medium. The hardware component may further include the memory (270) as an internal / external component.

[0077] The image segmentation unit (210) can divide an input image (or picture, frame) input to the encoding device (200) into one or more processing units. For example, the processing unit may include a superblock or a coding block. A single frame may be divided into a plurality of tiles. The tiles may be rectangular in shape. Uniform or non-uniform tile sizes may be determined on a frame-by-frame basis. In this disclosure, the terms frame and picture may be used interchangeably. A tile may consist of an integer number of superblocks. Superblocks within a tile may be coded in raster scan order. A superblock may be divided into one or more coding blocks. For example, based on the luminance component, the size of the superblock may be 128x128 or 64x64. Alternatively, based on the luminance component, the size of the superblock may include 256x256. A superblock may have dependencies only on specific surrounding superblocks. For example, a superblock may have dependencies only on the surrounding superblocks to the left and / or above.

[0078] A superblock can be recursively partitioned. For example, a superblock can be derived into a single coding block, or the superblock can be partitioned into coding blocks based on a binary tree, a terminal tree, or a quad tree. A single coding block can be recursively partitioned into multiple coding blocks of a deeper depth based on a binary tree, a terminal tree, or a quad tree. For example, recursive partitioning may be possible when a coding block is partitioned into PARTITION_NxN. A coding procedure according to the present disclosure may be performed based on a final coding block that is no longer partitioned. In this case, based on coding efficiency according to image characteristics, the superblock may be used directly as the final coding block, or, if necessary, the coding block may be recursively partitioned into coding blocks of a deeper depth so that a coding block of the optimal size is used as the final coding block. Here, the term "coding procedure" may include procedures such as prediction, transformation, and restoration described below. Meanwhile, the processing unit may further include a prediction block (PB) or a transform block (TB). In this case, the prediction block or the transform block may be divided or partitioned from the final coding block described above. For example, a single transform block of the same size as the coding block may be derived, or multiple transform blocks may be derived from the coding block based on a quad tree or binary tree. The transform block may also be recursively divided into transform blocks of a deeper depth. The prediction block may be a unit for prediction (or for deriving a prediction mode), and the transform block may be a unit for performing a transformation, a unit for deriving transformation coefficients, and / or a unit for deriving a residual signal from transformation coefficients.For example, the derivation of a prediction mode for intra-prediction can be performed at the level of the coding block or the prediction block, and the derivation of a prediction sample through the intra-prediction procedure can be performed at the level of a transformation block. Depending on the coding order, the block currently subject to processing may be called the current block.

[0079] The term "block" may be used interchangeably with terms such as "unit" or "area" depending on the context. In general, an MxN block may represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or a pixel value, and may represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chroma component. The term "sample" may be used as a counterpart to a pixel or pel of a frame (or image).

[0080] The encoding device (200) can generate a residual signal (residual block, residual sample array) by subtracting the predicted signal (predicted block, predicted sample array) output from the prediction unit from the input image signal (original block, original sample array), and the generated residual signal is transmitted to the conversion unit (232). In this case, as illustrated, the configuration for subtracting the predicted signal (predicted block, predicted sample array) from the input image signal (original block, original sample array) within the encoder (200) may be called a subtraction unit (231). The prediction unit performs a prediction for a block to be processed (hereinafter referred to as the current block) and can generate a predicted block containing prediction samples for the current block. The prediction unit can determine whether intra prediction is applied or inter prediction is applied on a current block or coding block basis. The prediction unit can generate various information regarding prediction, such as prediction mode information, as described below in the description of each prediction mode, and transmit it to the entropy encoding unit (240). The information regarding prediction can be encoded in the entropy encoding unit (240) and output in the form of a bitstream.

[0081] The intra prediction unit can predict the current block by referencing samples within the current frame. Depending on the prediction mode, the referenced samples may be located next to the current block or apart from it. Intra prediction can be performed on a per-transform block basis. If multiple transform blocks exist within a coding block, intra prediction can be performed sequentially in the raster order of the transform blocks. In this case, the procedure for deriving neighboring reference samples for intra prediction can be performed based on the transform block. Multiple prediction modes may be considered for intra prediction. These prediction modes may include multiple non-directional modes and multiple directional modes. The prediction mode used in the current block may be signaled from the encoding device to the decoding device; for example, the prediction modes may include a DC intra prediction mode, multiple directional intra prediction modes, multiple SMOOTH intra prediction modes, and / or PAETH intra prediction modes. These prediction modes may include an intra block copy (intrabc) mode. Whether the above-mentioned intra-block copy mode is applied can be signaled separately. Directional prediction modes may include, for example, eight or more prediction modes depending on the prediction direction. However, this is merely an example, and depending on the settings, more or fewer directional prediction modes may be used. The intra-prediction unit may determine the prediction mode applied to the current block based on the prediction mode applied to surrounding blocks. In the intra-block copy mode, a reference block is derived based on a vector, similar to the inter-prediction mode described later, and the current frame is used as the reference frame. The vector used to derive the reference block in the above-mentioned intra-block copy may be called a block vector.

[0082] The inter-prediction unit can derive a predicted block for the current block based on a reference block (reference sample array) identified by a motion vector on a reference frame. In this case, to reduce the amount of motion information transmitted in the inter-prediction mode, motion information can be predicted in block, sub-block, or sample units based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and / or a reference frame index. The motion information may further include information on the inter-prediction direction (L0 prediction, L1 prediction, compound prediction, etc.). In the case of inter-prediction, neighboring blocks may include spatial neighboring blocks existing within the current frame and temporal neighboring blocks existing in the reference frame. The reference frame containing the reference blocks and the reference frame containing the temporal neighboring blocks may be the same or different. The above temporal surrounding blocks may be referred to by names such as co-located reference blocks or colblocks, and a reference frame containing the above temporal surrounding blocks may be referred to as a co-located frame or colframe. For example, the inter-prediction unit may construct a motion information stack based on the surrounding blocks and generate information indicating which candidate is used to derive the motion vector and / or reference frame index of the current block. The motion information stack may be referred to as a motion information list. The motion information stack may include a motion vector stack. The motion vector stack may be referred to as RefStackMv. The motion vector stack may include eight or more candidates.Information regarding the (maximum) number of candidates for the above motion vector stack can be signaled on a frame or sequence basis. Motion modes may be further considered for inter-prediction. The above motion modes may include simple mode, OBMC (overlapped block motion compensation) mode and / or local warp mode. In OBMC mode, prediction performance can be improved by utilizing motion information of surrounding blocks for the left and / or upper boundaries of the current block, and in local warp mode, an affine model may be applied in addition to translational motion compensation.

[0083] Motion vectors can be derived based on various prediction modes; for example, in NEWMV mode, the motion vector of the current block can be indicated based on the motion vector of the reference stack and the motion vector difference. In some cases, in NEWMV mode, the motion vector of the current block can be indicated based on the motion vector difference without the motion vector of the reference stack. Information regarding the motion vector difference can be generated by an encoding device and signaled to a decoding device. ZEROMV mode may indicate that a zero vector or a default vector is used as the motion vector of the current block. REFMV mode may indicate that the motion vector of the motion information stack is used as the motion vector of the current block. However, these names are merely examples, and the motion vector of the current block may be indicated by various other names. For example, GLOBALMV mode may indicate that global motion information is used for the current block. For example, NEARSTMV mode may indicate that the first candidate (candidate index 0) of the motion information stack is used for the current block. For example, the NEARMV mode may indicate that a specific candidate of the motion information stack (a candidate indicated by refMVidx) is used in the current block. However, the above mode names are examples, and depending on the case, other names such as the first mode, second mode, etc. may be used in this disclosure.

[0084] When compound prediction is applied, inter-prediction can be performed using both reference frame lists L0 and L1. In this case, a motion vector for the L0 direction and a motion vector for the L1 direction can be derived, respectively. Additionally, an L0 reference frame for the L0 direction and an L1 reference frame for the L1 direction can be derived, respectively. In some cases, a first motion vector and a second motion vector may be used instead of the L0 motion vector and the L1 motion vector. In some cases, a first reference frame and a second reference frame may be used instead of the L0 reference frame and the L1 reference frame.

[0085] The prediction unit (220) can generate a prediction signal based on various prediction methods. For example, the prediction unit may apply intra prediction or inter prediction for the prediction of a single block, and may also apply intra prediction and inter prediction simultaneously. This may be called compound inter-intra prediction. In this case, the mode used as the intra prediction mode may include DC prediction mode, vertical prediction mode, horizontal prediction mode, and SMOOTH prediction mode. Additionally, the prediction unit may be based on the intra block copy mode described above or on the palette mode for the prediction of a block. As described above, the intra block copy mode is a type of intra prediction that basically performs prediction within the current frame, but can be performed similarly to inter prediction in that it derives a reference block based on a vector using the current frame as a reference frame. However, since the current frame is used as a reference frame, the term block vector may be used instead of motion vector for the vector for movement. That is, the intra-block copy mode may utilize at least one of the inter-prediction techniques described in this disclosure. In this case, a motion vector derived through a surrounding block or a motion information stack may be referenced to derive the block vector of the current block. For example, when the intra-block copy mode is applied, the NEWMV mode described above may be used to derive the block vector of the current block.

[0086] The prediction signal generated through the prediction unit (220) can be used to generate a restoration signal or to generate a residual signal. The transformation unit (232) can generate transform coefficients by applying a transformation technique to the residual signal. For example, the transformation technique may include at least one of DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT (Karhunen-Loeve Transform), GBT (Graph-Based Transform), or CNT (Conditionally Non-linear Transform). Additionally, for example, the transformation technique may include DCT, ADST (Asymmetric Discrete Sine Transform), FLIPADST (Flipped ADST), IDTX (Identity Transform), WHT (Walsh-Hadamard Transform), V_DCT, H_DCT, etc. DCT is one of the most widely used transformation techniques in image compression and primarily serves to convert the image signal into the frequency domain to concentrate energy on low-frequency components. ADST is similar to DCT but is a transformation method designed to ensure smooth signal connection at block boundaries. FLIPADST is a variation of ADST that applies the transformation direction in reverse, allowing for more efficient signal compression in specific block patterns. IDTX is an identity transformation method that preserves the signal without transformation. Identity transformation can be applied to only one of the vertical or horizontal transformations, or to both. For example, if a transformation method is specified for only one of the vertical or horizontal transformations, identity transformation may be implicitly applied to the transformation in the other direction.WHT is a linear transformation similar to the Discrete Fourier Transform (DFT) or Discrete Cosine Transform (DCT) that represents signals or data by transforming them into different bases. WHT does not use trigonometric functions (sine and cosine); instead, it can use orthogonal basis matrices composed of +1 and -1. V_DCT and H_DCT indicate that the DCT transformation is performed on the vertical direction of the block and the horizontal direction of the block, respectively. For example, DCT can be used to increase compression ratios in simple blocks, ADST or FLIPADST can be used when smooth connections are required at boundaries, and IDTX can be applied to blocks where the signal remains almost unchanged.

[0087] The quantization unit (233) quantizes the transformation coefficients and transmits them to the entropy encoding unit (240), and the entropy encoding unit (240) can encode the quantized signal (information regarding the quantized transformation coefficients) and output it as a bitstream. The information regarding the quantized transformation coefficients may be called residual information. The quantization unit (233) can rearrange the block-shaped quantized transformation coefficients into a one-dimensional vector form based on the coefficient scan order, and can also generate information regarding the quantized transformation coefficients based on the one-dimensional vector-shaped quantized transformation coefficients. The entropy encoding unit (240) can perform various encoding methods, such as, for example, CDF (Cumulative Distribution Function) and CABAC (Context-Adaptive Binary Arithmetic Coding). CDF is a method of storing the cumulative value of a probability distribution, and using CDF allows a symbol to be represented with fewer bits than directly storing the probability. In CDF, the probability of a symbol can be adaptively adjusted based on the context. The entropy encoding unit (240) may encode information necessary for video / image restoration (e.g., values ​​of syntax elements) together or separately, in addition to the quantized transform coefficients. The encoded information (e.g., encoded video / image information) may be packetized into open bitstream units (OBU) in the form of a bitstream and transmitted or stored. The video / image information may further include information commonly applied to a certain range, such as tile headers, frame headers, and sequence headers. In this disclosure, information and / or syntax elements transmitted / signaled from the encoding device to the decoding device may be included in the video / image information. The video / image information may be encoded through the encoding procedure described above and included in the bitstream.The above bitstream may be transmitted via a network or stored in a digital storage medium. Here, the network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmission unit (not shown) that transmits the signal output from the entropy encoding unit (240) and / or a storage unit (not shown) that stores it may be configured as internal / external elements of the encoding device (200), or the transmission unit may be included in the entropy encoding unit (240).

[0088] Quantized transformation coefficients output from the quantization unit (233) can be used to generate a prediction signal. For example, a residual signal (residual block or residual samples) can be restored by applying inverse quantization and inverse transformation to the quantized transformation coefficients through the inverse quantization unit (234) and the inverse transformation unit (235). The addition unit (250) can generate a reconstructed signal (reconstructed frame, reconstructed block, reconstructed sample array) by adding the restored residual signal to the prediction signal output from the prediction unit. In cases where there is no residual for the block to be processed, such as when a skip mode is applied, the predicted block can be used as the reconstructed block. The addition unit (250) may be called a reconstructed unit or a reconstructed block generation unit. The generated reconstructed signal can be used for intra prediction of the next block to be processed within the current frame, and can also be used for inter prediction of the next frame after undergoing filtering as described below.

[0089] The filtering unit (260) can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit (260) can generate a modified restored frame by applying various filtering methods to the restored frame, and can store the modified restored frame in memory (270), specifically in the frame buffer of memory (270). The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filtering unit (260) can generate information regarding filtering and transmit it to the entropy encoding unit (240). The information regarding filtering can be encoded in the entropy encoding unit (240) and output in the form of a bitstream.

[0090] The modified restored frame transmitted to the memory (270) can be used as a reference frame in the inter-prediction unit. Through this, when inter-prediction is applied, the encoding device can avoid prediction mismatches between the encoding device (200) and the decoding device, and can also improve encoding efficiency.

[0091] Memory (270) can store a modified restored frame to be used as a reference frame in the inter-prediction unit. Memory (270) can store motion information of blocks from which motion information is derived (or encoded) within the current frame and / or motion information of blocks within the already restored frame. The stored motion information can be transmitted to the inter-prediction unit to be used as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. Memory (270) can store restoration samples of blocks restored within the current frame and transmit them to the intra-prediction unit.

[0092] FIG. 3 is a diagram schematically illustrating the configuration of a video / image decoding device to which embodiments of the present disclosure may be applied. The term "decoding device" below may include an image decoding device and / or a video decoding device.

[0093] Referring to FIG. 3, the decoding device (300) may be configured to include an entropy decoder (310), a residual processor (320), a predictor (330), an adder (340), a filter (350), and a memory (360). The predictor (330) may include an inter-predictor and an intra-predictor. The residual processor (320) may include a dequantizer (321) and an inverse transformer (322). The aforementioned entropy decoding unit (310), residual processing unit (320), prediction unit (330), addition unit (340), and filtering unit (350) may be configured by a single hardware component (e.g., a decoder chipset or a processor) according to an embodiment. Additionally, the memory (360) may include a frame buffer and may be configured by a digital storage medium. The hardware component may further include the memory (360) as an internal / external component.

[0094] When a bitstream containing video / image information is input, the decoding device (300) can restore the image in accordance with the process in which the video / image information is processed by the encoding device of FIG. 2. For example, the decoding device (300) can derive units / blocks based on block division information obtained from the bitstream. The decoding device (300) can perform decoding using a processing unit applied by the encoding device. Thus, the processing unit for decoding may be, for example, a super block or a coding block, and the coding block may be divided from the super block according to a quad tree structure, a binary tree structure and / or a binary tree structure, etc. One or more prediction blocks or transformation blocks may be derived from the coding unit. And, the restored image signal decoded and output through the decoding device (300) can be played back through a playback device.

[0095] The decoding device (300) can receive a signal output from the encoding device in the form of a bitstream, and the received signal can be decoded through the entropy decoding unit (310). For example, the entropy decoding unit (310) can parse the bitstream to derive information (e.g., video / image information) necessary for image restoration (or frame restoration). The video / image information may further include information commonly applied to a certain range, such as a tile header, a frame header, and a sequence header. The decoding device can decode the frame based on the header information. The signaling / received information and / or syntax elements described below in this disclosure can be obtained from the bitstream by decoding through the decoding procedure. For example, the entropy decoding unit (310) can decode information within the bitstream based on coding methods such as CDF, CABAC, etc., and output the values ​​of syntax elements necessary for image restoration and the quantized values ​​of conversion coefficients regarding residuals. More specifically, the CDF-based coding method may include a procedure for storing the probability distribution of a symbol in the form of a CDF, analyzing the context of the current block to select an appropriate CDF, encoding the symbol in the encoding stage based on the CDF, and restoring the syntax elements / information in the decoding stage using the same CDF. Additionally, the CABAC coding method may determine a context model using the target syntax elements / information, coding information of surrounding and target blocks, or information on symbols / bins coded in the previous stage; in the encoding stage, predicting the probability of bin occurrence according to the determined context model and performing arithmetic encoding of the bin to generate a bit sequence; and in the decoding stage, predicting the probability of bin occurrence according to the determined context model and performing arithmetic decoding of the bin to generate a symbol corresponding to the value of each syntax element. At this time, after determining the context model, the context model may be updated using the information on the coded symbols / bins for the context model of the next symbol / bin.Information regarding prediction among the information decoded in the entropy decoding unit (310) is provided to the prediction unit (330), and residual values, i.e., quantized transformation coefficients and related parameter information, for which entropy decoding has been performed in the entropy decoding unit (310) can be input to the residual processing unit (320). The residual processing unit (320) can derive residual signals (residual blocks, residual samples, residual sample arrays). Additionally, information regarding filtering among the information decoded in the entropy decoding unit (310) can be provided to the filtering unit (350). Meanwhile, a receiving unit (not shown) that receives a signal output from an encoding device may be further configured as an internal / external element of the decoding device (300), or the receiving unit may be a component of the entropy decoding unit (310). Meanwhile, the decoding device according to the present disclosure may be called a video / image / frame decoding device, and the decoding device may be divided into an information decoder (video / image / frame information decoder) and a sample decoder (video / image / frame sample decoder). The information decoder may include the entropy decoding unit (310), and the sample decoder may include at least one of the inverse quantization unit (321), inverse transform unit (322), adder (340), filtering unit (350), memory (360), and prediction unit (330).

[0096] In the inverse quantization unit (321), the quantized transformation coefficients can be inversely quantized to output transformation coefficients. The inverse quantization unit (321) can rearrange the quantized transformation coefficients into a two-dimensional block form. In this case, the rearrangement can be performed based on the coefficient scan order performed by the encoding device. The inverse quantization unit (321) can perform inverse quantization on the quantized transformation coefficients using quantization parameters (e.g., quantization step size information) and obtain transformation coefficients.

[0097] In the inverse conversion unit (322), the conversion coefficients are inversely converted to obtain a residual signal (residual block, residual sample array).

[0098] The prediction unit performs a prediction for the current block and can generate a predicted block containing prediction samples for the current block. Based on information regarding the prediction output from the entropy decoding unit (310), the prediction unit can determine whether an intra prediction or an inter prediction is applied to the current block and can determine a specific intra / inter prediction mode.

[0099] The prediction unit (330) can generate a prediction signal based on various prediction methods. For example, the prediction unit may apply intra prediction or inter prediction for the prediction of a single block, and may also apply intra prediction and inter prediction simultaneously. This can be called compound inter-intra prediction. In this case, the mode used as the intra prediction mode may include DC prediction mode, vertical prediction mode, horizontal prediction mode, and SMOOTH prediction mode. Additionally, the prediction unit may be based on the intra block copy mode described above or on a palette mode for the prediction of a block. As described above, the intra block copy mode is a type of intra prediction that basically performs prediction within the current frame, but can be performed similarly to inter prediction in that it derives a reference block based on a vector using the current frame as a reference frame. However, since the current frame is used as a reference frame, the term block vector may be used instead of motion vector for the vector for movement. That is, the intra-block copy mode may utilize at least one of the inter-prediction techniques described in this disclosure. In this case, a motion vector derived through a surrounding block or a motion information stack may be referenced to derive the block vector of the current block. For example, when the intra-block copy mode is applied, the NEWMV mode described above may be used to derive the block vector of the current block.

[0100] The intra prediction unit can predict the current block by referencing samples within the current frame. Depending on the prediction mode, the referenced samples may be located next to the current block or apart from it. Intra prediction can be performed on a per-transform block basis. If multiple transform blocks exist within a coding block, intra prediction can be performed sequentially in the raster order of the transform blocks. In this case, the procedure for deriving neighboring reference samples for intra prediction can be performed based on the transform block. Multiple prediction modes may be considered for intra prediction. These prediction modes may include multiple non-directional modes and multiple directional modes. The prediction mode used in the current block may be signaled from the encoding device to the decoding device; for example, the prediction modes may include a DC intra prediction mode, multiple directional intra prediction modes, multiple SMOOTH intra prediction modes, and / or PAETH intra prediction modes. These prediction modes may include an intra block copy (intrabc) mode. Whether the above-mentioned intra-block copy mode is applied can be signaled separately. Directional prediction modes may include, for example, eight or more prediction modes depending on the prediction direction. However, this is merely an example, and depending on the settings, more or fewer directional prediction modes may be used. The intra-prediction unit may determine the prediction mode applied to the current block based on the prediction mode applied to surrounding blocks. In the intra-block copy mode, a reference block is derived based on a vector, similar to the inter-prediction mode described later, and the current frame is used as the reference frame. The vector used to derive the reference block in the above-mentioned intra-block copy may be called a block vector.Intra prediction can be performed based on various prediction modes, and information regarding predictions obtainable through the bitstream may include information indicating an intra prediction mode for the current block.

[0101] The inter-prediction unit can derive a predicted block for the current block based on a reference block (reference sample array) identified by a motion vector on a reference frame. At this time, to reduce the amount of motion information transmitted in the inter-prediction mode, motion information can be predicted in blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and / or a reference frame index. The motion information may further include information on the inter-prediction direction (L0 prediction, L1 prediction, compound prediction, etc.). In the case of inter-prediction, neighboring blocks may include spatial neighboring blocks existing within the current frame and temporal neighboring blocks existing in the reference frame. The reference frame containing the reference blocks and the reference frame containing the temporal neighboring blocks may be the same or different. The temporal neighboring blocks may be referred to by names such as co-located reference blocks or co-located blocks, and the reference frame containing the temporal neighboring blocks may be referred to as co-located frames. For example, the inter-prediction unit may construct a motion information stack based on surrounding blocks and generate information indicating which candidate is used to derive the motion vector and / or reference frame index of the current block. The motion information stack may be called a motion information list. The motion information stack may include a motion vector stack. The motion vector stack may be called RefStackMv. The motion vector stack may include eight or more candidates. Information regarding the (maximum) number of candidates in the motion vector stack may be signaled on a frame or sequence basis.Motion modes may be further considered for inter prediction. The motion modes may include simple mode, OBMC (overlapped block motion compensation) mode, and / or local warp mode. In OBMC mode, prediction performance can be improved by utilizing motion information of surrounding blocks for the left and / or upper boundaries of the current block, and in local warp mode, an affine model may be applied in addition to translational motion compensation. Inter prediction can be performed based on various prediction modes, and the prediction information obtainable through the bitstream may include information indicating the mode for inter prediction regarding the current block. For example, the inter prediction unit may construct a motion information stack based on surrounding blocks and derive the motion vector and / or reference frame index of the current block based on received candidate selection information (e.g., reference motion vector index and / or reference frame index).

[0102] The adder (340) can generate a restoration signal (restoration frame, restoration block, restoration sample array) by adding the acquired residual signal to the prediction signal (predicted block, prediction sample array) output from the prediction unit (330). In cases where there is no residual for the block to be processed, such as when a skip mode is applied, the predicted block can be used as the restoration block.

[0103] The adder (340) may be called a restoration unit or a restoration block generation unit. The generated restoration signal may be used for intra prediction of the next processing target block within the current frame, may be output after filtering as described below, or may be used for inter prediction of the next frame.

[0104] The filtering unit (350) can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit (350) can generate a modified restored frame by applying various filtering methods to the restored frame, and can transmit the modified restored frame to memory (360), specifically to the frame buffer of memory (360). The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0105] The (modified) restored frame stored in memory (360) can be used as a reference frame in the inter-prediction unit. Memory (360) can store movement information of blocks from which movement information within the current frame has been derived (or decoded) and / or movement information of blocks within the already restored frame. The stored movement information can be transmitted to the inter-prediction unit to be used as movement information of spatially surrounding blocks or movement information of temporally surrounding blocks. Memory (360) can store restoration samples of blocks restored within the current frame and transmit them to the intra-prediction unit.

[0106] In this specification, the embodiments described in each configuration of the encoding device (200) may be applied to the corresponding configuration of the decoding device (300) in the same or corresponding manner.

[0107] As described above, prediction is performed to increase compression efficiency during video coding. Through this, a predicted block containing predicted samples for the current block, which is the block to be coded, can be generated. Here, the predicted block includes predicted samples in the spatial domain (or pixel domain). The predicted block is derived identically by both the encoding device and the decoding device, and the encoding device can increase video coding efficiency by signaling information regarding the residual between the original block and the predicted block (residual information) to the decoding device, rather than the original sample value of the original block itself. The decoding device derives a residual block containing residual samples based on the residual information, can generate a restored block containing restored samples by combining the residual block and the predicted block, and can generate a restored frame containing the restored blocks.

[0108] The above residual information can be generated through transformation and quantization procedures. For example, an encoding device may derive a residual block between the original block and the predicted block, perform a transformation procedure on residual samples (residual sample array) included in the residual block to derive transformation coefficients, perform a quantization procedure on the transformation coefficients to derive quantized transformation coefficients, and signal the related residual information to a decoding device (via a bitstream). Here, the residual information may include information such as value information, position information, transformation technique, transformation kernel, and quantization parameters of the quantized transformation coefficients. The decoding device may perform an inverse quantization / inverse transformation procedure based on the residual information and derive residual samples (or residual blocks). The decoding device may generate a restored frame based on the predicted block and the residual block. The encoding device can also derive a residual block by inversely quantizing / inversely transforming the quantized transform coefficients for reference to inter-prediction of subsequent frames, and generate a restored frame based thereon.

[0109] In the present disclosure, at least one of quantization / inverse quantization and / or transformation / inverse transformation may be omitted. If the quantization / inverse quantization is omitted, the quantized transformation coefficient may be referred to as a transformation coefficient. If the transformation / inverse transformation is omitted, the transformation coefficient may be referred to as a coefficient or residual coefficient, or may still be referred to as a transformation coefficient for the sake of consistency of expression.

[0110] Additionally, in this disclosure, quantized transform coefficients and transform coefficients may be referred to as transform coefficients and scaled transform coefficients, respectively. In this case, residual information may include information regarding transform coefficient(s), and information regarding said transform coefficient(s) may be signaled through residual coding syntax. Transform coefficients may be derived based on said residual information (or information regarding said transform coefficient(s), and scaled transform coefficients may be derived through inverse transform (scaling) of said transform coefficients. Residual samples may be derived based on inverse transform (transform) of said scaled transform coefficients. This may be similarly applied / expressed in other parts of this disclosure.

[0111] The video / image coding method according to the present disclosure may be performed based on a partitioning structure. Specifically, procedures such as prediction, residual processing ((inverse)transform, (inverse)quantization, etc.), syntax element coding, and in-loop filtering may be performed based on a superblock, coding block, or transform block derived based on the partitioning structure. The block partitioning procedure may be performed in the image splitting unit of the encoding device described above, and the partition information may be processed (encoded) in the entropy encoding unit and transmitted to the decoding device in the form of a bitstream. The entropy decoding unit of the decoding device may derive a block partitioning structure of a frame based on the partition information obtained from the bitstream, and perform a series of procedures for decoding (e.g., prediction, residual processing, block / picture restoration, in-loop filtering, etc.) based thereon. The partition information may be referred to as split information.

[0112] As described above, the bitstream can be packetized through OBUs (open bitstream units). A single frame can be divided into one or more rectangular tiles. Tile-related information can be signaled from the encoding device to the decoding device. For example, tile-related information can be signaled through tile group OBU syntax. Uniform / non-uniform tile sizes can be determined on a frame-by-frame basis and signaled from the encoding device to the decoding device. For example, information regarding tile size can be transmitted at the frame header level or signaled through the aforementioned tile group OBU syntax.

[0113] A single image or frame can be divided into superblocks. Based on the luminance component, a superblock can have a size of, for example, 64×64, 128×128, or 256×256. A superblock of the chroma component can have a size equal to or smaller than that of a superblock of the luminance component. For example, when considering a 4:2:0 color format, a superblock of the chroma component can have a size of, for example, 32×32, 64×64, or 128×128. However, the aforementioned superblock sizes are examples, and it is also possible to set larger or smaller block sizes. A tile is composed of superblocks, and the superblocks within a tile can be arranged in raster scan order.

[0114] According to one embodiment of the present disclosure, a coding block can be divided into various partitioning structures starting from a superblock.

[0115] The following table shows examples of partitioning structures.

[0116] PartitionName of Partition0PARTITION_NONE1PARTITION_HORZ2PARTITION_VERT3PARTITION_SPLIT4PARTITION_HORZ_A5PARTITION_HORZ_B6PARTITION_VERT_A7PARTITION_VERT_B8PARTITION_HORZ_49PARTITION_VERT_4

[0117] PARTITION_NONE indicates that the superblock or coding block is not partitioned into smaller coding blocks. PARTITION_HORZ indicates that the superblock or coding block is binary partitioned horizontally. PARTITION_VERT indicates that the superblock or coding block is binary partitioned vertically. PARTITION_SPLIT indicates that the superblock or coding block is quadtree partitioned. PARTITION_HORZ_A indicates that the superblock or coding block is partitioned into three coding blocks, with the lower portion being a 2N×N coding block and the upper portion being two N×N coding blocks. PARTITION_HORZ_B indicates that the superblock or coding block is partitioned into three coding blocks, with the upper portion being a 2N×N coding block and the lower portion being two N×N coding blocks. PARTITION_VERT_A indicates that a superblock or coding block is divided into three coding blocks, with the right side being a 2N×N coding block and the left side being two N×N coding blocks. PARTITION_VERT_B indicates that a superblock or coding block is divided into three coding blocks, with the left side being a 2N×N coding block and the right side being two N×N coding blocks. PARTITION_HORZ_4 indicates that a superblock or coding block is divided into four coding blocks in the horizontal direction. PARTITION_VERT_4 indicates that a superblock or coding block is divided into four coding blocks in the vertical direction. For example, a coding block can be recursively divided if it is divided into a specific partitioning structure. For example, if a coding block is divided into a PARTITION_SPLIT structure, the decision to divide can be made recursively. Meanwhile, for example, if the size of a coding block is smaller than a specific size, recursive partitioning may not be allowed regardless of whether it is partitioned into a PARTITION_SPLIT structure. For example, if the size of a coding block is smaller than 8×8, recursive partitioning may not be allowed.

[0118] FIG. 4 shows an example of a partitioning structure according to one embodiment of the present disclosure.

[0119] Referring to FIG. 4, a coding block (including a superblock) may not be partitioned, may be partitioned into binary partitions, quad partitions, ab partitions, or 1-to-4 partitions. If the coding block is not partitioned, it may be represented as PARTITION_NONE. If the coding block is not partitioned, the block size may be expressed as 2N×2N. If the coding block is partitioned into binary partitions, it may be represented as PARTITION_HORZ or PARTITION_VERT. If the coding block is partitioned into quad partitions, it may be represented as PARTITION_SPLIT. If the coding block is partitioned into ab partitions, it may be represented as PARTITION_HORZ_A, PARTITION_HORZ_B, PARTITION_VERT_A, or PARTITION_VERT_B. If the coding block is partitioned into 1-to-4 partitions, it may be represented as PARTITION_HORZ_4 or PARTITION_VERT_4.

[0120] FIG. 5 shows an example of a partitioning structure considering recursive partitioning according to one embodiment of the present disclosure.

[0121] Referring to FIG. 5, a superblock or coding block can be partitioned based on various partitioning structures. If the superblock is partitioned into a quadtree, the partitioned coding blocks can be recursively checked for further partitioning. Meanwhile, the partitioned coding blocks may have partitioning candidates limited to a specific size (e.g., width and / or height of 8). In this case, for example, PARTITION_SPLIT, PARTITION_HORZ, and PARTITION_VERT may be candidates.

[0122] For example, different partitioning candidates may be available based on block size. If the size of the coding block (including the superblock) is a first size, first partitioning candidates may be available; if the size of the coding block is a second size, second partitioning candidates may be available; and if the size of the coding block is a third size, third partitioning candidates may be available. In this case, the first size may be larger than the second size, and the second size may be larger than the third size. Also, in this case, the number of second partitioning candidates b may be greater than the number of first partitioning candidates a, and the number of third partitioning candidates c may be smaller than the number of second partitioning candidates b. In this case, for example, a may be greater than c. Or, for example, a may be smaller than c. For example, the first partitioning candidates may be a subset of the second partitioning candidates. The third partitioning candidates may be a subset of the second partitioning candidates.

[0123] The following table illustrates the relationship between coding block sizes and partitioning candidates.

[0124] Coding Block Size Partitioning Candidates Size 1 (e.g., 128×128 or 256×256) Size 1 Partitioning Candidates (e.g., PARTITION_NONE, PARTITION_HORZ, PARTITION_VERT, PARTITION_SPLIT, PARTITION_HORZ_A, PARTITION_HORZ_B, PARTITION_VERT_A, PARTITION_VERT_B) Size 2 (e.g., 64×64 ~ 16×16) Size 2 Partitioning Candidates (e.g., PARTITION_NONE, PARTITION_HORZ, PARTITION_VERT, PARTITION_SPLIT, PARTITION_HORZ_A, PARTITION_HORZ_B, PARTITION_VERT_A, PARTITION_VERT_B, PARTITION_HORZ_4, PARTITION_VERT_4) Size 3 (e.g., 8×8) Size 3 Partitioning Candidates (ex. PARTITION_NONE, PARTITION_HORZ, PARTITION_VERT, PARTITION_SPLIT)

[0125] The partitioning structure described above may be indicated based on partition information. The partition information may indicate one of the partitioning candidates based on the block size. For example, the partition information may be signaled when the current block size is not smaller than 8×8.

[0126] Meanwhile, the current coding block, upon completion of the partitioning, can be divided into transformation blocks. Intra-prediction can be performed at the transformation block level. The transformation block can be a square block or a non-square block. For example, intra / inter mode, intra-prediction mode, etc., can be signaled at the coding block level, and intra-prediction using surrounding reference samples can be performed at the transformation block level within the coding block.

[0127] For example, if the current coding block is an intra-block, the division information of the transformation block may be indicated based on the size, shape, and / or depth information (transformation depth information) of the current coding block. For example, if the current coding block is an intra-block and a square block, it may be quadrupled if the depth information is greater than 0. For example, if the current coding block is an intra-block and a non-square block, it may be binaryrupled if the depth information is greater than 0.

[0128] In addition to the candidates mentioned above, the following partitioning structures may be used to improve image / video coding efficiency, either in addition to or in place of some of the candidates mentioned above.

[0129] FIG. 6 shows examples of partitioning candidates proposed in the present disclosure.

[0130] Referring to FIG. 6, a coding block can be asymmetrically binary divided into two sub-coding blocks. The two sub-coding blocks may be non-square and have different sizes. FIG. 6 (a), (b), (c) and (d) show examples of vertical division, and FIG. 6 (e), (f), (g) and (h) show examples of horizontal division.

[0131] Referring to FIG. 6(a), when the size of the coding block is 2N×2N, the size of the first partition (first sub-coding block) is n×2N, and the size of the second partition (second sub-coding block) is (2N-n)×2N. For example, n can be equal to or less than N / 2.

[0132] Referring to FIG. 6(b), when the size of the coding block is 2N×2N, the size of the first partition (first sub-coding block) is (2N-n)×2N, and the size of the second partition (second sub-coding block) is n×2N. For example, n can be equal to or less than N / 2.

[0133] Referring to Fig. 6(c), when the size of the coding block is 2N×2N, the size of the first partition (first sub-coding block) is k×2N, and the size of the second partition (second sub-coding block) is (2N-k)×2N. For example, k can be equal to or less than N / 4.

[0134] Referring to FIG. 6(d), when the size of the coding block is 2N×2N, the size of the first partition (first sub-coding block) is (2N-k)×2N, and the size of the second partition (second sub-coding block) is k×2N. For example, k can be equal to or smaller than N / 4.

[0135] Referring to FIG. 6(e), when the size of the coding block is 2N×2N, the size of the first partition (first sub-coding block) is 2N×n, and the size of the second partition (second sub-coding block) is 2N×(2N-n). For example, n can be equal to or less than N / 2.

[0136] Referring to (f) in Fig. 6, when the size of the coding block is 2N×2N, the size of the first partition (first sub-coding block) is 2N×(2N-n) and the size of the second partition (second sub-coding block) is 2N×n. For example, n can be equal to or less than N / 2.

[0137] Referring to Fig. 6 (g), when the size of the coding block is 2N×2N, the size of the first partition (first sub-coding block) is 2N×k, and the size of the second partition (second sub-coding block) is 2N×(2N-k). For example, k may be equal to or smaller than N / 4.

[0138] Referring to Fig. 6 (h), when the size of the coding block is 2N×2N, the size of the first partition (first sub-coding block) is 2N×(2N-k) and the size of the second partition (second sub-coding block) is 2N×k. For example, k may be equal to or smaller than N / 4.

[0139] The partitioning structure illustrated in (a) of FIG. 6 can be called a vertical A partition, the partitioning structure illustrated in (b) can be called a vertical B partition, the partitioning structure illustrated in (c) can be called a vertical a partition, the partitioning structure illustrated in (d) can be called a vertical b partition, the partitioning structure illustrated in (e) can be called a horizontal A partition, the partitioning structure illustrated in (f) can be called a horizontal B partition, the partitioning structure illustrated in (g) can be called a horizontal a partition, and the partitioning structure illustrated in (h) can be called a horizontal b partition. However, this is an example and may be referred to by other names. A partitioning structure like the one above can provide high coding efficiency in some image characteristics, such as when an object is located near the edge of a block.

[0140] FIG. 7 shows examples of partitioning candidates proposed in the present disclosure.

[0141] Referring to FIG. 7, the coding block can be divided into four sub-coding blocks. In this case, among the four sub-coding blocks, two square blocks and two non-square blocks may be included. For example, if the size of the coding block is 2N×2N, the size of the two square blocks may be N×N, and the size of the two non-square blocks may be n×2N. For example, if the size of the coding block is 2N×2N, the size of the two square blocks may be N×N, and the size of the two non-square blocks may be 2N×n.

[0142] The partitioning structure illustrated in (a) of Fig. 7 can be called a vertical H-type (shape) partition, and the partitioning structure illustrated in (b) can be called a horizontal H-type (shape) partition. However, this is an example and may be referred to by other names. Such a partitioning structure can provide high coding efficiency in some image characteristics, such as when there is an object in the central area of ​​a block.

[0143] FIG. 8 shows examples of partitioning candidates proposed in the present disclosure.

[0144] Referring to FIG. 8, a coding block can be asymmetrically divided into four sub-coding blocks. This can be referred to as asymmetric four-part vertical / horizontal partitioning. In this case, the four sub-coding blocks may include four non-square blocks. At least two or three of the four non-square blocks may have different sizes. FIG. 8 (a) and (b) show examples of vertical division, and FIG. 8 (c) and (d) show examples of horizontal division.

[0145] Referring to FIG. 8(a), when the size of the coding block is 2N×2N, the size of the first partition (first sub-coding block) may be l×2N, the size of the second partition (second sub-coding block) may be m×2N, the size of the third partition (third sub-coding block) may be n×2N, and the size of the fourth partition (fourth sub-coding block) may be k×2N. Here, m may be greater than l, n may be greater than m, and k may not be greater than n. For example, m may be twice l and n may be twice m. Or n may be equal to m+l or 2m+l. k may be equal to l.

[0146] Referring to FIG. 8(b), when the size of the coding block is 2N×2N, the size of the first partition (first sub-coding block) may be k×2N, the size of the second partition (second sub-coding block) may be n×2N, the size of the third partition (third sub-coding block) may be m×2N, and the size of the fourth partition (fourth sub-coding block) may be l×2N. Here, m may be greater than l, n may be greater than m, and k may not be greater than n. For example, m may be twice l and n may be twice m. Or n may be equal to m+l or 2m+l. k may be equal to l.

[0147] Referring to FIG. 8(c), when the size of the coding block is 2N×2N, the size of the first partition (first sub-coding block) may be 2N×l, the size of the second partition (second sub-coding block) may be 2N×m, the size of the third partition (third sub-coding block) may be 2N×n, and the size of the fourth partition (fourth sub-coding block) may be 2N×k. Here, m may be greater than l, n may be greater than m, and k may not be greater than n. For example, m may be twice l and n may be twice m. Or n may be equal to m+l or 2m+l. k may be equal to l.

[0148] Referring to FIG. 8(d), when the size of the coding block is 2N×2N, the size of the first partition (first sub-coding block) may be 2N×k, the size of the second partition (second sub-coding block) may be 2N×n, the size of the third partition (third sub-coding block) may be 2N×m, and the size of the fourth partition (fourth sub-coding block) may be 2N×l. Here, m may be greater than l, n may be greater than m, and k may not be greater than n. For example, m may be twice l and n may be twice m. Or n may be equal to m+l or 2m+l. k may be equal to l.

[0149] The partitioning structure illustrated in (a) of FIG. 8 can be called a vertical 4A partition, the partitioning structure illustrated in (b) can be called a vertical 4B partition, the partitioning structure illustrated in (c) can be called a horizontal 4A partition, and the partitioning structure illustrated in (d) can be called a horizontal 4B partition. However, this is an example and may be referred to by other names. The above partitioning structure can provide high coding efficiency in some image characteristics, such as when there are gradually changing image characteristics.

[0150] In the embodiments disclosed in FIGS. 6 to 8 above, the explanation was based on the case where the coding block to be divided is a square block, but this is an example, and it can be applied in the same way by reflecting the partitioning ratio even when the current coding block is a non-square block.

[0151] FIG. 9 shows an example of a partitioning structure according to another embodiment of the present disclosure.

[0152] Referring to FIG. 9, in addition to or by replacing some of the candidates in the partitioning structure described above, asymmetric binary partitioning, H-type partitioning, and / or asymmetric 1-to-4 partitioning may be included.

[0153] For example, (a) to (d) of FIG. 9 show examples where a coding block is asymmetrically binary divided, (e) and (h) show examples where a coding block is H-type divided, and (g) to (j) show examples where an asymmetrically 1 to 4 divided.

[0154] For example, the above partitioning structure may be allowed based on the block size. For example, when the size of the coding block is smaller than 128×128, the above asymmetric binary partitioning, H-type partitioning, and / or asymmetric 1to4 partitioning may be allowed.

[0155] Meanwhile, for example, even if the size of the coding block is smaller than 128×128, the partitioning structure as described above can be restricted by comparing the width and height of the block. For example, if the ratio of the width and height of the coding block is not 1:1 or 1:2 (or 2:1), the asymmetric binary partitioning, H-type partitioning, and / or asymmetric 1-to-4 partitioning as described above may not be allowed.

[0156] The partitioning structure may be restricted based on the size of the sub-coding block after partitioning. If the width of the sub-coding block after partitioning is greater than 64 and the height is less than 64, or if min(width, height) is less than 4 or 8, then the above asymmetric binary partitioning, H-type partitioning, and / or asymmetric 1-to-4 partitioning may not be allowed for the current coding block.

[0157] Based on the partitioning structure described above, coding blocks can be efficiently divided by considering various image characteristics, and procedures such as prediction and transformation reflecting the optimal block size can be performed.

[0158] Meanwhile, in the examples described above, recursive partitioning may be allowed except in cases where the coding block is not partitioned (PARTITION_NONE). However, even in this case, recursive partitioning may be restricted if the ratio of the width and height of the sub-coding block after partitioning is equal to or exceeds a certain ratio. For example, recursive partitioning may not be allowed if the ratio of the width and height of the sub-coding block is 1:4 (or 4:1) or exceeds (e.g., 1:8 or 8:1).

[0159] For example, different partitioning candidates may be available based on block size. If the size of the coding block (including the superblock) is a first size, first partitioning candidates may be available; if the size of the coding block is a second size, second partitioning candidates may be available; and if the size of the coding block is a third size, third partitioning candidates may be available. In this case, the first size may be larger than the second size, and the second size may be larger than the third size. Also, in this case, the number of second partitioning candidates b may be greater than the number of first partitioning candidates a, and the number of third partitioning candidates c may be smaller than the number of second partitioning candidates b. In this case, for example, a may be greater than c. Or, for example, a may be smaller than c. For example, the first partitioning candidates may be a subset of the second partitioning candidates. The third partitioning candidates may be a subset of the second partitioning candidates. For example, the first partitioning candidates may not be a subset of the second partitioning candidates. That is, at least one of the first partitioning candidates may not belong to the second partitioning candidates. The third partitioning candidates may be a subset of the second partitioning candidates.

[0160] The following table illustrates the relationship between coding block sizes and partitioning candidates.

[0161] Coding Block Size Partitioning Candidates Size 1 (e.g., 128×128, 128×256, 256×128, or 256×256) Size 1 Partitioning Candidates (e.g., PARTITION_NONE, PARTITION_HORZ, PARTITION_VERT, PARTITION_SPLIT) Size 2 (e.g., 64×256, 256×64, 64×128, 128×64, or 64×64) Size 2 Partitioning Candidates (e.g., PARTITION_NONE, PARTITION_HORZ, PARTITION_VERT, Asymmetric Binary Partition Candidates, H-Type Partition Candidates, and / or Asymmetric 1-to-4 Partition Candidates) Size 3 (e.g., 32×64, 64×32, 16×64, 64×16, 8×64, 65×8, or 8×8) Size 3 Partitioning candidates (e.g., PARTITION_NONE, PARTITION_HORZ, PARTITION_VERT)

[0162] The partitioning structure described above may be indicated based on partition information. The partition information may indicate one of the partitioning candidates based on the block size. For example, the partition information may be signaled when the current block size is not smaller than 8×8. For example, if the current block size is smaller than 8×8, PARTITION_NONE may be implicitly indicated.

[0163] Meanwhile, the above partition information may directly indicate one of the aforementioned partitioning candidates based on the block size, or it may indicate partition type information and partition direction information separately.

[0164] For example, partition type information and partition direction information can be signaled sequentially based on conditions as follows.

[0165]

[0166] For example, partition_type may represent at least one of non-partition (PARTITION_NONE), quad partition (PARTITION_SPLIT), (symmetric) binary partition, asymmetric binary partition A, asymmetric binary partition B, asymmetric binary partition a, asymmetric binary partition b, H-type partition, asymmetric 1 to 4 partition A and / or asymmetric 1 to 4 partition B.

[0167] partition_direction indicates whether the partition direction is vertical or horizontal. partition_direction may be signaled if partition_type does not indicate non-partition (PARTITION_NONE) or quad partition (PARTITION_SPLIT).

[0168] As another example, partition_type may represent at least one of non-partition (PARTITION_NONE), quad partition (PARTITION_SPLIT), (symmetric) binary partition, asymmetric binary partition, H-type partition, and / or asymmetric 1-to-4 partition. The following table shows examples of partition type indexing.

[0169] partition_typedescription0PARTITION_NONE1PARTITION_SPLIT2(symmetric) binary partition3unsymmetric binary partition4H type partition5unsymmetric 1to4 partition

[0170] partition_direction indicates whether the partition direction is vertical or horizontal and the mode (A, B, a, or b). The following table shows examples of partition direction indexing.

[0171] partition_directiondescription0Vertical (or Vertical A)1Horizontal (or Horizontal A)2Vertical B3Horizontal B4Vertical a5Horizontal a6Vertical b7Horizontal b

[0172] The partition_direction 0 described above may represent vertical or vertical A depending on the partition type. For example, if the partition type is (symmetric) binary partition, the partition_direction 0 represents a vertical partition, and if the partition type is an asymmetric type (e.g., asymmetric binary partition), the partition_direction 0 may represent a vertical A partition. For example, if the partition type is (symmetric) binary partition, the partition_direction 1 represents a horizontal partition, and if the partition type is an asymmetric type (e.g., asymmetric binary partition), the partition_direction 1 may represent a horizontal A partition. As described above, partition_direction may be signaled when partition_type does not indicate non-partition (PARTITION_NONE) or quad partition (PARTITION_SPLIT).

[0173] Meanwhile, as described above, the current coding block, upon completion of the coding block division, can be divided into transformation blocks. Intra prediction can be performed at the transformation block level. The transformation block may be a square block or a non-square block. For example, intra / inter mode, intra prediction mode, etc., can be signaled at the coding block level, and intra prediction using surrounding reference samples can be performed at the transformation block level within the coding block.

[0174] If the size of the current coding block after coding block partitioning is larger than a specific size, implicit transformation block partitioning can be performed up to that specific size without separate signaling. Information for additional transformation block partitioning can be signaled for the transformation blocks derived up to that specific size.

[0175] For example, if the size of the current coding block is 128×256 and the specific size is 128×128, two 128×128 transformation blocks are implicitly derived within the current coding block, and transformation block division information (e.g., transformation depth information) for each of the 128×128 transformation blocks can be signaled separately.

[0176] For example, if the size of the current coding block is 256×256 and the specific size is 128×128, four 128×128 transformation blocks are implicitly derived within the current coding block, and transformation block division information (e.g., transformation depth information) for each of the 128×128 transformation blocks can be signaled separately.

[0177] For example, if the size of the current coding block is 128×64 and a specific size is 64×64, two 64×64 transformation blocks are implicitly derived within the current coding block, and transformation block partitioning information (e.g., transformation depth information) for each of the 64×64 transformation blocks can be signaled separately.

[0178] For example, if the size of the current coding block is 128×128 and a specific size is 64×64, four 64×64 transformation blocks are implicitly derived within the current coding block, and transformation block division information (e.g., transformation depth information) for each of the 64×64 transformation blocks can be signaled separately.

[0179] For example, if the size of the current coding block is 128×256 and a specific size is 64×64, eight 64×64 transformation blocks are implicitly derived within the current coding block, and transformation block division information (e.g., transformation depth information) for each of the 64×64 transformation blocks can be signaled separately.

[0180] For example, if the size of the current coding block is 256×256 and a specific size is 64×64, 16 64×64 transformation blocks are implicitly derived within the current coding block, and transformation block division information (e.g., transformation depth information) for each of the 64×64 transformation blocks can be signaled separately.

[0181] Meanwhile, considering image characteristics, availability information regarding allowed partitioning types / structures may be signaled through upper-level syntax (e.g., sequence header or frame header). For example, at least one of asymmetric binary partitioning candidates, H-type partitioning candidates, and / or asymmetric 1-to-4 partitioning candidates may not be allowed through the availability information. In this case, the partition information may indicate one of the remaining candidates excluding the disallowed candidates. Indexing for the disallowed candidates is excluded, and indexing for the remaining candidates may be updated. The same applies to other examples below.

[0182] Additionally, the permissible partitioning type / structure may be set differently based on the coding block size. For example, if the width of the current coding block is greater than the height, at least one of the partitioning candidates having a horizontal orientation among the aforementioned candidates may not be allowed. For example, if the width of the current coding block is smaller than the height, at least one of the partitioning candidates having a vertical orientation among the aforementioned candidates may not be allowed. In this case, the partition information may indicate one of the remaining candidates excluding the disallowed candidates.

[0183] Meanwhile, in intra prediction, prediction samples for the current block are generated using reconstructed samples of surrounding blocks within the current frame. Here, the current block may be a coding block or a transformation block. For example, the derivation of the intra prediction mode / type may be performed at the coding block level, while the procedure for generating prediction samples based on surrounding reference samples may be performed at the transformation block level. When intra prediction is applied to the current block, surrounding reference samples to be used for the intra prediction of the current block may be derived. The surrounding reference samples of the current block may include H+W samples located to the left of the current block of size W / XH, W+H samples located to the top of the current block, and at least one sample adjacent to the top-left of the current block. Alternatively, the surrounding reference samples of the current block may include multiple rows of upper surrounding samples and multiple columns of left surrounding samples. If there are multiple transformation blocks within a coding block, intra prediction (deriving surrounding reference samples and generating prediction samples) can be performed sequentially in the order of raster scans. For example, the intra prediction procedure can be triggered based on a predict_intra function call.

[0184] The intra prediction procedure can be broadly classified into the intra prediction mode / type derivation procedure, the surrounding sample derivation procedure, and the (intra) prediction sample(s) generation procedure. However, some procedures may be omitted depending on the intra prediction mode.

[0185] Figure 10 illustrates an exemplary intra-prediction procedure.

[0186] With reference to FIG. 10, as described above, the process can be divided into a prediction mode / type derivation procedure, a peripheral sample derivation procedure, and a (intra) prediction sample(s) generation procedure. The intra prediction procedure can be performed identically or correspondingly in an encoding device and a decoding device as described above. In this disclosure, the term "coding device" may include an encoding device and / or a decoding device.

[0187] The coding device determines an intra-prediction mode / type for the current block (S1000). The current block may correspond to a coding block or a conversion block.

[0188] The encoding device can determine an intra prediction mode / type applied to the current block among the various intra prediction modes / types described in the present disclosure and can generate prediction-related information. The prediction-related information may include intra prediction mode information indicating an intra prediction mode applied to the current block and / or intra prediction type information indicating an intra prediction type applied to the current block. The decoding device can determine an intra prediction mode / type applied to the current block based on the prediction-related information.

[0189] The above intra prediction type may represent various prediction types for performing intra prediction. Intra prediction types may include, for example, MRLS (multiple reference line selection), IBP (intra bi-prediction), RIP (recursive intra prediction), intra prediction fusion, etc. MRLS may represent a prediction type that performs intra prediction by selecting some of the multiple reference sample lines for intra prediction. IBP may represent a prediction type that performs prediction using reference samples in the opposite direction to the reference samples in the prediction direction when performing intra prediction. RIP may represent a prediction type that divides the current block into m×n sample regions (e.g., m is 4, n is 2) subblocks (or patches) and then sequentially generates samples around those regions.

[0190] For example, when intra prediction is applied, the intra prediction mode applied to the current block may be determined based on the intra prediction modes of spatial / temporal neighboring blocks. For example, the coding device may derive available intra prediction modes for the current block based on the intra prediction modes of the current block's neighboring blocks (e.g., left and / or upper neighboring blocks) and / or the intra prediction modes of the temporal neighboring blocks (e.g., blocks at the same location in the reference frame). The available intra prediction modes may be prioritized (i.e., ranked) based on the intra prediction modes of the current block's neighboring blocks (e.g., left and / or upper neighboring blocks) and / or the intra prediction modes of the temporal neighboring blocks (e.g., blocks at the same location in the reference frame). In this case, the available intra prediction modes may be divided into multiple mode sets based on the priority (or rank). The encoding device may generate mode set index information and / or mode index information. The above mode set index information may indicate a mode set among the plurality of mode sets that includes an intra prediction mode applied to the current block. The above mode index information may indicate an index of the intra prediction mode applied to the current block within the indicated mode set.

[0191] The encoding device can perform predictions based on various intra prediction modes / types and determine the optimal intra prediction mode / type based on rate-distortion optimization (RDO) derived therefrom. In this case, the encoding device may determine the optimal intra prediction mode using candidates within the aforementioned mode set. Specifically, for example, if the intra prediction type of the current block is a specific type (e.g., MRLS, IBP, or RIP) rather than a normal intra prediction type, the encoding device may determine the optimal intra prediction mode by considering only the candidates from the first mode set among the aforementioned mode sets as intra prediction mode candidates for the current block. That is, in this case, the intra prediction mode for the current block can be determined only from among the first mode sets, and in this case, the mode set index information may not be explicitly coded / signaled. In this case, the decoding device may consider the first mode set to have been selected without explicitly parsing / signaling the mode set index information. The first mode set may correspond to the mode set having the highest priority (rank) indicated when the value of the mode set index information is 0. In this case, without signaling the mode set index, the index of the intra prediction mode applied to the current block within the first mode set can be indicated based on the signaling of the mode index information. In this case, the complexity can be reduced, the number of bits required for mode signaling can be reduced, and the intra prediction efficiency can be increased by using a high-priority intra prediction mode.

[0192] The coding device derives surrounding reference samples of the current block (S1010). The surrounding reference samples of the current block may include H+W samples located to the left of the current block of size WХH, W+H samples located to the top of the current block, and at least one sample adjacent to the top-left of the current block. Alternatively, the surrounding reference samples of the current block may include multiple rows of upper surrounding samples and multiple columns of left surrounding samples.

[0193] Some of the surrounding reference samples of the current block mentioned above may not yet be decoded / restored or may not be available. In this case, the coding device may construct the surrounding reference samples to be used for intra-prediction by padding or substituting the unavailable samples with the available samples.

[0194] Figure 11 shows an example of peripheral reference samples for intra-prediction.

[0195] Referring to FIG. 11, the surrounding reference samples of the current block may include W+H or more upper reference samples, H+W or more left reference samples, and one or more upper-left reference samples. The surrounding reference samples may include upper adjacent reference samples (A), upper-right reference samples (AR), left adjacent reference samples (L), lower-left reference samples (BL), and upper-left reference samples (AL).

[0196] Upper reference samples can be denoted, for example, as AboveRow[i], where i can have values ​​ranging from 0 to W+H-1. If upper reference samples are unavailable, and if left reference samples are available, one of the left reference samples can be used as the sample value for the upper reference samples. Specifically, for example, unavailable upper reference samples can be padded or replaced with the uppermost reference sample among the left adjacent reference samples. In other words, unavailable upper reference samples can be padded or replaced with the left reference sample at coordinates (x-1, y). Here, (x, y) can represent the upper-left sample position of the current block (e.g., coding block or transformation block). If upper reference samples are unavailable and left reference samples are also unavailable, the upper reference samples can be padded or replaced with a default value derived based on a predetermined bit depth. Here, the above default value can be set to, for example, (1<<(BitDepth-1))-1. Alternatively, unavailable upper reference samples may be padded or replaced to be equal to the upper-left reference sample.

[0197] If upper reference samples are available, the restored sample values ​​of the upper reference samples are basically used as the values ​​of the corresponding upper reference samples. Meanwhile, if upper reference samples are available, a first parameter may be determined. The first parameter may be determined, for example, as x+2W-1. If H > W, the restored sample values ​​are not used for specific upper reference samples having an x-coordinate greater than the first parameter, and for said specific upper reference samples, the value of the upper reference sample having the x-coordinate of the first parameter is copied (padding or replaced). That is, for said specific upper reference samples, the value of the upper reference sample at the coordinate (first parameter, y-1) may be copied (padding or replaced). Alternatively, if H > W, for upper reference samples after the 2Wth upper reference sample, the value of the 2Wth upper reference sample may be copied (padding or replaced).

[0198] Left reference samples can be denoted, for example, as LeftCol[i], where i can have values ​​ranging from 0 to W+H-1. If left samples are unavailable, and if upper reference samples are available, one of the upper reference samples can be used as the sample value for the said left reference samples. Specifically, for example, unavailable left reference samples can be padded or replaced to be equal to the leftmost reference sample among the upper adjacent reference samples. In other words, unavailable left reference samples can be padded or replaced to be equal to the upper reference sample at coordinates (x, y-1). Here, (x, y) can represent the upper-left sample position of the current block (e.g., coding block or transformation block). If left reference samples are unavailable and upper reference samples are also unavailable, the said left reference samples can be padded or replaced with a default value derived based on a predetermined bit depth. Here, the above default value can be set to, for example, (1<<(BitDepth-1))-1. Alternatively, unavailable left reference samples may be padded or replaced to be equal to the top-left reference sample.

[0199] If left reference samples are available, the restored sample values ​​of the left reference samples are used as the values ​​of the corresponding left reference samples by default. Meanwhile, if left reference samples are available, a second parameter may be determined. The second parameter may be determined, for example, as x+2H-1. If W>H, the restored sample values ​​are not used for specific left reference samples having a y-coordinate greater than the second parameter, and for said specific left reference samples, the value of the left reference sample having the y-coordinate of the second parameter is copied (padding or replaced). That is, for said specific left reference samples, the value of the left reference sample at the coordinate (x-1, second parameter) may be copied (padding or replaced). Alternatively, if W>H, for left reference samples after the 2Hth left reference sample, the value of the 2Hth left reference sample may be copied (padding or replaced).

[0200] The upper-left reference sample includes the reference sample at coordinates (x-1, y-1) and may be denoted as AboveRow[-1] or LeftCol[-1] for convenience. If the upper-left reference sample is available, the restored sample value of that reference sample is used. If the upper-left reference sample is not available, the value of the available left reference sample at coordinates (x-1, y) or the available upper reference sample at coordinates (x, y-1) may be copied (padding or replaced). If the upper-left reference sample is not available, and neither the left reference sample nor the upper reference sample is available, the upper-left reference samples may be padded or replaced with a default value derived based on a predetermined bit depth. Here, the default value may be set to, for example, (1<<(BitDepth-1))-1.

[0201] Referring again to FIG. 10, the coding device generates prediction samples of the current block (S1020). The coding device can generate the prediction samples based on the intra prediction mode / type and the reference samples. The coding device can derive a reference sample according to the intra prediction mode of the current block among the reference samples of the current block, and can derive a prediction sample of the current block based on the reference sample.

[0202] The intra prediction mode may be signaled based on intra prediction mode information, and the intra prediction type may be signaled based on intra prediction type information. The intra prediction mode information and / or the intra prediction type information may be encoded / decoded through the binarization and coding methods described in this disclosure. For example, the intra prediction mode information and / or the intra prediction type information may be encoded / decoded through various coding methods (e.g., CDF, CABAC).

[0203] Intra-prediction modes may include, for example, the following.

[0204] Intra-prediction mode numbering 0DC Intra-prediction mode 1~8 Directional Intra-prediction modes 9~11 SMOOTH Intra-prediction modes 12 PAETH Intra-prediction mode

[0205] Here, in DC intra prediction mode, the current block can be predicted using the average value of the surrounding adjacent reference samples of the current block. The DC intra prediction mode may be referred to as the DC prediction mode or DC mode. In directional intra prediction modes, prediction can be performed using the value of a reference sample located in a specific direction based on the sample position within the current block. Directional intra prediction modes may basically include horizontal direction mode, horizontal downward left (D203) mode, horizontal upward left (D157) mode, upward diagonal left (D135) mode, vertical direction mode, vertical upward left (D133) mode, vertical upward right (D67) mode, and upward diagonal right (D45) mode. Additionally, an angle delta value may be additionally signaled to indicate a more detailed intra prediction direction. In SMOOTH intra prediction modes, prediction can be performed by interpolating the values ​​of the surrounding reference samples of the current block. In this case, depending on the mode, vertical interpolation, horizontal interpolation, or both may be performed. In PAETH intra prediction mode, predictions can be performed using the upper-left reference sample, left reference sample, and upper reference sample of the current block. In this case, predictions can be performed using the difference between the left reference sample and the upper-left reference sample, and the difference between the upper reference sample and the upper-left reference sample. Meanwhile, a CFL (chroma from luma) intra prediction mode for chroma blocks may be further considered. In CFL intra prediction mode, chroma blocks can be predicted based on linear or non-linear models using reconstructed luma blocks. For example, CFL intra prediction mode may be assigned intra prediction mode number 13.

[0206] Meanwhile, as described above, multiple directional intra prediction modes may be considered to improve the performance of intra prediction.

[0207] FIG. 12 illustrates the directional intra prediction modes described in the present disclosure as an example.

[0208] Referring to FIG. 12, directional intra prediction modes may basically include a horizontal direction mode, a horizontal downward-left mode (D203), a horizontal upward-left mode (D157), an upward-left diagonal mode (D135), a vertical direction mode, a vertical upward-left mode (D133), a vertical upward-right mode (D67), and an upward-right diagonal mode (D45). Additionally, an angle delta value may be additionally signaled to indicate a more detailed intra prediction direction. When a directional intra prediction mode is applied, a prediction sample may be generated using a reference sample located in the intra prediction direction relative to the position of the target sample within the current block. However, this is an example, and more directional prediction modes may be used.

[0209] In addition, as described above, the angle delta value can be additionally signaled to indicate a more detailed intra-prediction direction.

[0210] Figure 13 exemplarily illustrates directional intra prediction modes extended based on angle delta values.

[0211] Referring to FIG. 13, information representing one of n angle deltas (or offsets) can be signaled based on the basic intra prediction directions described above, such as horizontal direction, horizontal downward left (D203), horizontal upward left (D157), diagonal upward left (D135), vertical direction, vertical upward left (D133), vertical upward right (D67), and diagonal upward right (D45). Here, an example is shown where n is 6. That is, based on the basic intra prediction direction, there may be three angle delta values ​​to the left and right in addition to the basic 0 degrees. In other words, each basic intra prediction direction may have a total of 8 detailed intra prediction directions.

[0212] For example, information regarding angle delta can be signaled based on the block size. For example, if the intra prediction mode of the current block is a directional intra prediction mode and the width and / or height of the current block is 8 or greater, information regarding angle delta can be additionally signaled.

[0213] For example, if the intra prediction mode of the current block is a directional intra prediction mode, an angle delta flag is signaled first, and if the value of the angle delta flag is 1, angle delta index information may be signaled. The angle delta index information may indicate one of the angle deltas described above. If the value of the angle delta flag is 0, the signaling of the angle delta index information may be omitted.

[0214] Meanwhile, as described above, the surrounding reference samples for intra-prediction may be based on Multi-Reference Line Selection (MRLS). In this case, the surrounding reference samples may include upper surrounding reference samples of multiple rows and left surrounding reference samples of multiple columns.

[0215] Figure 14 shows an example of peripheral reference samples of multiple reference lines for intra-prediction. Here, four lines are shown, but this is an example and more or fewer lines may be used.

[0216] Referring to FIG. 14, the surrounding reference samples of a line of the current block may include W+H or more upper reference samples, H+W or more left reference samples, and one or more upper-left reference samples. Here, Line 1 (Reference Sample Line 1) represents a reference sample line adjacent to the current block. Line 2 (Reference Sample Line 2) represents a reference sample line located at a distance of 1 sample from the left / upper boundary of the current block. Line 3 (Reference Sample Line 3) represents a reference sample line located at a distance of 2 sample from the left / upper boundary of the current block. Line 4 (Reference Sample Line 4) represents a reference sample line located at a distance of 3 sample from the left / upper boundary of the current block.

[0217] For example, the upper-left reference sample area of ​​line 1 may contain 1 upper-left reference sample. The upper-left reference sample area of ​​line 2 may contain 3 upper-left reference samples. The upper-left reference sample area of ​​line 3 may contain 5 upper-left reference samples. The upper-left reference sample area of ​​line 4 may contain 7 upper-left reference samples.

[0218] For example, if the number of the reference sample line increases, the number of upper-left reference samples in the upper-left reference sample area increases, but the number of upper-left reference samples in the upper reference sample area may not increase. For example, the number of upper-left reference samples in line n+1 may be greater than the number of upper-left reference samples in line n, and the number of upper-left reference samples in line n may be equal to the number of upper-left reference samples in line n+1. For example, n may be 1, 2, 3, etc.

[0219] For example, if the reference sample line number increases, the number of upper-left reference samples in the upper-left reference sample area increases, but the number of left-left reference samples in the left-left reference sample area may not increase. For example, the number of upper-left reference samples in line n+1 may be greater than the number of upper-left reference samples in line n, and the number of left-left reference samples in line n may be equal to the number of left-left reference samples in line n+1. For example, n can be 1, 2, 3, etc.

[0220] For example, if the intra-prediction direction based on the position of a specific sample within the current block indicates to the right of the position of the rightmost reference sample of the upper reference line, the predicted sample value of said specific sample may be set to be equal to the value of said rightmost reference sample. For example, if the intra-prediction direction based on the position of a specific sample within the current block indicates to the lower of the position of the bottommost reference sample of the left reference line, the predicted sample value of said specific sample may be set to be equal to the value of said bottommost reference sample. Through this method, even when multiple reference lines are used, a fixed number (e.g., W+H) of upper reference samples and left reference samples can be used regardless of the line.

[0221] Even when multiple reference lines as described above are used, some of the surrounding reference samples may not yet be decoded / restored or may not be available. In this case, the coding device may construct the surrounding reference samples to be used for intra-prediction by padding or substituting the unavailable samples with the available samples. The reference line index may be denoted as rlidx.

[0222] The upper reference samples of the n-th reference sample line can be denoted, for example, as AboveRow[i], where i can have values ​​ranging from 0 to W+H-1. If the upper reference samples are unavailable, and if the left reference samples are available, one of the left reference samples can be used as the sample value for the upper reference samples. Specifically, for example, the unavailable upper reference samples can be padded or replaced with the uppermost reference sample among the left adjacent reference samples. In other words, the unavailable upper reference samples can be padded or replaced with the left reference sample at the coordinate (x-1-rlidx, y). Here, (x, y) can represent the upper-left sample position of the current block (e.g., coding block or transformation block). If the upper reference samples are unavailable and the left reference samples are also unavailable, the upper reference samples can be padded or replaced with a default value derived based on a predetermined bit depth. Here, the above default value may be set to, for example, (1<<(BitDepth-1))-1. Alternatively, unavailable upper reference samples may be padded or replaced with the upper-left reference sample at coordinates (x-1, y-1-rlidx).

[0223] If upper reference samples are available, the restored sample values ​​of the upper reference samples are basically used as the values ​​of the corresponding upper reference samples. Meanwhile, if upper reference samples are available, a first parameter may be determined. The first parameter may be determined, for example, as x+2W-1. If H > W, the restored sample values ​​are not used for specific upper reference samples having an x-coordinate greater than the first parameter, and for said specific upper reference samples, the value of the upper reference sample having the x-coordinate of the first parameter is copied (padding or replaced). That is, for said specific upper reference samples, the value of the upper reference sample at the coordinate (first parameter, y-1) may be copied (padding or replaced). Alternatively, if H > W, for upper reference samples after the 2Wth upper reference sample, the value of the 2Wth upper reference sample may be copied (padding or replaced).

[0224] Left reference samples can be denoted, for example, as LeftCol[i], where i can have values ​​ranging from 0 to W+H-1. If left samples are unavailable, and if upper reference samples are available, one of the upper reference samples can be used as the sample value for the said left reference samples. Specifically, for example, unavailable left reference samples can be padded or replaced to be equal to the leftmost reference sample among the upper adjacent reference samples. In other words, unavailable left reference samples can be padded or replaced to be equal to the upper reference sample at coordinates (x, y-1-rlidx). Here, (x, y) can represent the upper-left sample position of the current block (e.g., coding block or transformation block). If left reference samples are unavailable and upper reference samples are also unavailable, the said left reference samples can be padded or replaced with a default value derived based on a predetermined bit depth. Here, the above default value can be set to, for example, (1<<(BitDepth-1))-1. Alternatively, unavailable left reference samples may be padded or replaced with the upper-left reference sample at coordinates (x-1-rlidx, y-1).

[0225] If left reference samples are available, the restored sample values ​​of the left reference samples are used as the values ​​of the corresponding left reference samples by default. Meanwhile, if left reference samples are available, a second parameter may be determined. The second parameter may be determined, for example, as x+2H-1. If W>H, the restored sample values ​​are not used for specific left reference samples having a y-coordinate greater than the second parameter, and for said specific left reference samples, the value of the left reference sample having the y-coordinate of the second parameter is copied (padding or replaced). That is, for said specific left reference samples, the value of the left reference sample at the coordinate (x-1, second parameter) may be copied (padding or replaced). Alternatively, if W>H, for left reference samples after the 2Hth left reference sample, the value of the 2Hth left reference sample may be copied (padding or replaced).

[0226] The upper-left reference samples may be denoted as AboveRow[-i] and / or LeftCol[-j] for convenience, where i may include 1...rlidx and j may include 1...rlidx-1. If the upper-left reference sample is available, the restored sample value of that reference sample is used. If the upper-left reference sample is not available, the value of the available left reference sample at coordinates (x-1-rlidx, y) or the available upper reference sample at coordinates (x, y-1-rlidx) may be copied (padding or replaced). If the upper-left reference sample is not available, and neither the left reference sample nor the upper reference sample is available, the upper-left reference samples may be padded or replaced with a default value derived based on a predetermined bit depth. Here, the default value may be set to, for example, (1<<(BitDepth-1))-1.

[0227] Reference line selection information (or reference line index information) related to the aforementioned plurality of reference lines may be signaled. The reference line selection information may be signaled in units of coding blocks. That is, the reference line selection information is signaled in units of coding blocks, and transformation blocks within the coding blocks may derive reference samples using the same reference line number and perform intra prediction sequentially. In this case, intra prediction may be performed on the transformation blocks in the order of raster scans.

[0228] For example, the plurality of reference lines may be applied only when the current block is a luma component block. That is, the reference line selection information may be signaled when the coding block is a luma component block.

[0229] For example, if the prediction type is IBP and / or RIP, the reference line selection information may not be explicitly signaled. If the reference line selection information is not explicitly signaled, the value may be 0, which may point to reference line 1.

[0230] For example, the above reference line selection information can be signaled after intra-prediction mode information signaling.

[0231]

[0232] Here, intra_pred_mode represents the intra prediction mode information, and reference_line represents the reference line selection information. is_directional_mode is a variable indicating whether the intra prediction mode of the current block is a directional intra prediction mode. The is_directional_mode is not explicitly signaled and can be derived based on the intra prediction mode information.

[0233] For example, if a coding block is located at a specific boundary (adjacently), the reference line selection information may not be signaled. In this case, the value of the reference line selection information is implicitly inferred to be 0, and the first reference line on the left / upper side may be selected.

[0234] As another example, the reference line selection information may be signaled, but the indicated reference line may be set differently. For example, if the coding block is located (adjacently) at the specific boundary, the reference line selection information may be signaled, but the indicated reference line may be set differently. For example, if the coding block is located at the specific boundary, the reference line selection information may indicate the left line number and the upper line number differently. For example, if the coding block is located at the specific boundary, the reference line selection information may selectively indicate one of reference lines 1 through 4 for the left line, and indicate only reference line 1 for the upper line. In this case, the burden on the memory buffer can be reduced.

[0235] The specific boundary mentioned above may be a frame boundary, a tile boundary, and / or a superblock boundary. The specific boundary mentioned above may be an upper boundary of a frame, an upper boundary of a tile, and / or an upper boundary of a superblock.

[0236] Meanwhile, it is necessary to reconfigure multiple reference lines in consideration of cases where the left line number and the upper line number are indicated differently as described above. For example, when reference line 4 is selected in the left line and reference line 1 is selected in the upper line, some empty space occurs between surrounding reference samples according to the existing reference line configuration when performing intra prediction in the upper-left direction and the surrounding direction.

[0237] Figure 15 shows an example of multiple line surrounding reference samples for intra prediction.

[0238] Referring to FIG. 15, the surrounding reference samples of a line of the current block may include W+H or more upper reference samples, H+W or more left reference samples, and k (e.g., 4) or more upper-left reference samples. Here, line 1 (reference sample line 1) represents a reference sample line adjacent to the current block. Line 2 (reference sample line 2) represents a reference sample line located at a distance of 1 sample from the left / upper boundary of the current block. Line 3 (reference sample line 3) represents a reference sample line located at a distance of 2 sample from the left / upper boundary of the current block. Line 4 (reference sample line 4) represents a reference sample line located at a distance of 3 sample from the left / upper boundary of the current block.

[0239] The number of upper-left reference samples may be the same regardless of the reference sample line number. For example, the upper-left reference sample area of ​​line 1 may contain 4 upper-left reference samples. The upper-left reference sample area of ​​line 2 may contain 4 upper-left reference samples. The upper-left reference sample area of ​​line 3 may contain 4 upper-left reference samples. The upper-left reference sample area of ​​line 4 may contain 4 upper-left reference samples.

[0240] For example, if the number of the reference sample line increases, the number of upper-left reference samples in the upper-left reference sample area may not increase, and the number of upper-left reference samples in the upper reference sample area may not increase either. For example, the number of upper-left reference samples in line n may be equal to the number of upper-left reference samples in line n+1, and the number of upper-left reference samples in line n may be equal to the number of upper-left reference samples in line n+1. For example, n may be 1, 2, 3, etc.

[0241] For example, if the number of the reference sample line increases, the number of upper-left reference samples in the upper-left reference sample area may not increase, and the number of left-left reference samples in the left-left reference sample area may not increase either. For example, the number of upper-left reference samples in line n may be equal to the number of upper-left reference samples in line n+1, and the number of left-left reference samples in line n may be equal to the number of left-left reference samples in line n+1. For example, n may be 1, 2, 3, etc.

[0242] Based on the structure described above, even when the left line number and the upper line number are determined differently, there are no empty spaces in the surrounding reference samples, allowing for smooth intra-prediction.

[0243] Figure 16 shows an example of the configuration of surrounding reference samples when the left line number and the upper line number are determined differently.

[0244] Referring to Fig. 16, the upper reference line is the first reference line, and the left reference line is the fourth reference line. In this case, the upper first reference line and the left fourth reference line are connected without gaps, allowing for smooth coverage of intra-prediction directions. Through this, errors that may occur in specific intra-prediction directions when the left line number and the upper line number are determined differently can be avoided.

[0245] Referring again to FIG. 15, even when multiple reference lines as described above are used, some of the surrounding reference samples may not yet be decoded / restored or may not be available. In this case, the coding device may construct the surrounding reference samples to be used for intra-prediction by padding or substituting the unavailable samples with the available samples. The reference line index may be denoted as rlidx.

[0246] The upper reference samples of the n-th reference sample line can be denoted, for example, as AboveRow[i], where i can have values ​​ranging from 0 to W+H-1. If the upper reference samples are unavailable, and if the left reference samples are available, one of the left reference samples can be used as the sample value for the upper reference samples. Specifically, for example, the unavailable upper reference samples can be padded or replaced with the uppermost reference sample among the left adjacent reference samples. In other words, the unavailable upper reference samples can be padded or replaced with the left reference sample at the coordinate (x-1-rlidx, y). Here, (x, y) can represent the upper-left sample position of the current block (e.g., coding block or transformation block). If the upper reference samples are unavailable and the left reference samples are also unavailable, the upper reference samples can be padded or replaced with a default value derived based on a predetermined bit depth. Here, the above default value may be set to, for example, (1<<(BitDepth-1))-1. Alternatively, unavailable upper reference samples may be padded or replaced with the upper-left reference sample at coordinates (x-1, y-1-rlidx).

[0247] If upper reference samples are available, the restored sample values ​​of the upper reference samples are basically used as the values ​​of the corresponding upper reference samples. Meanwhile, if upper reference samples are available, a first parameter may be determined. The first parameter may be determined, for example, as x+2W-1. If H > W, the restored sample values ​​are not used for specific upper reference samples having an x-coordinate greater than the first parameter, and for said specific upper reference samples, the value of the upper reference sample having the x-coordinate of the first parameter is copied (padding or replaced). That is, for said specific upper reference samples, the value of the upper reference sample at the coordinate (first parameter, y-1) may be copied (padding or replaced). Alternatively, if H > W, for upper reference samples after the 2Wth upper reference sample, the value of the 2Wth upper reference sample may be copied (padding or replaced).

[0248] Left reference samples can be denoted, for example, as LeftCol[i], where i can have values ​​ranging from 0 to W+H-1. If left samples are unavailable, and if upper reference samples are available, one of the upper reference samples can be used as the sample value for the said left reference samples. Specifically, for example, unavailable left reference samples can be padded or replaced to be equal to the leftmost reference sample among the upper adjacent reference samples. In other words, unavailable left reference samples can be padded or replaced to be equal to the upper reference sample at coordinates (x, y-1-rlidx). Here, (x, y) can represent the upper-left sample position of the current block (e.g., coding block or transformation block). If left reference samples are unavailable and upper reference samples are also unavailable, the said left reference samples can be padded or replaced with a default value derived based on a predetermined bit depth. Here, the above default value may be set to, for example, (1<<(BitDepth-1))-1. Alternatively, unavailable left reference samples may be padded or replaced with the upper-left reference sample at coordinates (x-1-rlidx, y-1). Unavailable left reference samples may be padded or replaced with the upper-left reference sample at reference line 1.

[0249] If left reference samples are available, the restored sample values ​​of the left reference samples are used as the values ​​of the corresponding left reference samples by default. Meanwhile, if left reference samples are available, a second parameter may be determined. The second parameter may be determined, for example, as x+2H-1. If W>H, the restored sample values ​​are not used for specific left reference samples having a y-coordinate greater than the second parameter, and for said specific left reference samples, the value of the left reference sample having the y-coordinate of the second parameter is copied (padding or replaced). That is, for said specific left reference samples, the value of the left reference sample at the coordinate (x-1, second parameter) may be copied (padding or replaced). Alternatively, if W>H, for left reference samples after the 2Hth left reference sample, the value of the 2Hth left reference sample may be copied (padding or replaced).

[0250] The upper-left reference samples may be denoted as AboveRow[-i] for convenience, where i can range from 1 to k (e.g., k is 4). If the upper-left reference sample is available, the restored sample value of that reference sample is used. If the upper-left reference sample is not available, the value of the left reference sample at the available coordinates (x-1-rlidx, y) or the upper reference sample at the available coordinates (x, y-1-rlidx) may be copied (padding or replaced). If the upper-left reference sample is not available, and neither the left reference sample nor the upper reference sample is available, the upper reference samples may be padded or replaced with a default value derived based on a predetermined bit depth. Here, the default value may be set to, for example, (1<<(BitDepth-1))-1.

[0251] Meanwhile, if the left line number and the top line number are determined differently, the reference sample used for padding may be configured differently.

[0252] For example, if the left line number and the upper line number are determined differently, the available reference samples used for padding may be reference samples on the same reference line. Specifically, for example, if the upper reference line is determined as reference line 1 and the left reference line is determined differently as reference line n based on rlidx, the left reference sample of the left reference line 1 is used for padding the reference sample of the upper reference line 1. For example, the left reference sample at the (x-1, y) coordinate of the left reference line 1 can be used for padding the reference sample of the upper reference line 1. If the upper-left reference sample of the 1st reference line is not available, the value of the left reference sample at the (x-1, y) coordinate of the left reference line 1 or the upper reference sample at the (x, y-1) coordinate of the upper reference line 1 can be copied (padding or replaced).

[0253] As another example, if the left line number and the upper line number are determined differently, the available reference samples used for padding may be reference samples from other reference lines. Specifically, for example, if the upper reference line is determined as reference line 1 and the left reference line is determined differently as reference line n based on rlidx, the left reference sample of the left reference line n is used for reference sample padding of the upper reference line 1. For example, the left reference sample at the coordinate (x-1-rlidx, y) of the left reference line n can be used for reference sample padding of the upper reference line 1. If the upper-left reference sample of the 1st reference line is not available, the value of the left reference sample at the coordinate (x-1-rlidx, y) of the left reference line n or the upper reference sample at the coordinate (x, y-1-rlidx) of the upper reference line 1 can be copied (padding or replaced).

[0254] Meanwhile, multi-reference line selection may be applied to only some of the directional prediction modes. For example, the multi-reference line selection may be applied only when the angle delta value described above is not zero. Alternatively, the multi-reference line selection may be applied when the intra prediction mode is not a vertical mode and a horizontal mode.

[0255] Meanwhile, depending on the case, two or more reference lines may be selected. For example, a multi-reference line index can represent, for instance, the following.

[0256] Reference Line Index Description Reference Line 01 Reference Line 12 Reference Line 23 Reference Line 34 Reference Line 41 and 2 Reference Line 52 and 3 Reference Line 63 and 4 Reference Line 71 and 4

[0257] When two or more reference lines are selected as described above, a first prediction block according to the intra prediction direction is generated for the selected n-th reference line, a second prediction block according to the intra prediction direction is generated for the selected k-th reference line, and a final prediction block is generated based on the weighted sum of the first prediction block and the second prediction block.

[0258] Figure 17 shows an example of generating a final prediction block based on the weighted sum of two prediction blocks based on multiple reference lines according to an intra prediction mode.

[0259] Referring to FIG. 17, reference line 1 and reference line 2 are selected, and according to the derived intra prediction mode (intra prediction direction), a first prediction can be performed based on reference line 1 and a second prediction can be performed based on reference line 2 to obtain a weighted sum.

[0260] In addition, if the intra prediction direction points to a fractional sample location, a prediction sample can be generated through a 2D interpolation filter based on two reference samples of the n-th reference line and two reference samples of the k-th reference line around the fractional sample location.

[0261] Figure 18 shows an example of intra-predicted sample generation through 2D interpolation.

[0262] Referring to FIG. 18, when multiple reference lines are selected, a prediction sample can be generated by interpolating four or more reference samples located around the intra prediction direction relative to the target sample, thereby improving intra prediction performance.

[0263] Meanwhile, filtering can be performed on the derived surrounding reference samples before generating intra-prediction samples. Depending on the case, a 3-tap filter or a 5-tap filter may be applied for filtering the reference samples. For example, an edge filter strength may be derived based on the block size and / or angle delta value, and a filtering procedure may be performed based on the edge filter strength. If the edge filter strength is 0, filtering may be omitted. Such reference sample filtering may be performed on a reference line basis, or cross-line filtering may be performed. That is, filtering using samples from multiple reference lines may be performed.

[0264] Figure 19 shows an example of cross-line filtering for reference samples.

[0265] Referring to FIG. 19, as illustrated in FIG. 19 (a) and (b), filtering can be performed on reference samples of the second or third reference line using four surrounding reference samples located above, below, left, and right of the target reference sample. Specifically, for example, if the target reference sample is located at reference line 2, two surrounding reference samples of reference line 2, one reference sample of reference line 3, and one reference sample of reference line 1 can be used for filtering. Specifically, for example, if the target reference sample is located at reference line 3, two surrounding reference samples of reference line 3, one reference sample of reference line 4, and one reference sample of reference line 2 can be used for filtering.

[0266] As illustrated in FIG. 19 (c) and (d), for reference samples of reference line 1 or 4, filtering can be performed using three reference samples surrounding the target reference sample. Specifically, for example, if the target reference sample is located at reference line 1, two reference samples surrounding reference line 1 and one reference sample of reference line 2 may be used for filtering. Specifically, for example, if the target reference sample is located at reference line 4, two reference samples surrounding reference line 4 and one reference sample of reference line 3 may be used for filtering.

[0267] Whether cross-line filtering is allowed for the reference samples mentioned above can be signaled via higher-level syntax (e.g., sequence header, frame header, or tile header). If cross-line filtering is allowed, cross-line filtering is performed as described above, and if cross-line filtering is not allowed, line-by-line filtering can be performed.

[0268] Meanwhile, the number of tabs and / or type of the interpolation filter can be set differently for each reference line. For example, a k-tab interpolation filter may be applied to reference line 1, and an n-tab interpolation filter may be applied to reference line 2. For example, k can be 4 or 6, and n can be 2, 3, or 5. In this case, the k-tab interpolation filter may be a combined filter (CF) and the k-tab interpolation filter may be a bilinear filter (BF). For example, when performing prediction according to the intra-prediction direction after reference sample filtering is applied, an interpolation filter may be applied if the intra-prediction direction points to a fractional sample position. Meanwhile, reference sample filtering may be omitted when the k-tab interpolation filter is applied.

[0269] Meanwhile, if a reference line other than reference line 1 is selected through multiple reference line selection, a first prediction block using a non-directional intra prediction mode can be generated for reference line 1, and a second prediction block using a directional intra prediction mode can be generated for the selected reference line. In this case, a final prediction block can be generated through the weighted sum of the first prediction block and the second prediction block. For example, the non-directional intra prediction mode may include a DC intra prediction mode and / or a SMOOTH intra prediction mode. The SMOOTH intra prediction mode may include SMOOTH_PRED, SMOOTH_V_PRED, and / or SOOTH_H_PRED. In SMOOTH_V_PRED, upper peripheral samples in the vertical direction are used, and a prediction sample can be generated by interpolation (or distance-based weighted average) by further using the left peripheral sample located at the bottom among the left adjacent peripheral samples. In SMOOTH_H_PRED, prediction samples can be generated by interpolation (or distance-based weighted average) using horizontal left-side neighbor samples, while additionally utilizing the upper-side neighbor sample located furthest to the right among the upper-side adjacent neighbor samples. In SMOOTH_PRED, prediction samples can be generated based on bidirectional interpolation of the vertical upper-side neighbor sample, the left-side neighbor sample located furthest to the bottom among the left-side adjacent neighbor samples, the horizontal left-side neighbor sample, and the upper-side neighbor sample located furthest to the right among the upper-side adjacent neighbor samples. Whether to combine with a prediction block using the above-mentioned non-directional intra-prediction mode can be signaled through additional information. The above additional information may include IBP information.For example, if the value of the reference line index is greater than 0 and IBP is applied to the current block, as described above, a first prediction block is generated using a non-directional intra prediction mode for the first reference line, and a second prediction block is generated using a directional intra prediction mode for the selected reference line, and the first prediction block and the second prediction block can be weighted summed.

[0270] Meanwhile, as described above, when MRLS is applied (or when the reference line index is greater than 0), the intra prediction mode of the current block may be signaled among the intra prediction modes within a predetermined set. For example, the intra prediction mode for the current block may be determined only from the first mode set among a plurality of mode sets, in which case the mode set index information may not be explicitly coded / signaled. In this case, the decoding device may consider the first mode set to be selected without explicitly parsing / signaling the mode set index information. The first mode set may correspond to the mode set having the highest priority (rank) indicated when the value of the mode set index information is 0. In this case, without signaling the mode set index, the index of the intra prediction mode applied to the current block within the first mode set may be indicated based on the signaling of the mode index information. In this case, the complexity may be reduced, the number of bits required for mode signaling may be reduced, and the intra prediction efficiency may be increased by using a high-priority intra prediction mode.

[0271] Meanwhile, depending on the case, the reference line candidates may be configured differently. That is, a reference line candidate set can be adaptively determined from among multiple reference line candidate sets.

[0272] For example, in the case of the first reference line candidate set, reference line 1, reference line 3, and reference line 5 may be designated as candidates based on reference line indices 0 to 2, and in the case of the second reference line candidate set, reference line 2, reference line 4, and reference line 6 may be designated as candidates based on reference line indices 0 to 2. For example, the same line index value may be used, but the line candidates may be set differently based on the parity value. That is, among multiple reference line candidate sets, the reference line candidate set can be determined based on the parity value.

[0273] The above parity value may be implicitly derived or explicitly signaled through a bitstream. For example, the parity value may be signaled through a parity flag, and the signaling may be omitted depending on specific conditions. The above specific conditions may include whether the block size is within a certain range, whether the block is square or non-square, etc. If the signaling of the above parity flag is omitted, it may be derived as a default value of 0.

[0274] In addition to or generally in addition to the specific conditions above, a parity flag for the MRLS may be determined / signaled based on the value of the line index for the MRLS. For example, the parity flag may be signaled when the line index is greater than 0.

[0275]

[0276] Here, reference_line represents reference line selection information (i.e., reference line index), and line_parity represents a parity flag for the MRLS. If the value of reference_line is greater than 0, the line_parity may be explicitly signaled.

[0277] As described above, when the parity flag for the MRLS is determined / signaled based on the value of the line index for the MRLS, for example, the reference line candidates can be configured as follows.

[0278] For the first reference line candidate set, reference line 1, reference line 3, and reference line 5 are included as candidates based on reference line indices 0 to 2, and for the first reference line candidate set, reference line 2 and reference line 4 are included as candidates based on reference line indices 1 to 2. Here, when the reference line index is 0, the parity flag is implicitly derived as 0. Therefore, the number of line candidates when the parity flag is 0 is different from the number of line candidates when the parity flag is 1. Specifically, the number of line candidates when the parity flag is 1 may be one less than the number of line candidates when the parity flag is 0. By setting the reference line candidates differently as described above, various line candidates can be adaptively indicated, and signaling efficiency can be increased because candidates can be implicitly determined according to conditions.

[0279] Meanwhile, for coding blocks that touch or cross the frame boundary, it is determined whether they are the right or bottom boundary of the frame, and (symmetric / asymmetric) horizontal / vertical partitioning can be implicitly derived without partition information signaling. In this case, for coding blocks that touch or cross the frame boundary, it is determined whether they are the right or bottom boundary of the frame, and based on this, intra prediction mode candidates can be set differently. That is, if the frame boundary is the right boundary, intra prediction modes with an upper-right directionality (or intra prediction modes with a directionality to the right more than the vertical mode) can be excluded from the candidates, and if the frame boundary is the bottom boundary, intra prediction modes with a lower-left directionality (or intra prediction modes with a directionality to the bottom more than the horizontal mode) can be excluded from the candidates. Through this, the signaling efficiency of intra prediction modes can be improved.

[0280] Figure 20 shows an example where a coding block extends beyond the boundaries of a frame.

[0281] Referring to FIG. 20, the resolution of the frame may not match the ratio of block division, and a specific coding block may extend beyond the frame boundaries when considering the size according to the original partition structure. In such cases, it is not necessary to perform signaling of partition information by considering all partitioning candidates.

[0282] For example, if a coding block touches or extends beyond the right boundary of a frame, a quad partition or (symmetric / asymmetric) vertical partition can be derived for that coding block without partition information signaling. For example, if a coding block touches or extends beyond the bottom boundary of a frame, a quad partition or (symmetric / asymmetric) horizontal partition can be derived for that coding block without partition information signaling. For example, if a coding block touches or extends beyond the right and bottom boundaries of a frame, a quad partition can be derived for that coding block without partition information signaling.

[0283] The partitioning structure can also be derived without partition information signaling by further considering the size of the coding block. For example, if a coding block touches or extends beyond the right boundary of a frame, it can be implicitly derived as a quad partition if the block is square, or as a (symmetric / asymmetric) vertical partition if the block is non-square. For example, if a coding block touches or extends beyond the bottom boundary of a frame, it can be implicitly derived as a quad partition if the block is square, or as a (symmetric / asymmetric) horizontal partition if the block is non-square.

[0284] Figures 21 and 22 show examples of division where the coding block extends beyond the boundaries of the frame.

[0285] Referring to FIG. 21, one of the aforementioned symmetric / asymmetric vertical (binary) partitions can be implicitly derived based on where the right boundary of the frame is located within the coding block.

[0286] For example, if the right boundary of the frame is located within the first range of the coding block, a (symmetric) vertical (binary) partition can be implicitly derived for the coding block. If the right boundary of the frame is located within the second range of the coding block, a vertical A partition can be implicitly derived for the coding block. If the right boundary of the frame is located within the third range of the coding block, a vertical B partition can be implicitly derived for the coding block. In this case, for example, the first range may be greater than or equal to W / 2 and less than 3W / 4 of the coding block based on the x-coordinate. For example, the second range may be less than W / 2 of the coding block based on the x-coordinate. For example, the third range may be greater than or equal to 3W / 4 (and less than W) of the coding block based on the x-coordinate. Here, W may represent the width of the coding block. W may be equal to 2N.

[0287] For example, if the right boundary of the frame is located within the first range of the coding block, a (symmetric) vertical (binary) partition can be implicitly derived for the coding block. If the right boundary of the frame is located within the second range of the coding block, a vertical A partition can be implicitly derived for the coding block. If the right boundary of the frame is located within the third range of the coding block, a vertical B partition can be implicitly derived for the coding block. If the right boundary of the frame is located within the fourth range of the coding block, a vertical a partition can be implicitly derived for the coding block. If the right boundary of the frame is located within the fifth range of the coding block, a vertical b partition can be implicitly derived for the coding block. In this case, for example, the first range may be greater than W / 2 and less than 3W / 4 of the coding block based on the x-coordinate. For example, the second range may be greater than W / 4 and less than W / 2 of the coding block based on the x-coordinate. For example, the third range may be 3W / 4 or greater and less than 7W / 8 of the coding block based on the x-coordinate. For example, the fourth range may be less than W / 4 of the coding block based on the x-coordinate. For example, the fifth range may be 7W / 8 or greater (and less than W) of the coding block based on the x-coordinate. Here, W may represent the width of the coding block. W may be equal to 2N.

[0288] Referring to FIG. 22, one of the aforementioned symmetric / asymmetric horizontal (binary) partitions can be implicitly derived based on where the lower boundary of the frame is located within the coding block.

[0289] For example, if the lower boundary of the frame is located within the first range of the coding block, a (symmetric) horizontal (binary) partition can be implicitly derived for the coding block. If the lower boundary of the frame is located within the second range of the coding block, a horizontal A partition can be implicitly derived for the coding block. If the lower boundary of the frame is located within the third range of the coding block, a horizontal B partition can be implicitly derived for the coding block. In this case, for example, the first range may be greater than or equal to W / 2 and less than 3W / 4 of the coding block based on the x-coordinate (horizontal). For example, the second range may be less than W / 2 of the coding block based on the x-coordinate. For example, the third range may be greater than or equal to 3W / 4 (and less than W) of the coding block based on the x-coordinate. Here, W may represent the width of the coding block. W may be equal to 2N.

[0290] For example, if the lower boundary of the frame is located within the first range of the coding block, a (symmetric) horizontal (binary) partition can be implicitly derived for the coding block. If the lower boundary of the frame is located within the second range of the coding block, a horizontal A partition can be implicitly derived for the coding block. If the lower boundary of the frame is located within the third range of the coding block, a horizontal B partition can be implicitly derived for the coding block. If the lower boundary of the frame is located within the fourth range of the coding block, a horizontal a partition can be implicitly derived for the coding block. If the lower boundary of the frame is located within the fifth range of the coding block, a horizontal b partition can be implicitly derived for the coding block. In this case, for example, the first range may be greater than or equal to H / 2 and less than 3H / 4 of the coding block based on the y-coordinate (vertical). For example, the second range may be H / 4 or greater and less than H / 2 of the coding block based on the y-coordinate. For example, the third range may be 3H / 4 or greater and less than 7H / 8 of the coding block based on the y-coordinate. For example, the fourth range may be less than H / 4 of the coding block based on the y-coordinate. For example, the fifth range may be 7H / 8 or greater (and less than H) of the coding block based on the y-coordinate. Here, H may represent the height of the coding block. H may be equal to 2N.

[0291] Through the partitioning structure described above, a block partitioning structure that minimizes invalid areas to be processed can be efficiently derived while minimizing explicit signaling.

[0292] Meanwhile, to limit redundant partitioning, partitioning structures that can be directly derived from a higher-level coding block may be excluded from candidates when signaling partitioning information in a lower-level coding block.

[0293] Meanwhile, the luminance component block and the chroma component block may have the same partitioning structure, or they may have a semi-independent partitioning structure. For example, the same luminance / chroma partitioning structure may be applied up to a first range (e.g., 128×128) for the size of the coding block for the luminance component, and independent luminance / chroma partitioning structures may be applied in a second range that is smaller than that. The same partitioning structure may be applied between the chroma component blocks (U component block, V component block).

[0294] Figure 23 shows an example of a semi-independent partitioning structure indication.

[0295] Referring to FIG. 23, the same partitioning structure can be applied to the luminance component block and the chroma component block up to the first range of the size of the coding block for the luminance component. In this case, the partitioning structure of the luminance component block and the chroma component block can be derived based on a single partition information. Meanwhile, when the size of the coding block for the luminance component falls within the second range or is less than that, independent luminance / chroma partitioning structures can be applied. In this case, the luminance partition information and the chroma partition information can be signaled individually and may have different values. Meanwhile, when the size of the coding block for the luminance component falls within the third range, the available partitioning candidates may be changed. Through such hierarchical semi-independent partition structure signaling, the optimal coding block size can be efficiently signaled.

[0296] The decision on whether to use the aforementioned semi-independent partitioning structure may further consider the frame type. Frame type information can be signaled in the frame header or tile header.

[0297] Various frame types can be used for efficient compression and random access. Each frame type can be classified according to the reference method and compression method. The following table shows examples of frame types.

[0298] frame_typedescription0KEY_FRAME1INTER_FRAME2INTRA_ONLY_FRAME3SWITCH_FRAME

[0299] A Key Frame (KEY_FRAME) represents a frame used to start a new sequence of video. A Key Frame can be coded independently without a previous frame (reference frame). An Inter Frame (INTER_FRAME) represents a frame coded by prediction based on the previous frame (Reference Frame). Inter-prediction and intra-prediction can be used in Inter Frames. An Intra-Only Frame (INTRA_ONLY_FRAME) is similar to a Key Frame but represents a frame that performs only intra-coding while maintaining the existing reference frame. A Switch Frame (SWITCH_FRAME) is similar to an Intra-Only Frame but represents a frame designed to allow random access. A Switch Frame can start a new sequence while maintaining the existing reference frame.

[0300] For example, when the frame type is a KEY_FRAME or an INTRA_ONLY_FRAME, as described above, the same partitioning structure for luminance and chroma is applied up to the first range (e.g., 128×128) for the size of the coding block for the luminance component, and an independent partitioning structure for luminance and chroma may be applied in the second range, which is smaller than that. When the frame type is an INTER_FRAME or a SWITCH_FRAME, the luminance component block and the chroma component block may have the same partitioning structure regardless of size.

[0301] For example, from a signaling perspective, if the frame type is a key frame (KEY_FRAME) or an intra-only frame (INTRA_ONLY_FRAME), partition information is signaled up to the first range of the size of the coding block (including the superblock), and if the size of the coding block is the second range, luminance partition information and chroma partition information can be signaled, respectively.

[0302] Through hierarchical semi-independent partition structure signaling considering the frame type described above, the optimal coding block size according to frame characteristics can be efficiently signaled.

[0303] According to the above-described embodiment(s), partitioning information can be efficiently signaled by considering block size and image characteristics. Additionally, a partition structure can be efficiently derived for coding blocks that extend beyond the frame boundary without signaling partition information. Furthermore, the optimal coding block size according to frame characteristics can be efficiently signaled through hierarchical semi-independent partition structure signaling that considers the frame type.

[0304] FIG. 24 schematically illustrates a video / image encoding method according to an embodiment(s) of the present disclosure. The method disclosed in FIG. 24 may be performed by the encoding device disclosed in FIG. 2. Specifically, for example, S2400 to S2420 of FIG. 24 may be performed by the image segmentation unit (210) of the encoding device (200), and S2430 of FIG. 24 may be performed by the entropy encoding unit (240) of the encoding device (200). The method disclosed in FIG. 24 may include the embodiments described above in the present disclosure.

[0305] Referring to FIG. 24, the encoding device derives a partitioning structure for a coding block (S2400). The partitioning structure may be indicated based on one of the various partitioning candidates described above in the present disclosure. The coding block may include a superblock.

[0306] The encoding device can derive the optimal partitioning candidate for the coding block based on the RD cost for various partitioning candidates.

[0307] The encoding device derives the current coding block based on the partitioning structure (S2410). The encoding device may derive the current coding block by applying the partitioning structure to the coding block. In this case, depending on the case, the partitioning structure may be derived and applied recursively to finally derive the current coding block.

[0308] The encoding device generates partition information (S2420). The encoding device can generate partition information that indicates one of the various partitioning candidates described above based on the partitioning structure. The partition information may indicate one of a plurality of partitioning candidates based on the block size. As described above, the partition information may include information that is commonly applied to the luminance coding block and the chroma coding block. Alternatively, the partition information may include luminance partition information and chroma partition information, wherein the luminance partition information is applied to the luminance coding block and the chroma partition information is applied to the chroma coding block.

[0309] The above coding block may be a luminance component coding block. If the block size of the above luminance component coding block falls within a first range, the same partitioning structure may be derived for the above luminance component coding block and the above chroma component coding block. If the block size of the above luminance component coding block falls within a second range, independent partitioning structures may be derived for each of the above luminance component coding block and the above chroma component coding block. If the block size of the above luminance component coding block falls within a third range, independent partitioning structures are derived for each of the above luminance component coding block and the above chroma component coding block, and if the block size of the above luminance component coding block falls within a first range, first partitioning candidates are available, and if the block size of the above luminance component coding block falls within a third range, second partitioning candidates are available, and the number of the second partitioning candidates may be different from the number of the first partitioning candidates. When the block size of the above-mentioned luminous component coding block falls within a third range, an independent partitioning structure is derived for each of the above-mentioned luminous component coding block and the above-mentioned chroma component coding block, and when the block size of the above-mentioned luminous component coding block falls within a first range, first partitioning candidates are available, and when the block size of the above-mentioned luminous component coding block falls within a second range, first partitioning candidates are available, and when the block size of the above-mentioned luminous component coding block falls within a third range, second partitioning candidates are available, and the number of the second partitioning candidates may be greater than the number of the first partitioning candidates.

[0310] The encoding device can determine whether to apply semi-independent partitioning based on the frame type. Based on the case where the semi-independent partitioning is applied, if the block size of the luminance component coding block falls within a first range, the same partitioning structure is derived for the luminance component coding block and the chroma component coding block, and if the block size of the luminance component coding block falls within a second range, an independent partitioning structure can be derived for each of the luminance component coding block and the chroma component coding block. For example, it may be determined that the semi-independent partitioning is applied based on the case where the frame type indicates a key frame. For example, it may be determined that the semi-independent partitioning is applied based on the case where the frame type indicates an intra-only frame. For example, it may be determined that the semi-independent partitioning is not applied based on the case where the frame type indicates a switch frame. For example, based on the case where the above frame type indicates an inter-frame, it may be determined that the above quasi-independent partitioning is not applied.

[0311] Based on the case where it is determined that the above-mentioned semi-independent partitioning is applied, if the block size of the above-mentioned luminance component coding block falls within a first range, the partition information is applied commonly to the above-mentioned luminance component coding block and the above-mentioned chroma component coding block, and if the block size of the above-mentioned luminance component coding block falls within a second range, the partition information includes luminance partition information and chroma partition information, the above-mentioned luminance partition information is applied to the above-mentioned luminance component coding block, and the above-mentioned chroma partition information can be applied to the above-mentioned chroma component coding block.

[0312] For example, if the above coding block crosses the boundary of the current frame, the boundary determines the boundary type of the current frame, and the boundary type is one of a right boundary, a lower boundary, or both a right boundary and a lower boundary, and the partitioning structure can be implicitly derived based on the boundary type.

[0313] For example, if the above coding block crosses the boundary of the current frame, the boundary type of the current frame is determined, and the boundary type is one of a right boundary, a bottom boundary, or both a right boundary and a bottom boundary, and the partitioning structure can be implicitly derived based on the boundary type and whether the coding block is a square block.

[0314] For example, one or more transformation blocks are divided from the current coding block, and if the size of the current coding block is larger than a specific size, it is implicitly divided into transformation blocks of a specific size, and transformation depth information for transformation block division is generated for the transformation blocks implicitly divided into the specific size and included in the image information.

[0315] The encoding device encodes image information including the partition information (S2430). The image information may further include the frame type information and transformation depth information described above. The encoding device may perform an encoding procedure (including a prediction procedure and a transformation procedure, etc.) for the current coding block to generate related information such as prediction information and transformation information, and the image information may further include the related information. The image information may be referred to as video information. The image information may include various information according to an embodiment of the present disclosure. For example, the image information may include information described above in the present disclosure.

[0316] Meanwhile, the above image information may include residual information. The above residual information is information regarding residual samples. The above residual information may include information regarding quantized transformation coefficients for the above residual samples.

[0317] Encoded video information can be output in the form of a bitstream. The bitstream can be transmitted to a decoding device via a network or a storage medium. For example, video data containing the bitstream can be transmitted to a decoding device by a transmission device (or transmission unit). In this case, the video data containing the bitstream can be transmitted to the decoding device via a streaming server.

[0318] In addition, as described above, the encoding device can generate a recovery frame (including recovery samples and a recovery block) based on the prediction samples and the residual samples. This is to ensure that the encoding device derives the same prediction results as those performed by the decoding device, thereby increasing coding efficiency. Accordingly, the encoding device can store the recovery frame (or recovery samples, recovery block) in memory and utilize it as a reference frame for inter-prediction. As described above, an in-loop filtering procedure, etc., may be further applied to the recovery frame.

[0319] According to the above-described embodiment(s), partitioning information can be efficiently signaled by considering block size and image characteristics. Additionally, a partition structure can be efficiently derived for coding blocks that extend beyond the frame boundary without signaling partition information. Furthermore, the optimal coding block size according to frame characteristics can be efficiently signaled through hierarchical semi-independent partition structure signaling that considers the frame type.

[0320] FIG. 25 schematically illustrates a video / image decoding method according to an embodiment(s) of the present disclosure. The method disclosed in FIG. 25 may be performed by the decoding device disclosed in FIG. 3. Specifically, for example, S2500 of FIG. 25 may be performed by the entropy decoding unit (310) of the decoding device (300), and S2510 to S2530 may be performed by the prediction unit (330) of the decoding device (300). The method disclosed in FIG. 25 may include the embodiments described above in the present disclosure.

[0321] Referring to FIG. 25, the decoding device obtains partition information through a bitstream (S2500). The decoding device may obtain image information including the partition information through the bitstream. The image information may further include prediction information, transformation information, and residual information, as described above. The partition information may indicate one of a plurality of partitioning candidates based on a block size.

[0322] When the block size of the above coding block is a first size, first partitioning candidates are available, and when the block size of the above coding block is a second size, second partitioning candidates are available. In this case, the number of the first partitioning candidates may differ from the number of the second partitioning candidates. In this case, based on the fact that the second size is smaller than the first size, the number of the second partitioning candidates may be greater than the number of the first partitioning candidates. At least one partitioning candidate among the first partitioning candidates may not be included among the second partitioning candidates.

[0323] When the block size of the above coding block is a first size, first partitioning candidates are available; when the block size of the above coding block is a second size, second partitioning candidates are available; and when the block size of the above coding block is a third size, third partitioning candidates may be available. In this case, the number of the first partitioning candidates is different from the number of the second partitioning candidates, and the number of the third partitioning candidates may be different from the number of the second partitioning candidates.

[0324] As described above, the plurality of partitioning candidates may include at least one of an asymmetric binary vertical partitioning candidate, an asymmetric binary horizontal partitioning candidate, a vertical H-shaped partitioning candidate, a horizontal H-shaped partitioning candidate, an asymmetric 4-partition vertical partitioning candidate, and an asymmetric 4-partition horizontal partitioning candidate.

[0325] The above block size is the block size of the sub-coding block derived after the above coding block is divided, and partitioning candidates that make the minimum value of the width and height of the above sub-coding block smaller than 4 may not be available.

[0326] The decoding device can further obtain frame type information from the bitstream.

[0327] The decoding device derives the partitioning structure of the coding block (S2510). The decoding device can derive the partitioning structure applied to the coding block based on the partition information. The coding block may include a superblock.

[0328] The above coding block may be a luminance component coding block. If the block size of the above luminance component coding block falls within a first range, the same partitioning structure may be derived for the above luminance component coding block and the above chroma component coding block. If the block size of the above luminance component coding block falls within a second range, independent partitioning structures may be derived for each of the above luminance component coding block and the above chroma component coding block. If the block size of the above luminance component coding block falls within a third range, independent partitioning structures are derived for each of the above luminance component coding block and the above chroma component coding block, and if the block size of the above luminance component coding block falls within a first range, first partitioning candidates are available, and if the block size of the above luminance component coding block falls within a third range, second partitioning candidates are available, and the number of the second partitioning candidates may be different from the number of the first partitioning candidates. When the block size of the above-mentioned luminous component coding block falls within a third range, an independent partitioning structure is derived for each of the above-mentioned luminous component coding block and the above-mentioned chroma component coding block, and when the block size of the above-mentioned luminous component coding block falls within a first range, first partitioning candidates are available, and when the block size of the above-mentioned luminous component coding block falls within a second range, first partitioning candidates are available, and when the block size of the above-mentioned luminous component coding block falls within a third range, second partitioning candidates are available, and the number of the second partitioning candidates may be greater than the number of the first partitioning candidates.

[0329] The decoding device can determine whether to apply semi-independent partitioning based on the frame type information. Based on the case where the semi-independent partitioning is applied, if the block size of the luminance component coding block falls within a first range, the same partitioning structure is derived for the luminance component coding block and the chroma component coding block, and if the block size of the luminance component coding block falls within a second range, an independent partitioning structure can be derived for each of the luminance component coding block and the chroma component coding block. For example, it may be determined that the semi-independent partitioning is applied based on the case where the frame type indicates a key frame. For example, it may be determined that the semi-independent partitioning is applied based on the case where the frame type indicates an intra-only frame. For example, it may be determined that the semi-independent partitioning is not applied based on the case where the frame type indicates a switch frame. For example, based on the case where the above frame type indicates an inter-frame, it may be determined that the above quasi-independent partitioning is not applied.

[0330] Based on the case where it is determined that the above-mentioned semi-independent partitioning is applied, if the block size of the above-mentioned luminance component coding block falls within a first range, the partition information is applied commonly to the above-mentioned luminance component coding block and the above-mentioned chroma component coding block, and if the block size of the above-mentioned luminance component coding block falls within a second range, the partition information includes luminance partition information and chroma partition information, the above-mentioned luminance partition information is applied to the above-mentioned luminance component coding block, and the above-mentioned chroma partition information can be applied to the above-mentioned chroma component coding block.

[0331] For example, if the above coding block crosses the boundary of the current frame, the boundary determines the boundary type of the current frame, and the boundary type is one of a right boundary, a lower boundary, or both a right boundary and a lower boundary, and the partitioning structure can be implicitly derived based on the boundary type.

[0332] For example, if the above coding block crosses the boundary of the current frame, the boundary type of the current frame is determined, and the boundary type is one of a right boundary, a bottom boundary, or both a right boundary and a bottom boundary, and the partitioning structure can be implicitly derived based on the boundary type and whether the coding block is a square block.

[0333] The decoding device derives the current coding block (S2520). The decoding device may derive the current coding block by applying the partitioning structure to the coding block. In this case, depending on the case, the partitioning structure may be derived and applied recursively to finally derive the current coding block.

[0334] The decoding device performs a decoding procedure for the current coding block (S2530). For example, the decoding device may perform a series of procedures, such as prediction (inter / intra) and transformation, on the derived current coding block to derive a prediction block and a residual block, and generate a restoration block (restoration frame) based on the prediction block and the residual block.

[0335] For example, one or more transformation blocks are divided from the current coding block, and if the size of the current coding block is larger than a specific size, it is implicitly divided into transformation blocks of a specific size, and transformation depth information for transformation block division may be explicitly signaled for the transformation blocks implicitly divided into the specific size. In this case, the image information may further include the transformation depth information.

[0336] For example, prediction samples for the current coding block can be generated based on the derived intra prediction mode and surrounding reference samples. The decoding device can generate restoration samples based on the prediction samples of the current block. For example, the decoding device can generate the restoration samples for the current block based on the residual samples for the current block and the prediction samples. The residual samples for the current block can be generated based on received residual information. Additionally, the decoding device can generate a restoration frame including the restoration samples, for example. As previously described, an in-loop filtering procedure, etc., may be further applied to the restoration frame.

[0337] According to the above-described embodiment(s), partitioning information can be efficiently signaled by considering block size and image characteristics. Additionally, a partition structure can be efficiently derived for coding blocks that extend beyond the frame boundary without signaling partition information. Furthermore, the optimal coding block size according to frame characteristics can be efficiently signaled through hierarchical semi-independent partition structure signaling that considers the frame type.

[0338] In the embodiments described above, methods are described based on flowcharts as a series of steps or blocks, but the embodiments are not limited to the order of the steps, and some steps may occur in a different order or simultaneously with other steps as described above. Furthermore, those skilled in the art will understand that the steps shown in the flowcharts are not exclusive, and other steps may be included, or one or more steps of the flowcharts may be omitted without affecting the scope of the embodiments of the present disclosure.

[0339] The method according to the embodiments of the present disclosure described above may be implemented in the form of software, and the encoding device and / or decoding device according to the present disclosure may be included in a device that performs image processing, such as a TV, computer, smartphone, set-top box, display device, etc.

[0340] The embodiments of the present disclosure described above may also be implemented in the form of a recording medium containing computer-executable (program) instructions, such as program modules executed by a computer. The module may be stored in memory and executed by a processor. The memory may be located inside or outside the processor and may be connected to the processor by various well-known means. A computer-readable medium may be any available medium accessible by a computer and includes both volatile and non-volatile media, and both removable and non-removable media. Additionally, a computer-readable medium may include both computer storage media and communication media. A computer storage medium includes both volatile and non-volatile, removable and non-removable media implemented by any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data. A communication medium typically includes computer-readable instructions, data structures, program modules, or other data of modulated data signals such as carrier waves, or other transmission mechanisms, and includes any information transmission medium.

[0341] Additionally, the embodiments of the present disclosure described above may be implemented as a computer program (or computer program product) comprising instructions executable by a computer. The computer program includes programmable machine instructions processed by a processor and may be implemented in a high-level programming language, an object-oriented programming language, an assembly language, or a machine language, etc. Additionally, the computer program may be recorded on a tangible computer-readable recording medium (e.g., memory, a hard disk, a magnetic / optical medium, or a Solid-State Drive (SSD), etc.).

[0342] Accordingly, the embodiments of the present disclosure described above may be implemented by executing a computer program as described above by a computing device. The computing device may include at least some of a processor, memory, a storage device, a high-speed interface connected to the memory and a high-speed expansion port, and a low-speed interface connected to a low-speed bus and a storage device. Each of these components may be connected to one another using various buses and may be mounted on a common motherboard or mounted in other suitable ways.

[0343] Here, the processor can process instructions within the computing device, such as instructions stored in memory or storage devices to display graphic information for providing a Graphic User Interface (GUI) on external input and output devices, such as a display connected to a high-speed interface. In another embodiment, multiple processors and / or multiple buses may be used appropriately with multiple memories and memory types. Additionally, the processor may be implemented as a chipset comprising chips including multiple independent analog and / or digital processors.

[0344] In addition, memory stores information within a computing device. For example, memory may consist of volatile memory units or a set thereof. As another example, memory may consist of non-volatile memory units or a set thereof. Furthermore, memory may be other forms of computer-readable media, such as magnetic or optical discs.

[0345] And the storage device can provide a large amount of storage space to the computing device. The storage device may be a computer-readable medium or a configuration containing such a medium, and may include, for example, devices or other configurations within a Storage Area Network (SAN), and may be a floppy disk device, a hard disk device, an optical disk device, or a tape device, flash memory, or other similar semiconductor memory device or device array.

[0346] In addition, the network can be implemented as a wired network such as a Local Area Network (LAN), Wide Area Network (WAN), or Value Added Network (VAN), or as a mobile radio communication network or a satellite communication network, among other types of wireless networks.

[0347] The present disclosure described above has been explained with reference to the embodiments illustrated in the drawings, but this is merely illustrative, and those skilled in the art will understand that various modifications and variations of the embodiments are possible therefrom. That is, the scope of the present disclosure is not limited to the embodiments described above, and various modifications and improvements by those skilled in the art using the basic concepts of the embodiments defined in the following claims also fall within the scope of the embodiments. Accordingly, the true technical scope of protection of the present disclosure should be determined by the technical concept of the appended claims.

Claims

1. In a video decoding method performed by a decoding device, A step of obtaining partition information through a bitstream; A step of deriving the partitioning structure of coding blocks based on partition information; A step of deriving a current coding block based on the above partitioning structure; and It includes a step of performing a decoding procedure for the above-mentioned current coding block, and An image decoding method characterized by the above partition information indicating one of a plurality of partitioning candidates based on block size.

2. In Paragraph 1, When the block size of the above coding block is a first size, first partitioning candidates are available, and when the block size of the above coding block is a second size, second partitioning candidates are available, and An image decoding method characterized in that the number of the first partitioning candidates is different from the number of the second partitioning candidates.

3. In Paragraph 2, A video decoding method characterized in that, based on the fact that the second size is smaller than the first size, the number of second partitioning candidates is greater than the number of first partitioning candidates.

4. In Paragraph 3, An image decoding method characterized in that at least one of the first partitioning candidates is not included in the second partitioning candidates.

5. In Paragraph 1, When the block size of the above coding block is a first size, first partitioning candidates are available, when the block size of the above coding block is a second size, second partitioning candidates are available, and when the block size of the above coding block is a third size, third partitioning candidates are available. An image decoding method characterized in that the number of the first partitioning candidates is different from the number of the second partitioning candidates, and the number of the third partitioning candidates is different from the number of the second partitioning candidates.

6. In Paragraph 1 An image decoding method characterized in that the plurality of partitioning candidates include at least one of an asymmetric binary vertical partitioning candidate, an asymmetric binary horizontal partitioning candidate, a vertical H-shaped partitioning candidate, a horizontal H-shaped partitioning candidate, an asymmetric 4-partition vertical partitioning candidate, and an asymmetric 4-partition horizontal partitioning candidate.

7. In Paragraph 6 The above coding block is a Luma component coding block, and When the block size of the above-mentioned luminous component coding block falls within the first range, the same partitioning structure is derived for the above-mentioned luminous component coding block and the chroma component coding block, and An image decoding method characterized by deriving an independent partitioning structure for each of the luminance component coding block and the chroma component coding block when the block size of the luminance component coding block falls within a second range.

8. In Paragraph 7 When the block size of the above-mentioned luminous component coding block falls within the third range, an independent partitioning structure is derived for each of the above-mentioned luminous component coding block and chroma component coding block, and A video decoding method characterized in that when the block size of the above-mentioned luminance component coding block falls within a first range, first partitioning candidates are available, and when the block size of the above-mentioned luminance component coding block falls within a third range, second partitioning candidates are available, and the number of the second partitioning candidates is different from the number of the first partitioning candidates.

9. In Paragraph 7 When the block size of the above-mentioned luminous component coding block falls within the third range, an independent partitioning structure is derived for each of the above-mentioned luminous component coding block and chroma component coding block, and A video decoding method characterized in that when the block size of the above-mentioned luminous component coding block falls within a first range, first partitioning candidates are available; when the block size of the above-mentioned luminous component coding block falls within a second range, first partitioning candidates are available; when the block size of the above-mentioned luminous component coding block falls within a third range, second partitioning candidates are available; and the number of the second partitioning candidates is greater than the number of the first partitioning candidates.

10. In Paragraph 7, Obtain frame type information from the above bitstream, and Based on the above frame type information, determine whether to apply semi-independent partitioning, and Based on the case where the above-mentioned semi-independent partitioning is applied, When the block size of the above-mentioned luminous component coding block falls within the first range, the same partitioning structure is derived for the above-mentioned luminous component coding block and the chroma component coding block, and An image decoding method characterized by deriving an independent partitioning structure for each of the luminance component coding block and the chroma component coding block when the block size of the luminance component coding block falls within a second range.

11. In Paragraph 10, A video decoding method characterized by determining that the above-mentioned semi-independent partitioning is applied based on the case where the above-mentioned frame type indicates a key frame.

12. In Paragraph 10, A video decoding method characterized by determining that the above-mentioned semi-independent partitioning is not applied based on the case where the above-mentioned frame type indicates a switch frame.

13. In Paragraph 10, Based on the case where it is determined that the above-mentioned semi-independent partitioning is applied, When the block size of the above-mentioned luminous component coding block falls within the first range, the partition information is applied commonly to the above-mentioned luminous component coding block and the chroma component coding block, and A video decoding method characterized in that, when the block size of the above-mentioned luminance component coding block falls within a second range, the partition information includes luminance partition information and chroma partition information, the luminance partition information is applied to the above-mentioned luminance component coding block, and the chroma partition information is applied to the above-mentioned chroma component coding block.

14. In Paragraph 10 An image decoding method characterized in that the first range above includes a size equal to the size of the superblock.

15. In Paragraph 1 The above block size is the block size of the sub-coding block derived after the above coding block is divided, and An image decoding method characterized by not having available partitioning candidates that make the minimum values ​​of the width and height of the above-mentioned sub-coding block less than 4.

16. In Paragraph 1, One or more transformation blocks are divided from the above current coding block, and If the size of the above current coding block is larger than a specific size, it is implicitly divided into conversion blocks of a specific size, and An image decoding method characterized by explicitly signaling transformation depth information for transformation block division for a transformation block implicitly divided into the above specific size.

17. In Paragraph 1, If the above coding block crosses the boundary of the current frame, the boundary determines the boundary type of the current frame, and the boundary type is one of the right boundary, the bottom boundary, and both the right boundary and the bottom boundary, and An image decoding method characterized by implicitly deriving the partitioning structure based on the above boundary type.

18. In Paragraph 1, If the above coding block crosses the boundary of the current frame, the boundary determines the boundary type of the current frame, and the boundary type is one of the right boundary, the bottom boundary, and both the right boundary and the bottom boundary, and An image decoding method characterized by implicitly deriving the partitioning structure based on the boundary type and whether the coding block is a square block.

19. A video encoding method performed by an encoding device, Step of deriving the partitioning structure of coding blocks; A step of deriving the current coding block based on the above partitioning structure; A step of generating partition information indicating the above partitioning structure; and The method includes the step of encoding video information containing the above partition information to generate a bitstream, An image encoding method characterized by the above partition information indicating one of a plurality of partitioning candidates based on block size.

20. In a method for transmitting image data, Acquiring a bitstream generated by a video encoding method, wherein the video encoding method comprises the steps of: deriving a partitioning structure of a coding block; deriving a current coding block based on the partitioning structure; generating partition information indicating the partitioning structure; and encoding video information including the partition information to generate a bitstream; and The method includes the step of transmitting image data including the bitstream above, A transmission method characterized by the above partition information indicating one of a plurality of partitioning candidates based on block size.

Images