Video signal encoding / decoding equipment, storage media, and data transmission equipment
By using QTBT structure and syntax element parsing, the problems of efficient adjustment block partitioning and signal notification are solved, thereby improving the compression efficiency of video encoding.
Patent Information
- Application Number
- CN202311851497.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2018-05-24
- Filing Date
- 2019-03-27
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2039-03-27
AI Technical Summary
Existing technologies suffer from low coding efficiency when processing video content with high spatial resolution, high frame rate, and high scene representation dimension, and it is difficult to efficiently adjust block partitioning and signal notification QT, BT, TT partitioning structure information.
The block partitioning is performed using the QTBT structure, and the partitioning mode of the current block is determined by parsing the syntactic elements. It supports QT, BT, and TT partitioning, and signals are used to notify relevant information to improve coding efficiency.
It improves the compression efficiency of video encoding by efficiently adjusting block partitioning and signal notification.
Smart Images

Figure CN117793383B_ABST
Abstract
Description
[0001] This application is a divisional application of the original invention patent application No. 201980035375.2 (International Application No.: PCT / KR2019 / 003606, Application Date: March 27, 2019, Invention Title: Method and Apparatus for Processing Images). Technical Field
[0002] This disclosure relates to a method for processing still images or video images, and more specifically, to a method for determining the partitioning structure of blocks for performing encoding / decoding, and an apparatus supporting the method. Background Technology
[0003] Compression coding refers to a series of signal processing techniques for transmitting digital information over communication lines or for storing information in a form suitable for storage media. Media including images, videos, and audio can be targets of compression coding; specifically, the technique of performing compression coding on images is called video image compression.
[0004] Next-generation video content should feature high spatial resolution, high frame rates, and high scene representation dimensions. Processing this content will lead to a dramatic increase in memory storage, memory access speeds, and processing power.
[0005] Therefore, there is a need to design an efficient encoding tool for processing next-generation video content. Summary of the Invention
[0006] Technical issues
[0007] The purpose of this disclosure is to provide a method for efficiently adjusting block partitioning when performing block partitioning using a QTBT structure.
[0008] Another objective of this disclosure is to propose a method for supporting QT partitioning after BT partitioning and TT partitioning.
[0009] Another objective of this disclosure is to propose a striped tree (ST) structure partitioning method for dividing non-square blocks into blocks of the same size.
[0010] Another objective of this disclosure is to propose a method for efficiently signaling QT, BT, and TT to divide structural information at the same syntactic level.
[0011] The technical problems to be solved by the present invention are not limited to those described above. Those skilled in the art can clearly understand other technical problems not mentioned above from the following description.
[0012] Technical solution
[0013] In one aspect of this disclosure, a video decoding method may include the following steps: when the current block meets pre-configured conditions, parsing a first syntactic element indicating whether the current block is divided into multiple sub-blocks; when the first syntactic element indicates that the current block is divided, parsing a second syntactic element indicating whether the current block is divided using a quadtree structure; when the second syntactic element indicates that the current block is not divided using a quadtree structure, parsing at least one of a third syntactic element and a fourth syntactic element, wherein the third syntactic element indicates whether the current block is divided using a binary tree structure or a ternary tree structure, and the fourth syntactic element indicates the division direction of the current block; and determining the division mode of the current block based on at least one of the first syntactic element, the second syntactic element, the third syntactic element, and the fourth syntactic element.
[0014] Preferably, when the first syntactic element indicates that the current block has not been divided, the method may further include invoking the encoding unit syntax for decoding processing of the current block.
[0015] Preferably, when the value of the first syntactic element is 0, the current block may not be divided, and when the value of the first syntactic element is 1, the current block may be divided into 4 sub-blocks using a quadtree structure, or into 2 sub-blocks using a binary tree structure, or into 3 sub-blocks using a ternary tree structure.
[0016] Preferably, the method may further include invoking the coding tree unit syntax to determine the partitioning structure of sub-blocks from the current block based on the partitioning pattern.
[0017] Preferably, the first, second, third, and fourth syntactic elements can be parsed from the same level of syntax.
[0018] Preferably, the pre-configured conditions are met when the horizontal coordinate of the top left sample of the current block plus the width of the current block is equal to or less than the width of the current screen, and when the vertical coordinate of the top left sample of the current block plus the height of the current block is equal to or less than the height of the current screen.
[0019] Preferably, the method may further include parsing a fifth syntactic element that indicates whether the current block is partitioned using a quadtree structure, whether the current block is partitioned using a binary tree structure or a ternary tree structure from the block partitioned by the higher node.
[0020] In another aspect of this disclosure, a video decoding device may include: a syntax element parsing unit configured to: when the current block meets pre-configured conditions, parse a first syntax element indicating whether the current block is divided into multiple sub-blocks; when the first syntax element indicates that the current block is divided, parse a second syntax element indicating whether the current block is divided using a quadtree structure; and when the second syntax element indicates that the current block is not divided using a quadtree structure, parse at least one of a third syntax element and a fourth syntax element, wherein the third syntax element indicates whether the current block is divided using a binary tree structure or a ternary tree structure, and the fourth syntax element indicates the division direction of the current block; and a partitioning mode determining unit configured to determine the partitioning mode of the current block based on at least one of the first, second, third, and fourth syntax elements.
[0021] Preferably, the device may further include an encoding unit syntax invocation unit, which is configured to invoke the encoding unit syntax for decoding processing of the current block when the first syntax element indicates that the current block has not been divided.
[0022] Preferably, when the value of the first syntactic element is 0, the current block may not be divided, and when the value of the first syntactic element is 1, the current block may be divided into 4 sub-blocks using a quadtree structure, or into 2 sub-blocks using a binary tree structure, or into 3 sub-blocks using a ternary tree structure.
[0023] Preferably, the device may further include a coding tree unit syntax invocation unit, which is configured to invoke coding tree unit syntax to determine the partitioning structure of sub-blocks from the current block based on the partitioning pattern.
[0024] Preferably, the first, second, third, and fourth syntactic elements can be parsed from the same level of syntax.
[0025] Preferably, the pre-configured conditions are met when the horizontal coordinate of the top left sample of the current block plus the width of the current block is equal to or less than the width of the current screen, and when the vertical coordinate of the top left sample of the current block plus the height of the current block is equal to or less than the height of the current screen.
[0026] Preferably, the syntactic element parsing unit can parse a fifth syntactic element indicating whether the current block, which is partitioned from the higher node using a binary tree structure or a ternary tree structure, is partitioned using a quadtree structure.
[0027] Beneficial effects
[0028] According to embodiments of this disclosure, the QTBT partitioning structure is determined efficiently, and relevant information is communicated via signals, thereby improving compression efficiency.
[0029] The effects that can be obtained by this disclosure are not limited to those described above. Other technical effects not described above will be readily apparent to those skilled in the art from the following description. Attached Figure Description
[0030] The accompanying drawings are included to provide a further understanding of the invention and form part of the detailed description. The drawings illustrate embodiments of the invention and, together with the description, serve to illustrate the principles of the invention.
[0031] Figure 1 This is a schematic block diagram of an encoding apparatus for encoding video / image signals according to an embodiment of the present disclosure.
[0032] Figure 2 This is a schematic block diagram of a decoding device for decoding video / image signals, which applies an embodiment of the present disclosure.
[0033] Figure 3 This is a diagram illustrating an example of a multi-type tree structure to which the present disclosure can be applied.
[0034] Figure 4 This is a diagram illustrating a signaling mechanism for partitioning information of a quadtree with nested multi-type tree structures, which is an embodiment to which this disclosure can be applied.
[0035] Figure 5 This is a diagram illustrating a method for partitioning a CTU into multiple CUs based on a quadtree and nested multi-type tree structure, as an embodiment to which this disclosure can be applied.
[0036] Figure 6 This is a diagram illustrating a method for restricting ternary tree partitioning as an embodiment to which this disclosure can be applied.
[0037] Figure 7 This is a diagram illustrating redundant partitioning patterns that can be generated in binary tree partitioning and ternary tree partitioning as embodiments to which this disclosure can be applied.
[0038] Figure 8 This is a diagram illustrating a method for partitioning blocks based on quadtrees and nested multi-type tree structures, as an embodiment of the present disclosure.
[0039] Figure 9 This is a diagram illustrating a quadtree-based block partitioning structure as an embodiment to which the present disclosure can be applied.
[0040] Figure 10 This is a diagram illustrating a binary tree-based block partitioning structure as an embodiment to which the present disclosure can be applied.
[0041] Figure 11This is a diagram illustrating a ternary tree-based block partitioning structure as an embodiment to which the present disclosure can be applied.
[0042] Figure 12 This is a diagram illustrating a block partitioning structure based on binary and ternary trees as an embodiment to which this disclosure can be applied.
[0043] Figure 13 This is a diagram illustrating a method for dividing blocks based on a striped tree structure, as an embodiment of the present disclosure.
[0044] Figure 14 This is a diagram showing the reference positions of the current block and neighboring blocks as embodiments to which this disclosure can be applied.
[0045] Figure 15 This is a flowchart illustrating a method for decoding a video signal according to an embodiment of the present disclosure.
[0046] Figure 16 This is a flowchart illustrating a video signal decoding device according to an embodiment of the present disclosure.
[0047] Figure 17 The video encoding system using this disclosure is shown.
[0048] Figure 18 This is a configuration diagram of a content streaming system that applies an embodiment of the present disclosure. Detailed Implementation
[0049] Some embodiments of this disclosure are described in detail with reference to the accompanying drawings. (The accompanying drawings are also included.) Figure 1 The detailed description disclosed herein is intended to describe some embodiments of the present disclosure, and not only embodiments thereof. The following detailed description includes further details to provide a complete understanding of the present disclosure. However, those skilled in the art will understand that the present disclosure can be made without these further details.
[0050] In some cases, to avoid obscuring the concepts of this disclosure, known structures and devices are omitted or may be shown in block diagram form based on the core functions of each structure and device.
[0051] Although most of the terms used in this disclosure are selected from general terms widely used in the art, some terms are arbitrarily chosen by the applicant, and their meanings are explained in detail in the following description as needed. Therefore, this disclosure should be understood in terms of their intended meaning rather than their simple names or meanings.
[0052] Specific terminology used in the following description is provided to aid in understanding this disclosure, and the use of these specific terms may be modified in various forms without departing from the technical spirit of this disclosure. For example, in various encoding processes, signals, data, samples, images, frames, blocks, etc., may be appropriately replaced and interpreted.
[0053] In this specification, a "processing unit" refers to a unit that performs encoding / decoding processes such as prediction, transformation, and / or quantization. Hereinafter, for ease of description, a processing unit may be referred to as a "processing block" or "block".
[0054] Furthermore, the term "processing unit" can be interpreted as including units for the luminance component and units for the chrominance component. For example, a processing unit may correspond to a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), or a transform unit (TU).
[0055] Furthermore, a processing unit can be interpreted as a unit for the luma component or a unit for the chroma component. For example, a processing unit may correspond to a coding tree block (CTB), coding block (CB), prediction unit (PU), or transform block (TB) for the luma component. Additionally, a processing unit may correspond to a CTB, CB, PU, or TB for the chroma component. Moreover, the processing unit is not limited to these interpretations and can be interpreted to include both units for the luma component and units for the chroma component.
[0056] In addition, the processing unit is not necessarily limited to a square block, but can be configured as a polygon shape with three or more vertices.
[0057] Furthermore, in this specification, pixels are referred to as samples. Additionally, using a sample can mean using pixel values, etc.
[0058] Figure 1 This is a schematic block diagram of an encoding apparatus for encoding video / image signals according to an embodiment of the present disclosure.
[0059] Reference Figure 1The encoding device 100 may be configured to include an image segmenter 110, a subtractor 115, a transformer 120, a quantizer 130, a dequantizer 140, an inverse transformer 150, an adder 155, a filter 160, a memory 170, an inter-frame predictor 180, an intra-frame predictor 185, and an entropy encoder 190. The inter-frame predictor 180 and the intra-frame predictor 185 may be collectively referred to as the predictor. In other words, the predictor may include the inter-frame predictor 180 and the intra-frame predictor 185. The transformer 120, the quantizer 130, the dequantizer 140, and the inverse transformer 150 may be included in a residual processor. The residual processor may also include the subtractor 115. In one implementation, the image segmenter 110, subtractor 115, transformer 120, quantizer 130, dequantizer 140, inverse transformer 150, adder 155, filter 160, inter-frame predictor 180, intra-frame predictor 185, and entropy encoder 190 may be configured as a single hardware component (e.g., an encoder or processor). Furthermore, the memory 170 may include a decoded picture buffer (DPB) and may be implemented using a digital storage medium.
[0060] Image segmenter 110 can segment an input image (or picture or frame) input to encoding device 100 into one or more processing units. For example, a processing unit may be referred to as a coding unit (CU). In this case, the coding unit may be recursively partitioned from coding tree unit (CTU) or maximum coding unit (LCU) based on a quadtree-binary tree (QTBT) structure. For example, a coding unit may be partitioned into multiple coding units of greater depth based on a quadtree structure and / or a binary tree structure. In this case, for example, a quadtree structure may be applied first, followed by a binary tree structure. Alternatively, a binary tree structure may be applied first. The encoding process according to this disclosure may be performed based on a final coding unit that is no longer partitioned. In this case, the maximum coding unit may be directly used as the final coding unit based on coding efficiency according to image characteristics, or, if necessary, the coding unit may be recursively partitioned into deeper coding units. Thus, a coding unit with the optimal size may be used as the final coding unit. In this case, the encoding process may include processes such as prediction, transformation, or reconstruction, as described later. For example, the processing unit may also include a prediction unit (PU) or a transformation unit (TU). In this configuration, each of the prediction and transform units can be segmented or partitioned from the respective final coding units. The prediction unit can be a unit used for sample prediction, and the transform unit can be a unit for deriving transform coefficients and / or deriving the residual signal from the transform coefficients.
[0061] Units can be used interchangeably with blocks or regions, depending on the situation. Typically, an M×N block can indicate a set of samples or transform coefficients configured with M columns and N rows. Samples can typically indicate pixels or pixel values, and can indicate only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chrominance component. Within a sample, a frame (or image) can be used as the term corresponding to a pixel.
[0062] Encoding device 100 generates a residual signal (residual block or residual sample array) by subtracting the prediction signal (prediction block or prediction sample array) output by inter-frame predictor 180 or intra-frame predictor 185 from the input image signal (original block or original sample array). The generated residual signal is sent to converter 120. In this case, as shown, the unit within encoding device 100 that subtracts the prediction signal (prediction block or prediction sample array) from the input image signal (original block or original sample array) may be referred to as subtractor 115. The predictor can perform prediction on the processing target block (hereinafter referred to as the current block) and can generate a prediction block that includes prediction samples of the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction in the current block or CU unit. The predictor can generate various information about the prediction (e.g., prediction mode information, which will be described later in the description of the various prediction modes) and can send the information to entropy encoder 190. The information about the prediction can be encoded in entropy encoder 190 and can be output in bitstream form.
[0063] Intra-frame predictor 185 can refer to samples within the current frame to predict the current block. The referenced samples can be adjacent to the current block or spaced apart from it depending on the prediction mode. In intra-frame prediction, the prediction mode can include multiple non-corner modes and multiple corner modes. For example, non-corner modes can include DC mode and planar mode. For example, depending on the fineness of the prediction direction, corner modes can include 33 corner prediction modes or 65 corner prediction modes. In this case, for example, depending on the configuration, more or fewer corner prediction modes than 33 or 65 corner prediction modes can be used. Intra-frame predictor 185 can use prediction modes applied to neighboring blocks to determine the prediction mode applied to the current block.
[0064] Inter-frame predictor 180 can deduce the predicted block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference frame. In this case, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted as a block, sub-block, or sample unit based on the correlation between motion information of neighboring blocks and the current block. Motion information may include motion vectors and reference frame indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction) information. In the case of inter-frame prediction, neighboring blocks may include spatially neighboring blocks within the current frame and temporally neighboring blocks within the reference frame. The reference frame including the reference block and the reference frame including the temporally neighboring block may be the same or different. The temporally neighboring block may be referred to as a co-located reference block or a co-located CU (colCU). The reference frame including the temporally neighboring block may be referred to as a co-located frame (colPic). For example, inter-frame predictor 180 can construct a motion information candidate list based on neighboring blocks and can generate information indicating which candidate is used to deduce the motion vector and / or reference frame index of the current block. Inter-frame prediction can be performed based on various prediction modes. For example, in skip mode and merge mode, the inter-frame predictor 180 can use motion information from neighboring blocks as motion information for the current block. In skip mode, unlike merge mode, residual signals may not be sent. In motion vector prediction (MVP) mode, motion vectors from neighboring blocks can be used as motion vector predictors. The motion vector of the current block can be indicated by signaling the motion vector difference.
[0065] The predicted signal generated by the inter-frame predictor 180 or the intra-frame predictor 185 can be used to generate the reconstructed signal or the residual signal.
[0066] Transformer 120 can generate transform coefficients by applying a transform scheme to the residual signal. For example, the transform scheme may include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loève Transform (KLT), Graphical Based Transform (GBT), or Conditional Nonlinear Transform (CNT). In this case, GBT refers to the transform obtained from a graph if the relationship information between pixels is represented as such. CNT refers to the transform obtained based on a prediction signal generated using all previously reconstructed pixels. Furthermore, the transform process can be applied to pixel blocks of the same size in a square form, or to blocks of variable size that are not square.
[0067] Quantizer 130 quantizes the transform coefficients and sends them to entropy encoder 190. Entropy encoder 190 encodes the quantized signal (information about the quantized transform coefficients) and outputs it as a bitstream. Information about the quantized transform coefficients may be referred to as residual information. Quantizer 130 can rearrange the quantized transform coefficients in block form as a one-dimensional vector based on the coefficient scan order, and can generate information about the quantized transform coefficients based on the one-dimensional vector form. Entropy encoder 190 can perform various encoding methods such as exponential Golomb, context-adaptive variable-length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). In addition to the quantized transform coefficients, entropy encoder 190 can encode information required for video / image reconstruction (e.g., values of syntactic elements) together or separately. The encoded information (e.g., encoded video / image information) can be sent as a bitstream or stored in a Network Abstraction Layer (NAL) unit. The bitstream can be sent via a network or stored in a digital storage medium. In this case, the network may include a broadcast network and / or a communication network. Digital storage media may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. A transmitter (not shown) that transmits the signal output by the entropy encoder 190 and / or a storage device (not shown) for storing the signal may be configured as an internal / external component of the encoding device 100, or the transmitter may be a component of the entropy encoder 190.
[0068] The quantized transform coefficients output by quantizer 130 can be used to generate a prediction signal. For example, the residual signal can be reconstructed by applying dequantization and inverse transform to the quantized transform coefficients within a loop using dequantizer 140 and inverse transformer 150. Adder 155 can add the reconstructed residual signal to the prediction signal output by inter-frame predictor 180 or intra-frame predictor 185, thus generating a reconstructed signal (reconstructed frame, reconstructed block, or reconstructed sample array). If there is no residual for the processing target block, as in the case where skip mode has been applied, the prediction block can be used as a reconstructed block. Adder 155 can be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next processing target block in the current frame and can be filtered for inter-frame prediction of the next frame (as described later).
[0069] Filter 160 can improve subjective / observable image quality by applying filtering to the reconstructed signal. For example, filter 160 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image. The modified reconstructed image can be stored in memory 170, and more specifically, in the DPB of memory 170. For example, various filtering methods may include deblocking filtering, sample adaptive offset, adaptive loop filtering, and bilateral filtering. Filter 160 can generate various information for filtering (as described later in the description of the various filtering methods) and can send it to entropy encoder 190. The filtering information can be encoded by entropy encoder 190 and output as a bitstream.
[0070] The modified reconstructed frame sent to memory 170 can be used as a reference frame in inter-frame predictor 180. If inter-frame prediction is applied, the encoding device can avoid prediction mismatch in encoding device 100 and decoding device and improve encoding efficiency.
[0071] The DPB of memory 170 can store modified reconstructed frames for use as reference frames in inter-frame predictor 180. Memory 170 can store motion information of blocks in the current frame that derive (or encode) motion information and / or motion information of blocks in reconstructed frames. The stored motion information can be forwarded to inter-frame predictor 180 as motion information for spatially or temporally neighboring blocks. Memory 170 can store reconstructed samples of reconstructed blocks in the current frame and forward them to intra-frame predictor 185.
[0072] Figure 2 This is a schematic block diagram of a decoding device that applies the embodiments of this disclosure and decodes video / image signals.
[0073] Reference Figure 2 The decoding device 200 can be configured to include an entropy decoder 210, a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-frame predictor 260, and an intra-frame predictor 265. The inter-frame predictor 260 and the intra-frame predictor 265 can be collectively referred to as predictors. That is, predictors may include inter-frame predictor 180 and intra-frame predictor 185. The dequantizer 220 and the inverse transformer 230 can be collectively referred to as a residual processor. That is, the residual processor may include dequantizer 220 and inverse transformer 230. According to an embodiment, the entropy decoder 210, dequantizer 220, inverse transformer 230, adder 235, filter 240, inter-frame predictor 260, and intra-frame predictor 265 can be configured as a single hardware component (e.g., a decoder or a processor). Furthermore, the memory 170 may include a decoded picture buffer (DPB) and may be implemented using a digital storage medium.
[0074] When the input includes a bitstream containing video / image information, the decoding device 200 can, according to Figure 1 The process of processing video / image information in an encoding device reconstructs the image. For example, the decoding device 200 may use a processing unit applied in the encoding device to perform decoding. Therefore, for example, the processing unit used for decoding may be an encoding unit. Encoding units may be divided from encoding tree units or maximum encoding units according to a quadtree structure and / or a binary tree structure. Furthermore, the reconstructed image signal decoded and output by the decoding device 200 can be played back by a playback device.
[0075] Decoding device 200 can receive data in bitstream format. Figure 1 The received signal is output by the encoding device. The received signal can be decoded by the entropy decoder 210. For example, the entropy decoder 210 can deduce information (e.g., video / image information) for image reconstruction (or picture reconstruction) by parsing the bitstream. For example, the entropy decoder 210 can decode the information within the bitstream based on encoding methods such as exponential Golomb coding, CAVLC, or CABAC, and can output the values of the syntax elements for image reconstruction or quantized values of the transform coefficients with respect to the residuals. More specifically, in the CABAC entropy decoding method, bins corresponding to each syntax element can be received from the bitstream. A context model can be determined using information about the decoded target syntax elements and decoding information of neighboring and decoded target blocks, or information about symbols / bins decoded in previous steps. The probability of bin occurrence can be predicted based on the determined context model, and symbols corresponding to the values of each syntax element can be generated by performing arithmetic decoding on the bins. In this case, in the CABAC entropy decoding method, after determining the context model, the context model can be updated using information about the symbols / bins decoded for the next symbol / bin. Information about prediction from the information decoded in entropy decoder 2110 can be provided to the predictors (inter-frame predictor 260 and intra-frame predictor 265). Parameter information (i.e., quantized transform coefficients) related to the residual values from which entropy decoding has been performed in entropy decoder 210 can be input to dequantizer 220. Furthermore, information about filtering from the information decoded in entropy decoder 210 can be provided to filter 240. Additionally, a receiver (not shown) receiving the signal output from the encoding device can be further configured as an internal / external element of decoding device 200, or the receiver can be a component of entropy decoder 210.
[0076] Dequantizer 220 can dequantize the quantized transform coefficients and output the transform coefficients. Dequantizer 220 can rearrange the quantized transform coefficients in a two-dimensional block format. In this case, the rearrangement can be performed based on the coefficient scan order performed in the encoding device. Dequantizer 220 can use quantization parameters (e.g., quantization step size information) to perform dequantization on the quantized transform coefficients and obtain the transform coefficients.
[0077] The inverse converter 230 can output the residual signal (residual block or residual sample array) by applying an inverse transform to the transform coefficients.
[0078] The predictor can perform prediction on the current block and generate a prediction block that includes prediction samples of the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction to the current block based on the prediction information output by the entropy decoder 210, and can determine the detailed intra-frame / inter-frame prediction mode.
[0079] Intra-predictor 265 can refer to samples within the current frame to predict the current block. Depending on the prediction mode, the referenced samples can be located adjacent to the current block or spaced apart from it. In intra-prediction, the prediction mode can include multiple non-corner modes and multiple corner modes. Intra-predictor 265 can use the prediction modes applied to neighboring blocks to determine the prediction mode applied to the current block.
[0080] Inter-frame predictor 260 can deduce the predicted block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference frame. In this case, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted as a block, sub-block, or sample unit based on the correlation between motion information of neighboring blocks and the current block. Motion information may include motion vectors and reference frame indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction) information. In the case of inter-frame prediction, neighboring blocks may include spatially neighboring blocks within the current frame and temporally neighboring blocks within the reference frame. For example, inter-frame predictor 260 can configure a motion information candidate list based on neighboring blocks and can deduce the motion vector and / or reference frame index of the current block based on received candidate selection information. Inter-frame prediction can be performed based on various prediction modes. Information about the prediction may include information indicating the inter-frame prediction mode of the current block.
[0081] Adder 235 can generate a reconstruction signal (reconstructed frame, reconstruction block, or reconstruction sample array) by adding the obtained residual signal to the prediction signal (prediction block or prediction sample array) output by inter-frame predictor 260 or intra-frame predictor 265. If there is no residual for the processing target block, as in the case where skip mode has been applied, the prediction block can be used as a reconstruction block.
[0082] Adder 235 may be referred to as a reconstructor or reconstruction block generator. The generated reconstructed signal can be used for intra-frame prediction of the next processing target block in the current frame, and can also be used for inter-frame prediction of the next frame through filtering (as described later).
[0083] Filter 240 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 240 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image, and can send the modified reconstructed image to memory 250, more specifically, to the DPB of memory 250. For example, various filtering methods may include deblocking filtering, sample adaptive offset (SAO), adaptive loop filter (ALF), and bilateral filter.
[0084] The (modified) reconstructed frame stored in the DPB of memory 250 can be used as a reference frame in inter-frame predictor 260. Memory 250 can store motion information of blocks in the current frame that have had their motion information derived (or decoded) and / or motion information of blocks in reconstructed frames. The stored motion information can be forwarded to inter-frame predictor 260 as motion information for spatially or temporally neighboring blocks. Memory 170 can store reconstructed samples of reconstructed blocks in the current frame and forward them to intra-frame predictor 265.
[0085] In this disclosure, the embodiments described in the filter 160, inter-frame predictor 180 and intra-frame predictor 185 of the encoding device 100 can be applied to the filter 240, inter-frame predictor 260 and intra-frame predictor 265 of the decoding device 200 in the same or corresponding manner.
[0086] Block partition
[0087] The video / image coding method according to this disclosure can be implemented based on various detailed techniques, each of which is described below. It will be apparent to those skilled in the art that the techniques described herein can be associated with related processes in the video / image coding / decoding process described above and / or below (e.g., prediction, residual processing (inverse) transform, (de)quantization, etc., syntactic element encoding, filtering, partitioning / division).
[0088] The block partitioning process according to this disclosure can be performed in the image segmenter 110 of the aforementioned encoding device, and the partitioning-related information can be processed (encoded) in the entropy encoder 190 and forwarded to the decoding device in bitstream format. The entropy decoder 210 of the decoding device can obtain the block partitioning structure of the current frame based on the partitioning-related information obtained from the bitstream, and based on it, can perform a series of processes (e.g., prediction, residual processing, block reconstruction, loop filtering, etc.) for image decoding.
[0089] Divide the screen into CTUs
[0090] A frame can be segmented into a sequence of Code Tree Units (CTUs). A CTU may correspond to a Code Tree Block (CTB). Alternatively, a CTU may include a Code Tree Block for a luminance sample and two Code Tree Blocks for corresponding chrominance samples. In other words, for a frame containing three types of sample arrays, a CTU may include an N×N block of luminance samples and two corresponding chrominance samples.
[0091] The maximum support size of the CTU used for encoding and prediction can differ from the maximum support size of the CTU used for transformation. For example, the maximum support size of the luma block in the CTU can be 128×128.
[0092] CTU partitioning using a tree structure
[0093] The CTU can be partitioned into CUs based on a quadtree (QT) structure. The quadtree structure can be referred to as a quaternion structure. This is to reflect various local characteristics. Furthermore, in this disclosure, the CTU can be partitioned based on a multi-type tree structure partitioning, including binary trees (BT), ternary trees (TT), and quadtrees. Hereinafter, the QTBT structure can include quadtree and binary tree structures, and the QTBTTT can include partitioning structures based on binary and ternary trees. Alternatively, the QTBT structure can also include partitioning structures based on quadtrees, binary trees, and ternary trees. In the coding tree structure, the CU can have a square or rectangular shape. The CTU can first be partitioned into a quadtree structure, and then the leaf nodes of the quadtree structure can be further partitioned according to the multi-type tree structure.
[0094] Figure 3 This is a diagram illustrating an example of a multi-type tree structure to which the present disclosure can be applied.
[0095] In embodiments of this disclosure, the multi-type tree structure may include Figure 3 The four partitioning types are shown. These four partitioning types can include vertical binary partitioning (SPLIT_BT_VER), horizontal binary partitioning (SPLIT_BT_HOR), vertical ternary partitioning (SPLIT_TT_VER), and horizontal ternary partitioning (SPLIT_TT_HOR). Leaf nodes of the multi-type tree structure can be referred to as CUs. These CUs can be used in the prediction and transformation processes. In this disclosure, typically, CUs, PUs, and TUs can have the same block size. However, CUs and TUs can have different block sizes when the maximum supported transformation length is less than the width or height of the color component.
[0096] Figure 4 This is a diagram illustrating a signaling mechanism for partitioning information of a quadtree with nested multi-type tree structures, which is an embodiment to which this disclosure can be applied.
[0097] Here, the CTU can be considered the root of a quadtree and initially partitioned into a quadtree structure. Later, the leaf nodes of each quadtree can be further partitioned into multi-type tree structures. In the multi-type tree structure, a first flag (e.g., `mtt_split_cu_flag`) is signaled to indicate whether the corresponding node is further partitioned. If the corresponding node is further partitioned, a second flag (e.g., `mtt_split_cu_vertical_flag`) is signaled to indicate the partition direction. Later, a third flag (e.g., `mtt_split_cu_binary_flag`) is signaled to indicate whether the partition type is a binary or ternary partition. For example, based on `mtt_split_cu_vertical_flag` and `mtt_split_cu_binary_flag`, the multi-type tree partitioning mode (MttSplitMode) can be derived as shown in Table 1 below.
[0098] [Table 1]
[0099] MttSplitMode mtt_split_cu_vertical_flag mtt_split_cu_binary_flag SPLIT_TT_HOR 0 0 SPLIT_BT_HOR 0 1 SPLIT_TT_VER 1 0 SPLIT_BT_VER 1 1
[0100] Figure 5 This is a diagram illustrating a method for partitioning a CTU into multiple CUs based on a quadtree and nested multi-type tree structure, as an embodiment to which this disclosure can be applied.
[0101] Here, the bolded block edges represent quadtree partitions, and the remaining edges represent multi-type tree partitions. Quadtree partitions with nested multi-type trees provide a content-adaptive coding tree structure. A CU can correspond to a coding block (CB). Alternatively, a CU can include a coding block for luma samples and two coding blocks for corresponding chroma samples. The size of a CU can be the same as a CTU, or as small as 4×4 within a luma sample unit. For example, in the case of a 4:2:0 color format (or chroma format), the maximum chroma CB size can be 64×64, and the minimum chroma CB size can be 2×2.
[0102] In this disclosure, for example, the maximum supported luminance TB size can be 64×64, and the maximum supported chrominance TB size can be 32×32. If the width or height of a CB partitioned according to the tree structure is greater than the maximum transform width or height, the CB can be further partitioned until the TB size limits in the horizontal and vertical directions are automatically (or implicitly) met.
[0103] In addition, for quadtree coding schemes without nested multiple types, the following parameters can be defined or identified as SPS syntax elements.
[0104] -CTU size: Size of the root node of the quadtree
[0105] -MinQTSize: Minimum allowed size of quadtree leaf nodes
[0106] -MaxBtSize: Maximum allowed size of the root node of the binary tree
[0107] -MaxTtSize: Maximum allowed size of the root node of the ternary tree
[0108] -MaxMttDepth: The maximum allowed depth of partitioning from a quadtree leaf multitype tree.
[0109] -MinBtSize: Minimum allowed size of binary leaf nodes
[0110] -MinTtSize: Minimum allowed size of a ternary leaf node
[0111] As an example of a quadtree coding tree scheme with nested multi-type trees, the CTU size can be set to 128×128 luma samples and two corresponding chroma samples in a 64×64 block (in a 4:2:0 chroma sample). In this case, MinOTSize can be set to 16×16, MaxBtSize can be set to 128×128, MaxTtSize can be set to 64×64, MinBtSize and MinTtSize (for both width and height) can be set to 4×4, and MaxMttDepth can be set to 4. Quadtree partitioning can be applied to the CTU and generate quadtree leaf nodes. Quadtree leaf nodes can be called leaf QT nodes. Quadtree leaf nodes can have a size from 16×16 (i.e., MinOTSize) to 128×128 (i.e., CTU size). When the leaf QT node is 128×128, the leaf QT node may not be partitioned into a binary / ternary tree. This is because even when leaf QT nodes are partitioned, they exceed MaxBtsize and MaxTtsize (i.e., 64×64). In other cases, leaf QT nodes can be further partitioned into multi-type trees. Therefore, a leaf QT node can be the root node of a multi-type tree, and it can have a multi-type tree depth (mttDepth) of 0. When the multi-type tree depth reaches MaxMttdepth (e.g., 4), additional partitions are no longer considered. When the width of a multi-type tree node is equal to MinBtSize and less than or equal to 2xMinTtSize, additional horizontal partitions are no longer considered. When the height of a multi-type tree node is equal to MinBtSize and less than or equal to 2xMinTtSize, additional vertical partitions are no longer considered.
[0112] Figure 6 This is a diagram illustrating a method for restricting ternary tree partitioning as an embodiment to which this disclosure can be applied.
[0113] Reference Figure 6 To support 64×64 luma blocks and 32×32 chroma pipeline designs in the hardware decoder, the TT partitioning can be restricted in certain cases. For example, when the width or height of the luma encoding block is greater than a predetermined specific value (e.g., 32, 64), such as... Figure 6 As shown, the TT partition can be restricted.
[0114] In this disclosure, the coding tree scheme can support luma blocks and chroma blocks having separate block tree structures. For P-slices and B-slices, the luma and chroma CTBs within a single CTU can be restricted to having the same coding tree structure. However, for I-slices, luma blocks and chroma blocks can have correspondingly separate block tree structures. When applying a separate block tree mode, the luma CTB can be partitioned into CUs based on a specific coding tree structure, and the chroma CTB can be partitioned into chroma CUs based on different coding tree structures. This could mean that a CU in an I-slice could include a coding block of chroma components or a coding block of two chroma components, while a CU in a P-slice or B-slice could include a block of three color components.
[0115] In the above section on "Partitioning of CTUs Using Tree Structures," a quadtree coding tree scheme with nested multi-type trees was described; however, the partitioning structure of the CU is not limited to this. For example, BT and TT structures can be interpreted as concepts included in a multi-partition tree (MPT) structure, and can be interpreted as the CU being partitioned through QT and MPT structures. In the example of CU partitioning through QT and MPT structures, syntactic elements (e.g., MPT_split_type) including information about the number of blocks into which leaf nodes of the QT structure are partitioned, and syntactic elements (e.g., MPT_split_mode) including information about the direction in which leaf nodes of the QT structure are partitioned between the vertical and horizontal directions, can be used to determine the partitioning structure.
[0116] In another example, the CU can be partitioned in a manner different from the QT, BT, or TT structures. That is, unlike the QT structure which partitions a low-depth CU to 1 / 4 the size of a high-depth CU, the BT structure which partitions a low-depth CU to 1 / 2 the size of a high-depth CU, or the TT structure which partitions a low-depth CU to 1 / 4 or 1 / 2 the size of a high-depth CU, in some cases the low-depth CU can be partitioned to 1 / 5, 1 / 3, 3 / 8, 3 / 5, 2 / 3, or 5 / 8 the size of a high-depth CU, but the method of partitioning the CU is not limited to these.
[0117] If a portion of a tree node block extends beyond the bottom or right edge of the screen, the corresponding tree node block can be restricted so that all samples of all encoded CUs are within the screen boundaries. In this case, for example, the following partitioning rules can be applied.
[0118] -If part of a tree node block extends beyond both the bottom and right edge of the screen.
[0119] - If the block is a QT node and the block size is greater than the minimum QT size, then the block is forced to be partitioned in QT partitioning mode.
[0120] - Otherwise, force the blocks to be divided in the SPLIT_BT_HOR mode.
[0121] Otherwise, if part of the tree node block extends beyond the bottom frame boundary,
[0122] - If a block is a QT node, and the block size is greater than the minimum QT size and the block size is greater than the maximum BT size, then force the block to be partitioned in QT partitioning mode.
[0123] Otherwise, if the block is a QT node and the block size is greater than the minimum QT size and the block size is less than or equal to the maximum BT size, then force the block to be partitioned in either QT partitioning mode or SPLIT_BT_HOR mode.
[0124] - Otherwise (if the block is a BTT node or the block size is less than or equal to the minimum QT size), force the block to be divided in the SPLIT_BT_HOR mode.
[0125] - Otherwise, if part of the tree node block extends beyond the right screen boundary,
[0126] - If a block is a QT node, and the block size is greater than the minimum QT size and the block size is greater than the maximum BT size, then force the block to be partitioned in QT partitioning mode.
[0127] Otherwise, if the block is a QT node and the block size is greater than the minimum QT size and the block size is less than or equal to the maximum BT size, then force the block to be partitioned in QT partitioning mode or SPLIT_BT_VER mode.
[0128] - Otherwise (if the block is a BTT node or the block size is less than or equal to the minimum QT size), force the block to be divided in SPLIT_BT_VER mode.
[0129] Furthermore, the aforementioned quadtree coding tree scheme with nested multi-type trees provides a highly flexible partitioning structure. Due to the partitioning types supported in the multi-type trees, different partitioning patterns can result in the same coded block structure in some cases. By limiting the generation of these redundant partitioning patterns, the amount of partitioning information can be reduced. This is described with reference to the following figures.
[0130] Figure 7 This is a diagram illustrating redundant partitioning patterns that can be generated in binary tree partitioning and ternary tree partitioning as embodiments to which this disclosure can be applied.
[0131] like Figure 7 As shown, two consecutive binary partitions at two levels in one direction can have the same coded block structure as a binary partition of the central partition following a ternary partition. In this case, the binary partition of the central partition of the ternary partition can be restricted (in a given direction). This restriction can be applied to the CUs of all frames. When such a specific partition is restricted, the signaling of the corresponding syntax elements can be modified to reflect the restriction, and thereby the number of bits signaled for the partition can be reduced. For example, as... Figure 7 As illustrated in the example, when the binary tree partitioning of the central partition of the CU is restricted, the mtt_split_cu_binary_flag syntax element indicating whether the partition is a binary or ternary partition is not required to signal, and the decoder can infer that the value is 0.
[0132] As described above, CTU (or CTB (Code Tree Block), CU (C, CB)) can be partitioned based on multiple tree structures including binary trees (BT) and ternary trees (TT), as well as quadtrees. When partitioning blocks based on this partitioning structure and performing encoding based on the partitioned block units, embodiments of this disclosure propose a method for improving encoding rate.
[0133] The purpose of this disclosure is to provide a method for efficiently adjusting block partitioning when performing block partitioning using a QTBT structure.
[0134] In addition, the purpose of this disclosure is to propose a method for supporting QT partitioning after BT partitioning and TT partitioning.
[0135] In addition, the purpose of this disclosure is to propose a striped tree (ST) structure partitioning method for dividing non-square blocks into blocks of the same size.
[0136] Furthermore, the purpose of this disclosure is to propose a method for efficiently signaling QT, BT, and TT to divide structural information at the same syntactic level.
[0137] According to embodiments of this disclosure, the QTBT structure can be determined efficiently and relevant information can be communicated via signals, thereby significantly improving compression performance.
[0138] Implementation Method 1
[0139] In embodiments of this disclosure, the encoder / decoder can perform block partitioning using a BT (binary tree) or TT (ternary tree) structure (i.e., a multi-type tree structure). As an embodiment, a CTU or a single QT end block (i.e., a leaf node block of a QT structure) can be partitioned into two lower blocks using BT. Alternatively, a CTU or a single QT end block can be partitioned into three lower blocks using TT.
[0140] Figure 8 This is a diagram illustrating a method for partitioning blocks based on quadtrees and nested multi-type tree structures, as an embodiment of the present disclosure.
[0141] Reference Figure 8 This illustrates the partitioning methods using BT and TT. Figure 8 In this invention, it is assumed that square blocks are partitioned in the horizontal or vertical direction, but this disclosure is not limited to this, and non-square blocks can also be partitioned.
[0142] In the case of BT partitioning, a single block can be partitioned into two sub-blocks either horizontally or vertically, such as... Figure 8 (a) and Figure 8 As shown in (b), that is, the partitioned blocks can have two N×2N shapes or two 2N×N shapes.
[0143] In the case of TT partitioning, a single block can be divided into three sub-blocks either horizontally or vertically, such as... Figure 8 (c) and Figure 8 As shown in (d). When a block is divided into three blocks, one block is larger and two blocks are smaller, or the three blocks can be divided equally. As an example, in the case of horizontal partitioning, the height can be partitioned in a 1:2:1 ratio, and in the case of vertical partitioning, the width can be partitioned in a 1:2:1 ratio.
[0144] Implementation Method 2
[0145] Compression efficiency can be improved by increasing the degrees of freedom through block partitioning with various structures, but the increased complexity caused by the improved partitioning degrees of freedom can become a problem. Therefore, in the embodiments of this disclosure, when performing block partitioning based on QT structures, a method is proposed to send a syntax for adjusting the QT partitioning degree by considering the relationship between performance and complexity.
[0146] As an implementation, the encoder may send at least one syntax (or syntax element) of a maximum QT block size (MaxQTSize), a minimum QT block size (MinQTSize), and a maximum QT block partition depth (MaxQTDepth). Here, the maximum QT block size may represent the maximum block size that can be QT partitioned, and may be expressed in logarithmic scale form (e.g., log2). The minimum QT block size may represent the minimum block size that can be QT partitioned, and may be expressed in logarithmic scale form (e.g., log2). Additionally, the maximum QT block partition depth may represent the number of QT partitions from the maximum QT block size.
[0147] Depending on the type of slice (or tile group), tile, and image component, each syntactic information can be sent via the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Slice Header (SH) (or Tile Group Header (TGH)) or the header of a Network Abstraction Layer (NAL) unit.
[0148] Regarding a single image (or sequence, frame, tile, or group of tiles), the above syntax can be sent in a complementary manner. As an implementation, a single image can be sent in a combination of the following implementations.
[0149] (1) Maximum QT block size, minimum QT block size
[0150] (2) Maximum QT block size, maximum QT block partitioning depth
[0151] (3) Minimum QT block size, maximum QT block partitioning depth
[0152] (4) Maximum QT block size, minimum QT block size, maximum QT block partitioning depth
[0153] Referring to embodiment (1), the encoder can signal the maximum QT block size and minimum QT block size syntax (or syntax elements) to the decoder, and the decoder can use the maximum QT block size and minimum QT block size to derive the maximum QT block partition depth value. Alternatively, referring to embodiment (2), the encoder can signal the maximum QT block size and maximum QT block partition depth syntax to the decoder. The decoder can use the maximum QT block size and maximum QT block partition depth to derive the minimum QT block partition depth value. Alternatively, referring to embodiment (3), the encoder can signal the minimum QT block size and maximum QT block partition depth syntax to the decoder. The decoder can use the minimum QT block size and maximum QT block partition depth to derive the maximum QT block size value. Alternatively, referring to embodiment (4), the encoder can signal all three syntaxes. In this case, the value of the maximum QT block partition depth can be the same as the difference between the maximum QT block size and the minimum QT block size.
[0154] Implementation Method 3
[0155] In embodiments of this disclosure, when performing block partitioning based on the BT structure, syntax for adjusting the BT partitioning degree can be sent by considering the relationship between performance and complexity.
[0156] As an implementation, the encoder may send at least one syntax (or syntax element) of a maximum BT block size (MaxBTSize), a minimum BT block size (MinBTSize), and a maximum BT block partitioning depth (MaxBTDepth). Here, the maximum BT block size may represent the maximum block size that can be partitioned into BT blocks and may be expressed in logarithmic scale form (e.g., log2). The minimum BT block size may represent the minimum block size that can be partitioned into BT blocks and may be expressed in logarithmic scale form (e.g., log2). Additionally, the maximum BT block partitioning depth may represent the number of BT partitions from the maximum BT block size.
[0157] Depending on the type of slice (or tile group), tile, and image component, each syntactic information can be sent via the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Slice Header (SH) (or Tile Group Header (TGH)) or the header of a Network Abstraction Layer (NAL) unit.
[0158] Regarding a single image (or sequence, frame, tile, or group of tiles), the above syntax can be sent in a complementary manner. As an implementation, a single image can be sent in a combination of the following implementations.
[0159] (1) Maximum BT block size, minimum BT block size
[0160] (2) Maximum BT block size, maximum BT block partitioning depth
[0161] (3) Minimum BT block size, maximum BT block partitioning depth
[0162] (4) Maximum BT block size, minimum BT block size, maximum BT block partitioning depth
[0163] Referring to embodiment (1), the encoder can signal the maximum BT block size and minimum BT block size syntax (or syntax elements) to the decoder, and the decoder can use the maximum BT block size and minimum BT block size to derive the maximum BT block partitioning depth value. Alternatively, referring to embodiment (2), the encoder can signal the maximum BT block size and maximum BT block partitioning depth syntax to the decoder. The decoder can use the maximum BT block size and maximum BT block partitioning depth to derive the minimum BT block partitioning depth value. Alternatively, referring to embodiment (3), the encoder can signal the minimum BT block size and maximum BT block partitioning depth syntax to the decoder. The decoder can use the minimum BT block size and maximum BT block partitioning depth to derive the maximum BT block size value. Alternatively, referring to embodiment (4), the encoder can signal all three syntaxes. In this case, the value of the maximum BT block partitioning depth may differ from the difference between the maximum BT block size and the minimum BT block size. This is because the size of the block at the start of the BT partitioning depends on the QT partitioning.
[0164] In the implementation, the maximum BT block partitioning depth notified by the signal in (2) to (4) above can be used as the maximum depth of multi-type tree partitioning including BT structure and TT structure. In other words, the maximum BT block partitioning depth can be the same as the maximum MTT block partitioning depth.
[0165] The encoder / decoder can use the maximum BT block partitioning depth value to adjust (or control) performance and complexity. Additionally, if the initial BT partitioning block size is small, it reaches the minimum BT block size faster than the maximum BT partitioning depth. In this case, BT partitioning is no longer allowed for the corresponding block.
[0166] Implementation Method 4
[0167] In embodiments of this disclosure, when performing block partitioning based on the TT structure, the syntax for adjusting the TT partitioning degree can be sent by considering the relationship between performance and complexity.
[0168] As an implementation, the encoder may send at least one syntax (or syntax element) of a maximum TT block size (MaxTTSize), a minimum TT block size (MinTTSize), and a maximum TT block partitioning depth (MaxTTDepth). Here, the maximum TT block size may represent the maximum block size that can be partitioned into TT blocks and may be expressed in logarithmic scale form (e.g., log2). The minimum TT block size may represent the minimum block size that can be partitioned into TT blocks and may be expressed in logarithmic scale form (e.g., log2). Additionally, the maximum TT block partitioning depth may represent the number of TT partitions from the maximum TT block size.
[0169] Depending on the type of slice (or tile group), tile, and image component, each syntactic information can be sent via the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Slice Header (SH) (or Tile Group Header (TGH)) or the header of a Network Abstraction Layer (NAL) unit.
[0170] Regarding a single image (or sequence, frame, tile, or group of tiles), the above syntax can be sent in a complementary manner. As an implementation, a single image can be sent in a combination of the following implementations.
[0171] (1) Maximum TT block size, minimum TT block size
[0172] (2) Maximum TT block size, maximum TT block partitioning depth
[0173] (3) Minimum TT block size, maximum TT block partitioning depth
[0174] (4) Maximum TT block size, minimum TT block size, maximum TT block partitioning depth
[0175] Referring to implementation method (1), the encoder can signal the maximum TT block size and minimum TT block size syntax (or syntax element) to the decoder, and the decoder can use the maximum TT block size and minimum TT block size to derive the maximum TT block partitioning depth value.
[0176] Additionally, referring to embodiment (2), the encoder can signal the maximum TT block size and the maximum TT block partitioning depth to the decoder. The decoder can use the maximum TT block size and the maximum TT block partitioning depth to derive the minimum TT block partitioning depth value.
[0177] Additionally, referring to embodiment (3), the encoder can signal the minimum TT block size and the maximum TT block partitioning depth to the decoder. The decoder can use the minimum TT block size and the maximum TT block partitioning depth to derive the maximum TT block size value.
[0178] Additionally, referring to implementation (4), the encoder can signal all three syntaxes. In this case, the value of the maximum TT block partitioning depth can differ from the difference between the maximum TT block size and the minimum TT block size. This is because the block size at the start of the TT partitioning depends on the QT partitioning.
[0179] The encoder / decoder can use the maximum TT block partitioning depth value to control (or adjust) performance and complexity. Additionally, if the initial TT partitioning block size is small, it will reach the minimum TT block size earlier than the maximum TT partitioning depth. In this case, the corresponding block will no longer be allowed to undergo TT partitioning.
[0180] In the implementation, the maximum TT block partitioning depth, which is signaled in (2) to (4) above, can be used as the maximum depth of the multi-type tree partitioning, including BT and TT structures. In other words, the maximum BT block partitioning depth can be the same as the maximum MTT block partitioning depth.
[0181] In this implementation, the maximum TT block size can be defined as 16×16. In this case, the length of one side of the TT block can be defined as 4, 8, or 4. Additionally, the minimum TT block size can be represented as log2_min_luma_TT_block_size_minus4, and the difference between the minimum and maximum TT block sizes can be represented as log2_diff_max_min_luma_TT_block_size.
[0182] The decoder can use the minimum TT block size to derive the TT block size variable (TTbLog2SizeY). For example, equations 1 and 2 below can be used to derive the TT block size variable (TTbLog2SizeY).
[0183] [Formula 1]
[0184] MinTTbLog2SizeY=log2_min_luma_TT_block_size_minus4+4
[0185] [Equation 2]
[0186] TTbLog2SizeY=MinTTbLog2SizeY+log2_diff_max_min_luma_TT_block_size
[0187] The TT block size variable (TTbLog2SizeY) derived from this formula can be used in subsequent decoding processes. For example, the TT block size variable (TTbLog2SizeY) can be used in intra-frame prediction, inter-frame prediction, filtering, transform, quantization, and entropy decoding processes.
[0188] Implementation Method 5
[0189] In embodiments of this disclosure, when performing block partitioning based on BT and TT structures (which may be referred to as MTT or BTT as multi-type tree structures), the encoder / decoder can send syntax for adjusting the MTT partitioning degree by taking into account the relationship between performance and complexity.
[0190] As an implementation, the encoder may send at least one syntax (or syntax element) of a maximum MTT block size (MaxMTTSize), a minimum MTT block size (MinMTTSize), and a maximum MTT block partitioning depth (MaxBTTDepth). Here, the maximum MTT block size may represent the maximum block size that can be partitioned into MTTs and may be expressed in logarithmic scale form (e.g., log2). The minimum MTT block size may represent the minimum block size that can be partitioned into MTTs and may be expressed in logarithmic scale form (e.g., log2). Additionally, the maximum MTT block partitioning depth may represent the number of MTT partitions from the maximum MTT block size.
[0191] Depending on the type of slice (or tile group), tile, and image component, each syntactic information can be sent via the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Slice Header (SH) (or Tile Group Header (TGH)) or the header of a Network Abstraction Layer (NAL) unit.
[0192] Regarding a single image (or sequence, frame, tile, or group of tiles), the above syntax can be sent in a complementary manner. As an implementation, a single image can be sent in a combination of the following implementations.
[0193] (1) Maximum MTT block size, minimum MTT block size
[0194] (2) Maximum MTT block size, maximum MTT block partitioning depth
[0195] (3) Minimum MTT block size, maximum MTT block partitioning depth
[0196] (4) Maximum MTT block size, minimum MTT block size, maximum MTT block partitioning depth
[0197] Referring to implementation (1), the encoder can signal the maximum MTT block size and minimum MTT block size syntax (or syntax element) to the decoder, and the decoder can use the maximum MTT block size and minimum MTT block size to derive the maximum MTT block partitioning depth value.
[0198] Additionally, referring to embodiment (2), the encoder can signal the maximum MTT block size and the maximum MTT block partitioning depth to the decoder. The decoder can use the maximum MTT block size and the maximum MTT block partitioning depth to derive the minimum MTT block partitioning depth value.
[0199] Additionally, referring to embodiment (3), the encoder can signal the minimum MTT block size and the maximum MTT block partitioning depth to the decoder. The decoder can use the minimum MTT block size and the maximum MTT block partitioning depth to derive the maximum MTT block size value.
[0200] Additionally, referring to implementation (4), the encoder can signal all three syntaxes. In this case, the value of the maximum MTT block partitioning depth can differ from the difference between the maximum MTT block size and the minimum MTT block size. This is because the block size at the start of the MTT partitioning depends on the QT partitioning.
[0201] The encoder / decoder can use the maximum MTT block partitioning depth value to control (or adjust) performance and complexity. Additionally, if the initial MTT partitioning block size is small, it reaches the minimum MTT block size faster than the maximum MTT partitioning depth. In this case, the corresponding block is no longer allowed to undergo MTT partitioning.
[0202] Implementation Method 6
[0203] Figure 9 This is a diagram illustrating a quadtree-based block partitioning structure as an embodiment to which the present disclosure can be applied.
[0204] Reference Figure 9 In embodiments of this disclosure, a single block can be partitioned based on a quadtree (QT). Furthermore, a single sub-block partitioned using a QT structure can be further recursively partitioned using a QT structure. End blocks that are no longer partitioned using a QT structure (which may be referred to as leaf blocks or leaf node blocks) can be partitioned using at least one of a binary tree (BT), a ternary tree (TT), or an asymmetric tree (AT).
[0205] As an example, block A can be divided into four sub-blocks A0, A1, A2, and A3 using a QT structure. Furthermore, sub-block A1 can be further divided into four sub-blocks B0, B1, B2, and B3 using a QT structure.
[0206] Figure 10 This is a diagram illustrating a binary tree-based block partitioning structure as an embodiment to which the present disclosure can be applied.
[0207] Reference Figure 10 BT can have two partitioned shapes, including a horizontal BT (e.g., 2N×N, 2N×N) and a vertical BT (e.g., N×2N, N×2N).
[0208] As an example, a block (i.e., a leaf node block) B3 that is no longer divided using a QT structure can be divided into vertical BT C0 and C1 or horizontal BT D0 and D1. In the case of vertical division, similar to block C0, the individual sub-blocks can be recursively divided into horizontal BT E0 and E1 or vertical BT F0 and F1.
[0209] Figure 11 This is a diagram illustrating a ternary tree-based block partitioning structure as an embodiment to which the present disclosure can be applied.
[0210] Reference Figure 11 TT can have two partition shapes, including horizontal TT (e.g., 2N×1 / 2N, 2N×N, 2N×1 / 2N) and vertical TT (e.g., 1 / 2N×2N, N×2N, 1 / 2N×2N).
[0211] As an example, block B3, which is no longer divided using the QT structure, can be divided into vertical directions TT C0, C1, and C2 or horizontal directions TT D0, D1, and D2. Figure 11 As shown, block C1 can be recursively divided in the horizontal directions TT E0, E1 and E2 or the vertical directions TT F0, F1 and F2.
[0212] As another example, AT can have four partitioning structure shapes, including horizontally upward AT (2N×1 / 2N, 2N×3 / 2N), horizontally downward AT (2N×3 / 2N, 2N×1 / 2N), vertically to the left AT (1 / 2N×2N, 3 / 2N×2N), and vertically to the right AT (3 / 2N×2N, 1 / 2N×2N). Each of BT, TT, and AT can be further recursively partitioned using BT, TT, and AT.
[0213] Figure 12 This is a diagram illustrating a block partitioning structure based on binary and ternary trees as an embodiment to which this disclosure can be applied.
[0214] Reference Figure 12 Processing blocks can be divided using both BT and TT partitioning structures. For example, sub-blocks partitioned using a BT structure can be partitioned using a TT structure. Alternatively, sub-blocks partitioned using a TT structure can be partitioned using a BT structure.
[0215] exist Figure 12 In the diagram, the division represented by the thin solid line represents the first division, and the division represented by the thick dashed line represents the second division performed in the sub-blocks generated in the first division.
[0216] Table 2 shows the syntax structure of the encoding quadtree, and Table 3 shows the syntax of the encoding tree.
[0217] [Table 2]
[0218]
[0219] Referring to Table 2, the decoding process used to determine the quadtree partition structure is described. The encoding quadtree syntax (or function) can be invoked with the top-left coordinates (x0, y0) of the current block, the size of the current block (log2CbSize), and the depth of the current quadtree partition as input.
[0220] The decoder parses the `split_qt_flag` syntax element if the current block does not exceed the width or height of the current frame, and the current block is larger than the smallest coded block. The `split_qt_flag` syntax element indicates whether the current block is divided into four blocks using a quadtree structure. For example, a `split_qt_flag` value of 0 indicates that the current block is not divided into four blocks using a quadtree structure, and a `split_qt_flag` value of 1 indicates that the current block is divided into four blocks, each half the width and half the height. `x0` and `y0` represent the top-left position of the luminance image.
[0221] As a result of the parsing, if it is determined that the current block is divided into a quadtree structure, the decoder calls the encoding quadtree syntax (or function) again for the four sub-blocks divided from the current block.
[0222] As a result of the parsing, if it is determined that the current block is not partitioned in a quadtree structure, the decoder calls the encoded quadtree syntax (or function) for the current block to determine the subsequent partitioning structure (i.e., a multi-type tree structure).
[0223] [Table 3]
[0224]
[0225] Referring to Table 3, the decoding process used to determine the additional partitioning structure in a quadtree leaf node block is described. The encoding quadtree syntax (or function) can be invoked with the top-left coordinate (x0, y0), the width (log2CbSize), and the width (log2CbSizeH) of the current block as input.
[0226] If the current block does not exceed the width or height of the current frame and is larger than the minimum size MTT block, the decoder parses the `split_fur_flag` syntax element. The `split_fur_flag` syntax element indicates whether the current block is to be further divided. For example, a `split_fur_flag` value of 0 indicates that the current block will not be divided, while a `split_fur_flag` value of 1 indicates that the current block is to be divided.
[0227] As a result of the parsing, if it is determined that the current block needs to be further divided, the decoder calls the `split_bt_flag` and `split_dir` syntax elements (or functions). The `split_bt_flag` syntax indicates whether the current block is divided using a BT or TT structure. For example, a `split_bt_flag` value of 1 indicates that the block is divided using BT, and a `split_bt_flag` value of 0 indicates that the block is divided using TT.
[0228] The split type (SplitType) of the current block can be determined based on the split_fur_flag and split_bt_flag syntaxes as shown in Table 4 below.
[0229] [Table 4]
[0230] split_fur_flag split_bt_flag SplitType 0 NO_SPLIT 1 1 BT 1 0 TT
[0231] Additionally, in Table 3, the `split_dir` syntax element indicates the splitting direction. For example, a `split_dir` value of 0 indicates that the block is split horizontally, while a `split_dir` value of 1 indicates that the block is split vertically.
[0232] The final block partitioning mode (SplitMode) from the current block MTT can be obtained as shown in Table 5 below.
[0233] [Table 5]
[0234]
[0235] The decoder then calls the encoding tree syntax (or function) again for the sub-blocks divided according to the final determined partitioning structure.
[0236] Implementation Method 7
[0237] In embodiments of this disclosure, a method for supporting QT partitioning during partitioning is proposed after partitioning into multi-type tree (i.e., BT, TT) structures. As an implementation, as shown in Table 6 below, QT partitioning can be selectively supported after performing BT and TT partitioning via flag signaling in the Sequence Parameter Set (SPS).
[0238] [Table 6]
[0239]
[0240] Referring to Table 6, the `nesting_split_qt_enable_flag` syntax element indicates whether the `split_qt_flag` used for QT partitioning is sent in the lower-level syntax (e.g., the code tree) after QT partitioning. For example, when `nesting_split_qt_enable_flag` is 0, `split_qt_flag` is not sent in the code tree unit. When `nesting_split_qt_enable_flag` is 1, `split_qt_flag` can be sent in the code tree unit.
[0241] Table 6 above is merely an example, but this disclosure is not limited thereto. Picture Parameter Set (PPS), slice headers (or tile group headers), or other network abstraction layers (NALs) and SPS can be used to signal the nesting_split_qt_enable_flag syntax.
[0242] Table 7 below shows the syntactic structure of the coding tree according to an embodiment of the present disclosure.
[0243] [Table 7]
[0244]
[0245]
[0246] Referring to Table 7, the top-left coordinate (x0, y0), width (log2CbSizeW), and width (log2CbSizeH) of the current block can be used as input to call the encoding tree syntax (or function).
[0247] The decoder checks whether the current block does not exceed the width or height of the current frame and whether the current block is larger than the minimum size BTT block (or MTT block).
[0248] If the current block does not exceed the width or height of the current frame and the current block is larger than the smallest BTT block, the decoder can parse nesting_split_qt_flag when nesting_split_qt_enable_flag is 1 from the higher-level syntax, bttDepth is greater than 0, and the width and height of the blocks are equal.
[0249] Here, `bttDepth` can refer to the depth of the block divided by BT or TT, and can be calculated as the ratio of the current block size to the width and height of the initial BT or TT block. The `nesting_split_qt_flag` syntax indicates whether the current encoded block is divided into four blocks in a QT shape. For example, a `nesting_split_qt_flag` value of 0 indicates that the current block has not been divided, while a `nesting_split_qt_flag` value of 1 indicates that the current block has been divided into four blocks with half the width and half the height.
[0250] If nesting_split_qt_flag is not shown in the bitstream, it is assigned a value of 0. Later, if nesting_split_qt_flag is 0, the decoder parses split_fur_flag. Additionally, if the current block is identified as being further divided based on the split_fur_flag value, the decoder parses the split_bt_flag and split_dir syntax elements.
[0251] The `split_fur_flag` syntax indicates whether the current encoded block is to be further divided. For example, a `split_fur_flag` value of 0 indicates that the block will not be divided, while a `split_fur_flag` value of 1 indicates that the block has been divided.
[0252] The `split_bt_flag` syntax indicates whether the block is split using a BT or TT structure. For example, a `split_bt_flag` value of 1 indicates that the block is split using BT, while a `split_bt_flag` value of 0 indicates that the block is split using TT. The split type (SplitType) of the current block can be determined based on the `split_fur_flag` and `split_bt_flag` syntaxes, as shown in Table 8 below.
[0253] [Table 8]
[0254] nesting_split_qt_flag split_fur_flag split_bt_flag SplitType 0 0 NO_SPLIT 0 1 1 BT 0 1 0 TT 1 QT
[0255] Additionally, in Table 7, the `split_dir` syntax element indicates the splitting direction. For example, a `split_dir` value of 0 indicates that the block is split horizontally, while a `split_dir` value of 1 indicates that the block is split vertically.
[0256] The final block partitioning mode (SplitMode) from the current block MTT can be obtained as shown in Table 9 below.
[0257] [Table 9]
[0258]
[0259] The decoder then calls the encoding tree syntax (or function) again for the sub-blocks divided according to the final determined partitioning structure.
[0260] Implementation Method 8
[0261] Figure 13 This is a diagram illustrating a method for dividing blocks based on a striped tree structure, which is an embodiment of the present disclosure.
[0262] In terms of encoder and decoder hardware, square-shaped partitioning is more efficient than non-square partitioning. In the case of non-square blocks where the width and height differ significantly, re-dividing the non-square blocks into square blocks may be more efficient. Therefore, in embodiments of this disclosure, a striped tree (ST) block partitioning structure is proposed for dividing non-square blocks into four blocks of equal size.
[0263] Specifically, such as Figure 13 As shown, the encoder / decoder can be in the vertical direction ( Figure 13 (a) or vertical direction ( Figure 13 (b) uses an ST structure to divide the non-square block into four blocks of the same size.
[0264] As an implementation method, as shown in Table 10 below, support for ST partitioning can be selectively used via flag signaling in the Sequence Parameter Set (SPS).
[0265] [Table 10]
[0266]
[0267] Referring to Table 10, the `split_st_enable_flag` syntax element indicates whether the `split_st_flag` used for ST partitioning is sent at a lower-level syntax (e.g., the coding tree). For example, when `split_st_enable_flag` is 0, `split_st_flag` is not sent in the coding tree unit. When `split_st_enable_flag` is 1, `split_st_flag` can be sent in the coding tree unit.
[0268] Table 10 above is merely an example, but this disclosure is not limited thereto. The split_st_enable_flag syntax can be signaled using a Picture Parameter Set (PPS), a slice header (or tile group header), or other Network Abstraction Layers (NALs) and SPS.
[0269] Table 11 below shows the syntactic structure of the coding tree according to an embodiment of the present disclosure.
[0270] [Table 11]
[0271]
[0272]
[0273] Referring to Table 11, the top-left coordinates (x0, y0), width (log2CbSizeW), and width (log2CbSizeH) of the current block can be used as input to invoke the encoding tree syntax (or function). Descriptions that are repeated in Table 7 above are omitted.
[0274] The decoder parses `split_st_enable_flag` if the value of `split_st_enable_flag` is 1 in the higher-level syntax, and if the width of the current block is four times greater than its height, or vice versa. Here, the `split_st_flag` syntactic element indicates whether the current encoded block is divided into four blocks in an ST shape.
[0275] For example, a `split_st_flag` value of 0 indicates that the block is not divided, while a `split_st_flag` value of 1 indicates that the block is divided into four equal-sized blocks horizontally or vertically. If `split_st_flag` is not shown in the bitstream, it can be assumed to be 0. Later, when `split_st_flag` is 0, the `split_fur_flag`, `split_bt_flag`, and `split_dir` syntax can be sent.
[0276] The `split_fur_flag` syntax indicates that the current coding block is being further divided. For example, a `split_fur_flag` value of 0 means the block will not be divided, while a `split_fur_flag` value of 1 means the block will be divided.
[0277] The split type (SplitType) of the current block can be determined based on the split_fur_flag and split_bt_flag syntaxes as shown in Table 12 below.
[0278] [Table 12]
[0279] split_st_flag split_fur_flag split_bt_flag SplitType 0 0 NO_SPLIT 0 1 1 BT 0 1 0 TT 1 ST
[0280] Additionally, in Table 11, the `split_dir` syntax element indicates the splitting direction. For example, a `split_dir` value of 0 indicates that the block is split horizontally, while a `split_dir` value of 1 indicates that the block is split vertically.
[0281] The final block partitioning mode (SplitMode) from the current block MTT can be obtained as shown in Table 13 below.
[0282] [Table 13]
[0283]
[0284] Additionally, in the implementation, as shown in Table 14 below, the above-mentioned split_st_flag syntax element can be located after the split_fur_flag syntax.
[0285] [Table 14]
[0286]
[0287] Referring to Table 14, when the current block is further divided according to split_fur_flag, the decoder can parse split_st_enable_flag if the value of split_st_enable_flag is 1 in the higher-level syntax and the width of the current block is four times greater than its height or the height of the current block is four times greater than its width.
[0288] Additionally, in Tables 11 and 14 above, the split_st_flag is sent (or parsed) by assuming a difference of 4 times between the width and height in ST. However, this disclosure is not limited to this; the split_st_flag can be sent (or parsed) when the difference between the width and height is one (for the same size), two, eight, or sixteen times. Alternatively, the split_st_flag can be sent by configuring a combination of two or three ratios. Alternatively, the split_st_flag can be sent with a predetermined block ratio or higher.
[0289] Additionally, `split_st_flag` can be used in conjunction with `nesting_split_qt_flag` as described in Implementation 7. The syntactic structure when both syntactic elements are used together can be represented in Table 15 below.
[0290] [Table 15]
[0291]
[0292] Referring to Table 15, whether Qt partitioning has been performed can be identified by parsing `nesting_split_qt_flag`, and whether the current block has been partitioned can be identified by parsing `split_fur_flag`. If the current block is further partitioned according to `split_fur_flag`, the decoder can parse `split_st_enable_flag` if the value of `split_st_enable_flag` is 1 in the higher-level syntax and the width of the current block is four times greater than its height, or vice versa.
[0293] Implementation Method 9
[0294] In embodiments of this disclosure, a method is proposed for transmitting QT, BT, and TT partitioning structures with the same syntactic structure (i.e., the same level of syntax). In this embodiment, regardless of whether the block is a square block or not, the encoder / decoder can perform QT structure partitioning from the highest node (e.g., CTU). Furthermore, partitioning can also be performed from the highest node regardless of whether the BT and TT structures are leaf nodes of the QT partition. Table 16 shows the syntax of the encoded quadtree according to embodiments of this disclosure.
[0295] [Table 16]
[0296]
[0297] Referring to Table 16, the top-left coordinates (x0, y0) of the current block, the size of the current block (log2CbSize), and the depth of the current quadtree partition can be used as input to call the encoding quadtree syntax (or function).
[0298] If the current block does not exceed the width or height of the current frame and is larger than the minimum size of the encoded block, the decoder parses the `split_fur_flag` syntax element. Here, the `split_fur_flag` syntax element indicates whether the current block is divided into four blocks using a quadtree structure.
[0299] For example, a value of 0 for `split_fur_flag` indicates that the current block is not divided into four blocks using a quadtree structure, while a value of 1 indicates that the current block is divided into blocks of lower depth. That is, in this implementation, and when `split_fur_flag` is 1, the `split_qt_flag` syntax element can be sent.
[0300] That is, the decoder recognizes the `split_fur_flag` value, and when the value is 1, the decoder parses the `split_qt_flag` syntax element. Here, the `split_st_flag` syntax element indicates whether the current block is divided into four blocks using a QT structure. If the current block is further divided, the decoder can first check whether the current block is divided using a QT structure. For example, a `split_qt_flag` value of 0 indicates that the block is not divided using a QT structure, and a `split_qt_flag` value of 1 indicates that the block is divided into four blocks, each with half the width and half the height. In this implementation, when the `split_qt_flag` value is 0, `split_bt_flag` can be sent. In other words, if the current block is not divided using a QT structure, the decoder can determine whether the current block is divided using an MTT structure.
[0301] That is, when `split_qt_flag` is 0, the decoder can parse the `split_bt_flag` and / or `split_dir` syntax elements. Here, the `split_bt_flag` syntax element indicates whether the block is split using a multi-type tree structure (i.e., BT or TT). For example, a `split_bt_flag` value of 1 indicates that the block is split using BT, and a `split_bt_flag` value of 0 indicates that the block is split using TT. Alternatively, a `split_bt_flag` value of 0 indicates that the block is split using BT, and a `split_bt_flag` value of 1 indicates that the block is split using TT.
[0302] The split type (SplitType) of the current block can be determined based on the split_fur_flag, split_qt_flag, and split_bt_flag syntax elements as shown in Table 17 below.
[0303] [Table 17]
[0304] split_fur_flag split_qt_flag split_bt_flag SplitType 0 NO_SPLIT 1 1 QT 1 0 1 BT 1 0 0 TT
[0305] Hereinafter, an implementation method associated with the description in Implementation Method 4 above will be described.
[0306] In this implementation, the TT block size parameter (TTbLog2SizeY) described in Implementation 4 above can be used in the process of deriving the intra-prediction mode during intra-frame prediction. Specifically, the TT block size variable (TTbLog2SizeY) can be used in the process of deriving the candidate intra-prediction mode.
[0307] For example, if the prediction block exists at the top of the TT block, the candidate intra-prediction mode of the upper neighbor block can be set to INTRA_DC. More specific implementations may follow, but this disclosure is not limited thereto.
[0308] Figure 14 This is a diagram showing the reference positions of the current block and neighboring blocks as embodiments to which this disclosure can be applied.
[0309] Reference Figure 14 (xPb, yPb) indicates the position of the predicted block, (xNbB, yNbB) indicates the position of the sample adjacent to the top, and (xNbA, yNbA) indicates the position of the sample adjacent to the left. (xNbA, yNbA) can be set to (xPb-1, yPb), and (xNbB, yNbB) can be set to (xPb, yPb-1).
[0310] To derive a candidate intra-prediction mode, the encoder / decoder can check whether the left and top neighbor blocks are available (referred to as the first condition). If the left and top neighbor blocks are unavailable, the candidate intra-prediction mode (candIntraPredModeX) (X = A, B) can be set to INTRA_DC.
[0311] If not all left and top neighbor blocks are unavailable, the encoder / decoder can check whether the left or top neighbor block is in inter-frame mode or PCM mode (referred to as the second condition). If the left or top neighbor block is not in intra-frame prediction mode or the PCM flag is 1 (i.e., in the case of PCM encoding), the encoder / decoder can set the candidate intra-frame prediction mode (candIntraPredModeX) (X = A, B) to INTRA_DC.
[0312] When the first and second conditions are not met, if the prediction block exists at the top of the TT block, the candidate intra-prediction mode of the upper neighbor block can be set to INTRA_DC.
[0313] As described above, candidate intra-prediction modes can be derived based on the TT block size variable (TTbLog2SizeY), and these candidate intra-prediction modes can be used to derive the intra-prediction mode for the current block. Prediction blocks are generated based on the derived intra-prediction modes, and the decoder can reconstruct the video signal by adding the residual block and the prediction block.
[0314] As another implementation, the TT block size variable (TTbLog2SizeY) can be used in the processing of the motion vector at the prediction time during inter-frame prediction. For example, the TT block size variable (TTbLog2SizeY) can be used in the processing to derive the juxtaposed motion vector of the lower right block.
[0315] Specifically, based on the TT block size variable (TTbLog2SizeY), the juxtaposed motion vector of the lower right block can be obtained, and the juxtaposed motion vector of the lower right block can be used to generate the motion vector prediction value of the current prediction block.
[0316] The encoder / decoder generates motion vectors by adding the predicted motion vector values and the difference between the motion vectors, and uses these motion vectors to generate inter-frame prediction blocks. The decoder reconstructs the video signal by adding the residual blocks and the predicted blocks.
[0317] As another implementation, the TT block size variable (TTbLog2SizeY) can be used in the process of deriving quantization parameters. For example, the TT block size variable (TTbLog2SizeY) can be used in the process of deriving the quantization parameters of the left neighbor block or the top neighbor block.
[0318] As another implementation, the TT block size variable (TTbLog2SizeY) can be used in the process of performing SAO (Sample Adaptive Offset).
[0319] Additionally, in Table 16, the `split_dir` syntax element indicates the splitting direction. For example, a `split_dir` value of 0 indicates that the block is split horizontally, while a `split_dir` value of 1 indicates that the block is split vertically.
[0320] The final block partitioning mode (SplitMode) from the current block MTT can be obtained as shown in Table 18 below.
[0321] [Table 18]
[0322]
[0323] The decoder then calls the encoding quadtree syntax (or function) again for the sub-blocks divided according to the final determined partitioning structure.
[0324] In embodiments of this disclosure, when describing a method for sending partitioned structures of QT, BT, and TT with the same syntactic structure (i.e., the same level of syntax), for ease of description, the case of sending partitioned structures of QT, BT, and TT with encoded quadtree syntax is mainly described, but this disclosure is not limited thereto. For example, the partitioned structures of QT, BT, and TT can be sent with encoded tree syntax.
[0325] Up to this point, for ease of description, embodiments of the present disclosure have been described individually for each implementation, but the present disclosure is not limited thereto. In other words, embodiments 1 to 9 described above can be performed individually, or one or more embodiments can be performed in combination.
[0326] Figure 15 This is a flowchart illustrating a method for decoding a video signal according to an embodiment of the present disclosure.
[0327] Reference Figure 15 For ease of description, the decoder is described primarily, but this disclosure is not limited thereto. The video signal decoding method according to embodiments of this disclosure can be performed in the same manner in both the encoder and the decoder.
[0328] If the current block meets the pre-configured conditions, the decoder parses the first syntactic element indicating whether the current block is divided into multiple sub-blocks (step S1501).
[0329] As described in embodiments 6 to 9 above, when the value of the first syntactic element is 0, the current block may not be divided. When the value of the first syntactic element is 1, the current block may be divided into four sub-blocks using a quadtree structure, into two sub-blocks using a binary tree structure, or into three sub-blocks using a ternary tree structure.
[0330] Additionally, as described in the examples in Tables 2, 3, 7, 11, 14, 15, and 16 above, the pre-configured conditions can be met when the value of the horizontal coordinate of the top-left sample of the current block plus the width of the current block is equal to or less than the width of the current screen, and when the value of the vertical coordinate of the top-left sample of the current block plus the height of the current block is equal to or less than the height of the current screen.
[0331] Additionally, as described above, the decoder can invoke the encoding tree unit syntax used to determine the partitioning structure of the current block. In this case, the step of executing the encoding tree unit syntax for the current block can be performed before step S1501.
[0332] If the first syntactic element indicates that the current block is partitioned, the decoder parses the second syntactic element indicating whether the current block is partitioned using a quadtree structure (step S1502).
[0333] As described in embodiments 1 to 9 above, when the first syntax element indicates that the current block is not divided, the decoder may invoke the encoding unit syntax for decoding the current block.
[0334] If the second syntactic element indicates that the current block is not partitioned using a quadtree structure, the decoder parses at least one of the third syntactic element indicating whether the current block is partitioned using a binary tree structure or a ternary tree structure and the fourth syntactic element indicating the partitioning direction of the current block (step S1503).
[0335] The decoder determines the partitioning pattern (or partitioning type) of the current block based on at least one of the first, second, third, and fourth syntactic elements (step S1504). As an example, the partitioning type can be determined based on the first, second, third, and fourth syntactic elements as shown in Tables 4, 5, 8, 9, 12, 13, and 17 above.
[0336] Additionally, as described in embodiments 6 to 9 above, the decoder may invoke the coding tree unit syntax to determine the partitioning structure of sub-blocks partitioned from the current block based on the partitioning pattern.
[0337] Furthermore, as described in Embodiment 9 above, the first syntactic element, the second syntactic element, the third syntactic element, and the fourth syntactic element can be parsed from the same level of syntax.
[0338] Additionally, as described in Embodiment 7 above, the decoder can parse a fifth syntactic element indicating whether the current block, which is partitioned from the higher node using a binary tree structure or a ternary tree structure, is partitioned using a quadtree structure.
[0339] Figure 16 This is a flowchart illustrating a video signal decoding device according to an embodiment of the present disclosure.
[0340] exist Figure 16 In this text, for ease of description, the decoding device is depicted in a single block, but... Figure 16 The decoding device shown can be implemented as a component included in the encoder and / or decoder.
[0341] Reference Figure 16 Decoding devices achieve the above Figures 1 to 15 The functions, processing, and / or methods proposed in the document. Specifically, the decoding device may include a syntactic element parsing unit 1601 and a partitioning pattern determination unit 1602. Additionally, in some embodiments, the syntactic element parsing unit 1601 may be a component included in the partitioning pattern determination unit 1602.
[0342] If the current block meets the pre-configured conditions, the syntax element parsing unit 1601 parses the first syntax element indicating whether the current block is divided into multiple sub-blocks.
[0343] As described in embodiments 6 to 9 above, when the value of the first syntactic element is 0, the current block may not be divided. When the value of the first syntactic element is 1, the current block may be divided into four sub-blocks using a quadtree structure, into two sub-blocks using a binary tree structure, or into three sub-blocks using a ternary tree structure.
[0344] Additionally, as described in the examples in Tables 2, 3, 7, 11, 14, 15, and 16 above, the pre-configured conditions can be met when the value of the horizontal coordinate of the top-left sample of the current block plus the width of the current block is equal to or less than the width of the current screen, and when the value of the vertical coordinate of the top-left sample of the current block plus the height of the current block is equal to or less than the height of the current screen.
[0345] In addition, as mentioned above, the syntax element parsing unit 1601 can call the encoding tree unit syntax to determine the partitioning structure of the current block.
[0346] If the first syntactic element indicates that the current block is partitioned, the syntactic element parsing unit 1601 parses the second syntactic element indicating whether the current block is partitioned using a quadtree structure.
[0347] As described in embodiments 1 to 9 above, when the first syntactic element indicates that the current block is not divided, the syntactic element parsing unit 1601 may call the encoding unit syntax for decoding processing of the current block.
[0348] When the second syntactic element indicates that the current block is not partitioned using a quadtree structure, the syntactic element parsing unit 1601 parses at least one of the third syntactic element indicating whether the current block is partitioned using a binary tree structure or a ternary tree structure and the fourth syntactic element indicating the partitioning direction of the current block.
[0349] The partitioning pattern determination unit 1602 determines the partitioning pattern (or partitioning type) of the current block based on at least one of the first, second, third, and fourth syntactic elements. As an example, the partitioning type can be determined based on the first, second, third, and fourth syntactic elements as shown in Tables 4, 5, 8, 9, 12, 13, and 17 above.
[0350] Additionally, as described in embodiments 6 to 9 above, the partitioning pattern determination unit 1602 may invoke the coding tree unit syntax to determine the partitioning structure of the sub-blocks partitioned from the current block based on the partitioning pattern.
[0351] In addition, as described in Embodiment 9 above, the first syntactic element, the second syntactic element, the third syntactic element, and the fourth syntactic element can be parsed from the same level of syntax.
[0352] Additionally, as described in Embodiment 7 above, the partitioning mode determination unit 1602 can parse the fifth syntactic element indicating whether the current block is partitioned using a quadtree structure, which indicates whether the current block is partitioned using a binary tree structure or a ternary tree structure from the block of the higher node.
[0353] Figure 17 The video encoding system using this disclosure is shown.
[0354] A video encoding system may include a source device and a receiving device. The source device may forward encoded video / image information or data to the receiving device in file or stream format via digital storage media or a network.
[0355] The source device may include a video source, an encoding device, and a transmitter. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, which may be configured as a stand-alone device or an external component.
[0356] Video sources can be obtained through processes such as capture, compositing, or generation to produce video / images. Video sources may include video / image capture devices and / or video / image generation devices. For example, a video / image capture device may include one or more cameras, a video / image archive comprising previously captured video / images, etc. A video / image generation device may include, for example, a computer, tablet computer, and smartphone, and may generate video / images, for example (electrically). For example, virtual video / images can be generated by a computer, and in this case, the video / image capture process may be replaced by a process that generates related data.
[0357] Encoding devices can encode input video / images. For compression and encoding efficiency, encoding devices can perform a series of processes including prediction, transformation, quantization, etc.
[0358] The transmitter can forward encoded video / image information or data, output in bitstream format, to the receiver of the receiving device in file or stream format via digital storage media or a network. Digital storage media may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter may include elements for generating media files in a predetermined file format and may include elements for transmission via broadcast / communication networks. The receiver can extract the bitstream and forward it to a decoding device.
[0359] Decoding devices can perform a series of processes (including dequantization, inverse transform, prediction, etc.) corresponding to the operations of encoding devices and decode video / images.
[0360] The renderer can render decoded video / images. The rendered video / images can be displayed through the display unit.
[0361] Figure 18 This is a configuration diagram of a content streaming system that applies an embodiment of the present disclosure.
[0362] Content streaming systems employing this disclosure may include encoding servers, streaming servers, web servers, media storage devices, user devices, and multimedia input devices.
[0363] An encoding server is used to compress content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data to generate a bitstream and then send that bitstream to a streaming server. As another example, when multimedia input devices such as smartphones, cameras, and camcorders directly generate bitstreams, the encoding server can be omitted.
[0364] Bitstreams can be generated by applying the encoding method or bitstream generation method disclosed herein, and the stream server can temporarily store the bitstreams during the processing of sending or receiving bitstreams.
[0365] The streaming server sends multimedia data to the user's device via a web server based on user requests, and the web server acts as a medium for informing the user of services. When a user sends a request for a desired service to the web server, the web server forwards the request to the streaming server, and the streaming server sends the multimedia data to the user. Here, the content streaming system may include an additional control server, and in this case, the control server is used to control the commands / responses between devices in the content streaming system.
[0366] A streaming server can receive content from media storage devices and / or encoding servers. For example, when receiving content from an encoding server, the streaming server can receive content in real time. In this case, the streaming server can store the bitstream for a predetermined period of time to provide a smooth streaming service.
[0367] Examples of user equipment may include cellular phones, smartphones, laptops, digital broadcast terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation devices, Slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, and HMDs (head-mounted displays)), digital TVs, desktop computers, digital signage, etc.
[0368] In a content streaming system, each server can operate as a distributed server, and in this case, the data received by each server can be processed in a distributed manner.
[0369] The embodiments described in this disclosure can be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional units shown in the figures can be implemented and executed on a computer, processor, microprocessor, controller, or chip.
[0370] Furthermore, the decoders and encoders disclosed herein can be included in multimedia broadcasting transmitting and receiving devices, mobile communication terminals, home theater video devices, digital cinema video devices, cameras for surveillance, video conferencing devices, real-time communication devices such as video communication, mobile streaming devices, storage media, video-on-demand (VoD) service providers, over-the-top (OTT) video devices, internet streaming service providers, three-dimensional (3D) video devices, video telephony devices, and medical video devices, and can be used to process video signals or data signals. For example, OTT video devices may include game consoles, Blu-ray players, internet access TVs, home theater systems, smartphones, tablet PCs, and digital video recorders (DVRs).
[0371] Furthermore, the processing methods of this disclosure can be generated in the form of a program executed by a computer and can be stored in a computer-readable recording medium. Multimedia data having the data structure according to this disclosure can also be stored in a computer-readable recording medium. Computer-readable recording media include all types of storage devices that store computer-readable data. For example, computer-readable recording media may include Blu-ray discs (BD), Universal Serial Bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disks, and optical data storage devices. In addition, computer-readable recording media include media implemented in the form of carrier waves (e.g., transmission via the Internet). Furthermore, bitstreams generated using encoding methods can be stored in computer-readable recording media or transmitted via wired and wireless communication networks.
[0372] Furthermore, embodiments of this disclosure can be implemented as a computer program product using program code. The program code can be executed by a computer according to embodiments of this disclosure. The program code can be stored on a computer-readable medium.
[0373] In the above embodiments, the elements and features of this disclosure have been combined in a specific form. Unless explicitly described otherwise, each element or feature may be considered optional. Each element or feature may be implemented without being combined with other elements or features. Furthermore, some elements and / or features may be combined to form embodiments of this disclosure. The order of operations described in the embodiments of this disclosure may be changed. Some elements or features of an embodiment may be included in another embodiment, or may be replaced by corresponding elements or features of another embodiment. Obviously, embodiments may be constructed by combining claims that do not have an explicit referential relationship, or may be included as new claims after filing the application.
[0374] The embodiments of this disclosure can be implemented by various means, such as hardware, firmware, software, or combinations thereof. In the case of hardware implementation, the embodiments of this disclosure can be implemented using one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, etc.
[0375] When implemented via firmware or software, embodiments of this disclosure can be implemented as modules, processes, or functions that perform the functions or operations described above. The software code can be stored in memory and driven by a processor. The memory can be located inside or outside the processor and can exchange data with the processor through various known means.
[0376] It will be apparent to those skilled in the art that this disclosure may be implemented in other specific forms without departing from its essential characteristics. Therefore, the detailed description should not be construed as restrictive, but rather as illustrative in all respects. The scope of this disclosure should be determined by a reasonable analysis of the appended claims, and all modifications within the equivalent scope of this disclosure are included within its scope.
[0377] Industrial applicability
[0378] The above preferred embodiments of this disclosure are disclosed for illustrative purposes. Various other embodiments may be improved, modified, substituted or added by those skilled in the art without departing from the technical spirit and scope of this disclosure as disclosed in the appended claims.
Claims
1. An apparatus for decoding video signals, the apparatus comprising: A memory configured to store the video signal; as well as The processor connected to the memory, The processor is configured as follows: Based on the horizontal coordinate value of the top-left sample of the current block plus the width of the current block being equal to or less than the width of the current screen, based on the vertical coordinate value of the top-left sample of the current block plus the height of the current block being equal to or less than the height of the current screen, and based on the size of the current block being greater than the minimum size coded block, parse the first syntactic element including information indicating whether the current block is divided into multiple sub-blocks; Based on the first syntactic element including information indicating that the current block is partitioned, the second syntactic element including information indicating whether the current block is partitioned based on a quadtree structure is parsed. Based on the second syntactic element, which includes information indicating that the current block is not partitioned based on the quadtree structure, at least one of a third and a fourth syntactic element is parsed. The third syntactic element includes information indicating whether the current block is partitioned based on a binary tree or a ternary tree structure, and the fourth syntactic element includes information about the partitioning direction of the current block. The partitioning pattern of the current block is determined based on at least one of the first syntactic element, the second syntactic element, the third syntactic element, and the fourth syntactic element.
2. The device according to claim 1, wherein the processor is further configured to: Based on the information included in the first syntactic element indicating that the current block has not been divided, the encoding unit syntax for decoding processing of the current block is invoked.
3. The device according to claim 1, wherein, If the value of the first syntactic element is 0, the current block is not divided; if the value of the first syntactic element is 1, the current block is divided into 4 sub-blocks based on the quadtree structure, or into 2 sub-blocks based on the binary tree structure, or into 3 sub-blocks based on the ternary tree structure.
4. The device according to claim 3, wherein the processor is further configured to: The encoding tree unit syntax is invoked to determine the partitioning structure of sub-blocks from the current block based on the partitioning pattern.
5. The device according to claim 1, wherein, The first syntactic element, the second syntactic element, the third syntactic element, and the fourth syntactic element are parsed from the same level of syntax.
6. The device according to claim 1, wherein the processor is further configured to: Parse the fifth syntactic element, which includes information indicating whether the current block is partitioned based on the quadtree structure, or based on the binary tree structure or the ternary tree structure, from the higher node.
7. An apparatus for encoding video signals, the apparatus comprising: A memory configured to store the video signal; as well as The processor connected to the memory, The processor is configured as follows: Determine the partitioning pattern of the current block: Based on the determined partitioning pattern, at least one of the first syntactic element, the second syntactic element, the third syntactic element, and the fourth syntactic element is generated. When the horizontal coordinate of the top-left sample of the current block plus the width of the current block is equal to or less than the width of the current screen, the vertical coordinate of the top-left sample of the current block plus the height of the current block is equal to or less than the height of the current screen, and the size of the current block is greater than the smallest coded block, the first syntactic element including information indicating whether the current block is divided into multiple sub-blocks is encoded. When the first syntactic element includes information indicating that the current block is partitioned, the second syntactic element, including information indicating whether the current block is partitioned based on a quadtree structure, is encoded; and When the second syntactic element includes information indicating that the current block is not partitioned based on the quadtree structure, at least one of the third and fourth syntactic elements is encoded, the third syntactic element including information indicating whether the current block is partitioned based on a binary tree structure or a ternary tree structure, and the fourth syntactic element including information about the partitioning direction of the current block.
8. A computer-readable medium for storing one or more instructions executable by one or more processors, said one or more instructions controlling a video data processing apparatus to: Determine the partitioning pattern of the current block: Based on the determined partitioning pattern, at least one of the first syntactic element, the second syntactic element, the third syntactic element, and the fourth syntactic element is generated. When the horizontal coordinate of the top-left sample of the current block plus the width of the current block is equal to or less than the width of the current screen, the vertical coordinate of the top-left sample of the current block plus the height of the current block is equal to or less than the height of the current screen, and the size of the current block is greater than the smallest coded block, the first syntactic element including information indicating whether the current block is divided into multiple sub-blocks is encoded. When the first syntactic element includes information indicating that the current block is partitioned, the second syntactic element, which includes information indicating whether the current block is partitioned based on a quadtree structure, is encoded. as well as When the second syntactic element includes information indicating that the current block is not partitioned based on the quadtree structure, at least one of the third and fourth syntactic elements is encoded, the third syntactic element including information indicating whether the current block is partitioned based on a binary tree structure or a ternary tree structure, and the fourth syntactic element including information about the partitioning direction of the current block.
9. An apparatus for transmitting data for an image, the apparatus comprising: A processor configured to obtain a bitstream for the image; as well as A transmitter configured to send the data including the bit stream. The processor is configured as follows: Obtain the bitstream used for the image; and Send the data including the bit stream. In order to obtain the bit stream, the processor is configured to: Determine the partitioning pattern of the current block: Based on the determined partitioning pattern, at least one of the first syntactic element, the second syntactic element, the third syntactic element, and the fourth syntactic element is generated. When the value of the horizontal coordinate of the top left sample of the current block plus the width of the current block is equal to or less than the width of the current screen, and the value of the vertical coordinate of the top left sample of the current block plus the height of the current block is equal to or less than the height of the current screen, and the size of the current block is greater than the smallest coded block, the first syntactic element including information indicating whether the current block is divided into multiple sub-blocks is encoded. When the first syntactic element includes information indicating that the current block is partitioned, the second syntactic element, including information indicating whether the current block is partitioned based on a quadtree structure, is encoded; and The second syntactic element includes information indicating that the current block is not partitioned based on the quadtree structure. At least one of the third and fourth syntactic elements is encoded. The third syntactic element includes information indicating whether the current block is partitioned based on a binary tree structure or a ternary tree structure. The fourth syntactic element includes information about the partitioning direction of the current block.
10. A computer-readable medium storing one or more instructions and a bit stream, wherein when the one or more instructions are executed by one or more processors, an encoding method is performed to generate the bit stream, and the encoding method includes: Determine the partitioning pattern of the current block: Based on the determined partitioning pattern, at least one of the first syntactic element, the second syntactic element, the third syntactic element, and the fourth syntactic element is generated. When the horizontal coordinate of the top-left sample of the current block plus the width of the current block is equal to or less than the width of the current screen, the vertical coordinate of the top-left sample of the current block plus the height of the current block is equal to or less than the height of the current screen, and the size of the current block is greater than the smallest coded block, the first syntactic element including information indicating whether the current block is divided into multiple sub-blocks is encoded. When the first syntactic element includes information indicating that the current block is partitioned, the second syntactic element, which includes information indicating whether the current block is partitioned based on a quadtree structure, is encoded. as well as When the second syntactic element includes information indicating that the current block is not partitioned based on the quadtree structure, at least one of the third and fourth syntactic elements is encoded, the third syntactic element including information indicating whether the current block is partitioned based on a binary tree structure or a ternary tree structure, and the fourth syntactic element including information about the partitioning direction of the current block.
Citation Information
Patent Citations
Simplified non-square quadtree transforms for video coding
US20130272381A1
Binary ternary quad tree partitioning for jvet
US20170347128A1
