Image encoding / decoding method and apparatus and recording medium storing bitstream

Abstract

The image decoding method and apparatus according to the present disclosure can determine whether to apply a geometric partition mode to a current block based on a size of the current block and predetermined threshold size information, generate a prediction block for the current block based on the geometric partition mode if it is determined to apply the geometric partition mode to the current block, and reconstruct the current block based on the prediction block.

Description

Technical Field

[0001] This disclosure relates to an image encoding / decoding method and apparatus, as well as a recording medium for storing bit streams. Background Technology

[0002] Recently, the demand for high-resolution and high-quality images, such as HD (high-definition) and UHD (ultra-high-definition) images, has been increasing in various application areas, and therefore, efficient image compression technologies are being discussed.

[0003] Various techniques exist, such as inter-frame prediction techniques that use video compression technology to predict pixel values ​​included in the current image from images before or after the current image, intra-frame prediction techniques that use pixel information in the current image to predict pixel values ​​included in the current image, and entropy coding techniques that assign short symbols to values ​​that occur frequently and long symbols to values ​​that occur infrequently. These image compression techniques can be used to efficiently compress image data and send or store it. Summary of the Invention

[0004] Technical issues

[0005] This disclosure provides a method and apparatus for applying a geometric partitioning pattern.

[0006] This disclosure provides a method and apparatus for signaling syntax elements associated with a geometric partitioning pattern.

[0007] Technical solution

[0008] The image decoding method and apparatus according to this disclosure can determine whether to apply a geometric partitioning pattern to the current block based on the size of the current block and a predetermined threshold size. When it is determined that a geometric partitioning pattern should be applied to the current block, a prediction block of the current block can be generated based on the geometric partitioning pattern, and the current block can be reconstructed based on the prediction block.

[0009] In the image decoding method and apparatus according to the present disclosure, the threshold size information may include at least one of the following: minimum GPM block size with respect to the minimum size of the coding unit to which the geometric partitioning pattern is applicable, maximum GPM block size with respect to the maximum size of the coding unit to which the geometric partitioning pattern is applicable, or the ratio between the block width and the block height to which the geometric partitioning pattern is applicable.

[0010] In the image decoding method and apparatus according to this disclosure, the ratio between block width and block height can be applied differently depending on the size of the current block.

[0011] In the image decoding method and apparatus according to the present disclosure, at least one of the following can be signaled from the bitstream: information about the minimum GPM block size, information about the maximum GPM block size, or information about the ratio between the block width and the block height.

[0012] In the image decoding method and apparatus according to the present disclosure, the minimum GPM block size can be derived based on a value obtained by adding a first value to a value containing information about the minimum GPM block size, and the maximum GPM block size can be derived based on a value obtained by adding a second value to a value containing information about the maximum GPM block size.

[0013] In the image decoding method and apparatus according to the present disclosure, the minimum GPM block size can be derived based on a value obtained by adding a first value to a value containing information about the minimum GPM block size, and the maximum GPM block size can be derived based on a value obtained by adding a first value to a value containing information about the minimum GPM block size and a value containing information about the maximum GPM block size.

[0014] In the image decoding method and apparatus according to this disclosure, the ratio between block width and block height can be derived based on a value obtained by adding a third value to a value containing information about the ratio between block width and block height.

[0015] In the image decoding method and apparatus according to the present disclosure, when generating a prediction block for the current block, the number of available mixing regions for the current block can be determined based on the size of the current block.

[0016] In the image decoding method and apparatus according to the present disclosure, when generating a prediction block for the current block, the application of at least one of a plurality of detailed modes based on the geometric partitioning pattern can be restricted according to the size of the current block.

[0017] In the image decoding method and apparatus according to the present disclosure, the multiple detailed modes may include at least one of the following: an adaptive blending mode that performs blending by selectively using any of a multiple blending region sizes; a GPM intra-frame mode that applies intra-frame prediction to at least one of a multiple partitions in the current block; a GPM MMVD (merging using motion vector differences) mode that applies MMVD to at least one of a multiple partitions; a GPM TM (template matching) mode that applies TM to at least one of a multiple partitions; or a GPM affine mode that performs affine inter-frame prediction for each of a multiple partitions.

[0018] The image coding method and apparatus of this disclosure can determine whether to apply a geometric partitioning pattern to the current block based on the size of the current block and a predetermined threshold size. When it is determined that a geometric partitioning pattern should be applied to the current block, a prediction block of the current block can be generated based on the geometric partitioning pattern, a residual block of the current block can be generated based on the prediction block, the transform coefficients of the current block can be derived based on the residual block, and the residual information of the transform coefficients can be encoded.

[0019] A computer-readable digital storage medium is provided for storing encoded video / image information, which enables an apparatus according to the present disclosure to perform a method according to the present disclosure.

[0020] A computer-readable digital storage medium is provided for storing video / image information generated according to the method of this disclosure.

[0021] A method and apparatus are provided for transmitting video / image information generated according to the method of this disclosure.

[0022] Beneficial effects

[0023] According to this disclosure, image compression efficiency can be improved by adaptively using a geometric partitioning pattern that takes into account the size of the current block and a predetermined threshold size.

[0024] According to this disclosure, threshold size information can be efficiently communicated using signals. Attached Figure Description

[0025] Figure 1 A video / image encoding system according to this disclosure is shown.

[0026] Figure 2 A schematic block diagram of an encoding apparatus that can be applied to embodiments of the present disclosure and perform encoding of video / image signals is shown.

[0027] Figure 3 A schematic block diagram of a decoding apparatus that can be implemented using embodiments of the present disclosure and perform decoding of video / image signals is shown.

[0028] Figure 4 An image decoding method performed by a decoding device 300 as an embodiment of the present disclosure is illustrated.

[0029] Figure 5 An illustrative configuration of a decoding device 300 performing a decoding method according to the present disclosure is shown.

[0030] Figure 6 An encoding method performed by the encoding device 200 as an embodiment of the present disclosure is illustrated.

[0031] Figure 7An illustrative configuration of an encoding device 200 performing an encoding method according to the present disclosure is shown.

[0032] Figure 8 Examples of content streaming systems to which embodiments of the present disclosure may be applied are illustrated.

[0033] Best practice

[0034] The image decoding method and apparatus according to this disclosure can determine whether to apply a geometric partitioning pattern to the current block based on the size of the current block and a predetermined threshold size. When it is determined that a geometric partitioning pattern should be applied to the current block, a prediction block of the current block can be generated based on the geometric partitioning pattern, and the current block can be reconstructed based on the prediction block.

[0035] In the image decoding method and apparatus according to the present disclosure, the threshold size information may include at least one of the following: minimum GPM block size with respect to the minimum size of the coding unit to which the geometric partitioning pattern is applicable, maximum GPM block size with respect to the maximum size of the coding unit to which the geometric partitioning pattern is applicable, or the ratio between the block width and the block height to which the geometric partitioning pattern is applicable.

[0036] In the image decoding method and apparatus according to this disclosure, the ratio between block width and block height can be applied differently depending on the size of the current block.

[0037] In the image decoding method and apparatus according to the present disclosure, at least one of the following can be signaled from the bitstream: information about the minimum GPM block size, information about the maximum GPM block size, or information about the ratio between the block width and the block height.

[0038] In the image decoding method and apparatus according to the present disclosure, the minimum GPM block size can be derived based on a value obtained by adding a first value to a value containing information about the minimum GPM block size, and the maximum GPM block size can be derived based on a value obtained by adding a second value to a value containing information about the maximum GPM block size.

[0039] In the image decoding method and apparatus according to the present disclosure, the minimum GPM block size can be derived based on a value obtained by adding a first value to a value containing information about the minimum GPM block size, and the maximum GPM block size can be derived based on a value obtained by adding a first value to a value containing information about the minimum GPM block size and a value containing information about the maximum GPM block size.

[0040] In the image decoding method and apparatus according to this disclosure, the ratio between block width and block height can be derived based on a value obtained by adding a third value to a value containing information about the ratio between block width and block height.

[0041] In the image decoding method and apparatus according to the present disclosure, when generating a prediction block for the current block, the number of available mixing regions for the current block can be determined based on the size of the current block.

[0042] In the image decoding method and apparatus according to the present disclosure, when generating a prediction block for the current block, the application of at least one of a plurality of detailed modes based on the geometric partitioning pattern can be restricted according to the size of the current block.

[0043] In the image decoding method and apparatus according to the present disclosure, the multiple detailed modes may include at least one of the following: an adaptive blending mode that performs blending by selectively using any of a multiple blending region sizes; a GPM intra-frame mode that applies intra-frame prediction to at least one of a multiple partitions in the current block; a GPM MMVD (merging using motion vector differences) mode that applies MMVD to at least one of a multiple partitions; a GPM TM (template matching) mode that applies TM to at least one of a multiple partitions; or a GPM affine mode that performs affine inter-frame prediction for each of a multiple partitions.

[0044] The image coding method and apparatus of this disclosure can determine whether to apply a geometric partitioning pattern to the current block based on the size of the current block and a predetermined threshold size. When it is determined that a geometric partitioning pattern should be applied to the current block, a prediction block of the current block can be generated based on the geometric partitioning pattern, a residual block of the current block can be generated based on the prediction block, the transform coefficients of the current block can be derived based on the residual block, and the residual information of the transform coefficients can be encoded.

[0045] A computer-readable digital storage medium is provided for storing encoded video / image information, which enables an apparatus according to the present disclosure to perform a method according to the present disclosure.

[0046] A computer-readable digital storage medium is provided for storing video / image information generated according to the method of this disclosure.

[0047] A method and apparatus are provided for transmitting video / image information generated according to the method of this disclosure. Detailed Implementation

[0048] Because this disclosure can be modified in various ways and has multiple embodiments, specific embodiments will be shown in the accompanying drawings and described in detail in the specific embodiments. However, this disclosure is not intended to be limited to the specific embodiments, but should be understood to include all changes, equivalents, and substitutions included within the spirit and scope of this disclosure. Similar reference numerals are used for similar components in the description of the various figures.

[0049] Terms such as "first," "second," etc., may be used to describe various components, but components should not be limited by these terms. Terms are used only to distinguish one component from others. For example, without departing from the scope of this disclosure, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component. The term "and / or" includes any combination of one or more of the associated items.

[0050] When a component is described as "connected" or "linked" to another component, it should be understood that it can be directly connected or linked to the other component, but the other component can exist in between. On the other hand, when a component is described as "directly connected" or "directly linked" to another component, it should be understood that there is no other component in between.

[0051] The terminology used in this application is for describing particular embodiments only and is not intended to limit this disclosure. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, it should be understood that terms such as “comprising” or “having” are intended to specify the presence of the features, quantities, steps, operations, components, portions, or combinations thereof described in this specification, but do not preclude the possibility of the presence or addition of one or more other features, quantities, steps, operations, components, portions, or combinations thereof.

[0052] This disclosure relates to video / image coding. For example, the methods / implementations disclosed herein can be applied to methods disclosed in the Multifunctional Video Coding (VVC) standard. Additionally, the methods / implementations disclosed herein can be applied to methods disclosed in the Basic Video Coding (EVC) standard, the AOMedia Video 1 (AV1) standard, the Audio Video Coding 2 (AVS2) standard, or next-generation video / image coding standards (e.g., H.267 or H.268).

[0053] This specification sets forth various implementations of video / image encoding, and unless otherwise specified, these implementations may be combined with each other.

[0054] In this article, video can refer to a collection of images over time. An image typically refers to a unit representing an image within a specific time period, and a slice / tile is a unit that forms part of an image during encoding. A slice / tile can include at least one Code Tree Unit (CTU). An image can consist of at least one slice / tile. A tile is a rectangular area composed of multiple CTUs within a specific tile column and a specific tile row of an image. A tile column is a rectangular area of ​​CTUs with a height equal to the height of the image and a width specified by the syntax requirements of the image parameter set. A tile row is a rectangular area of ​​CTUs with a height specified by the image parameter set and a width equal to the width of the image. CTUs within a tile can be arranged continuously according to a CTU raster scan, and tiles within an image can be arranged continuously according to a tile raster scan. A slice can include an integer number of complete tiles of an image that can be exclusively included in a single NAL unit, or an integer number of consecutive complete CTU rows within a tile. Furthermore, an image can be divided into at least two sub-images. A sub-image can be a rectangular area of ​​at least one slice within an image.

[0055] A pixel, or pel, can represent the smallest unit that makes up a picture (or image). Additionally, the term "sample" can be used as the counterpart to a pixel. A sample can typically represent a pixel or a pixel value, and can represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chrominance component.

[0056] A unit can represent a basic unit of image processing. A unit may include a specific region of an image and at least one of the information associated with that region. A unit may include a luminance block and two chrominance (e.g., cb, cr) blocks. In some cases, the term "unit" may be used interchangeably with terms such as "block" or "region". In general, an M×N block may include a set (or array) of transform coefficients or samples (or sample arrays) consisting of M columns and N rows.

[0057] In this document, “A or B” can mean “A only”, “B only”, or “both A and B”. In other words, “A or B” can be interpreted as “A and / or B”. For example, “A, B or C” can mean “A only”, “B only”, “C only”, or “any combination of A, B and C”.

[0058] The forward slash ( / ) or comma used in this article can indicate "and / or". For example, "A / B" can mean "A and / or B". Therefore, "A / B" can mean "A only", "B only", or "both A and B". For example, "A, B, C" can mean "A, B, or C".

[0059] In this document, "at least one of A and B" can mean "only A", "only B" or "both A and B". Furthermore, in this document, expressions such as "at least one of A or B" or "at least one of A and / or B" can be interpreted in the same way as "at least one of A and B".

[0060] Additionally, in this document, "at least one of A, B, and C" can mean "A only", "B only", "C only" or "any combination of A, B, and C". Furthermore, "at least one of A, B, or C" or "at least one of A, B, and / or C" can mean "at least one of A, B, and C".

[0061] Additionally, the parentheses used in this document can indicate "for example". Specifically, when the indication is "prediction (intra-frame prediction)", "intra-frame prediction" can be cited as an example of "prediction". In other words, "prediction" in this document is not limited to "intra-frame prediction", and "intra-frame prediction" can be cited as an example of "prediction". Furthermore, even when the indication is "prediction (i.e., intra-frame prediction)", "intra-frame prediction" can be cited as an example of "prediction".

[0062] In this article, the technical features described individually in a single diagram can be implemented individually or simultaneously.

[0063] Figure 1 A video / image encoding system according to this disclosure is shown.

[0064] Reference Figure 1 A video / image encoding system may include a first device (source device) and a second device (receiving device).

[0065] A source device can transmit encoded video / image information or data to a receiving device in the form of a file or stream via a digital storage medium or network. The source device may include a video source, an encoding device, and a transmitting unit. The receiving device may include a receiving unit, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, and the display unit may consist of a separate device or external components.

[0066] A video source can acquire video / images through processes that capture, synthesize, or generate video / images. A video source may include means for capturing video / images and means for generating video / images. Means for capturing video / images may include at least one camera, a video / image archive containing previously captured video / images, etc. Means for generating video / images may include a computer, tablet computer, smartphone, etc., and can generate video / images (electronically). For example, virtual video / images can be generated by a computer, etc., and in this case, the process of capturing video / images can be replaced by a process of generating related data.

[0067] Encoding devices can encode input video / images. They can perform a series of processes such as prediction, transformation, and quantization for compression and encoding efficiency. The encoded data (encoded video / image information) can be output as a bitstream.

[0068] The transmitting unit can send encoded video / image information or data, output in bitstream form, to the receiving unit of the receiving device in the form of a file or stream via a digital storage medium or network. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitting unit can include elements for generating media files according to a predetermined file format, and may include elements for transmission via a broadcast / communication network. The receiving unit can receive / extract the bitstream and send it to a decoding device.

[0069] Decoding devices can decode video / images by performing a series of processes, such as dequantization, inverse transform, and prediction, that correspond to the operations of encoding devices.

[0070] The renderer can render decoded video / images. The rendered video / images can be displayed through a display unit.

[0071] Figure 2 A rough block diagram of an encoding apparatus that can be applied to embodiments of the present disclosure and perform encoding of video / image signals is shown.

[0072] Reference Figure 2The encoding device 200 may consist of an image segmenter 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-frame predictor 221 and an intra-frame predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may also include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstruction block generator. According to embodiments, the image segmenter 210, predictor 220, residual processor 230, entropy encoder 240, adder 250, and filter 260 may be configured by at least one hardware component (e.g., an encoder chipset or processor). Additionally, the memory 270 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may also include the memory 270 as an internal / external component.

[0073] Image segmenter 210 can segment an input image (or picture or frame) input to encoding device 200 into at least one processing unit. As an example, a processing unit can be referred to as a coding unit (CU). In this case, coding units can be recursively partitioned from coding tree units (CTUs) or maximum coding units (LCUs) according to a quadtree-binary-tritree (QTBTTT) structure.

[0074] For example, a coding unit can be partitioned into multiple deeper coding units based on a quadtree, binary tree, and / or ternary tree structure. In this case, for example, a quadtree structure can be applied first, followed by a binary tree and / or ternary tree structure. Alternatively, a binary tree structure can be applied before the quadtree structure. The coding process according to this specification can be performed based on the final coding unit that is no longer partitioned. In this case, based on image characteristics, coding efficiency, etc., the largest coding unit can be directly used as the final coding unit, or, if necessary, the coding unit can be recursively partitioned into deeper coding units, and the coding unit with the optimal size can be used as the final coding unit. Here, the coding process can include processes such as prediction, transformation, and reconstruction, as described later.

[0075] As another example, the processing unit may also include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit can be divided or partitioned from the final coding unit described above. The prediction unit may be a unit for predicting samples, and the transform unit may be a unit for deriving transform coefficients and / or a unit for deriving residual signals from transform coefficients.

[0076] In some cases, a unit can be used interchangeably with terms such as block or region. Generally, an M×N block can represent a set of transform coefficients or samples consisting of M columns and N rows. Samples can typically represent pixels or pixel values, and can represent only the pixel / pixel value of the luminance component, or only the pixel / pixel value of the chrominance component. Samples can be used as a term to form a picture (or image) corresponding to a pixel or cell.

[0077] Encoding device 200 can subtract the prediction signal (prediction block, prediction sample array) output from inter-frame predictor 221 or intra-frame predictor 222 from the input image signal (original block, original sample array) to generate a residual signal (residual signal, residual sample array), and the generated residual signal is sent to converter 232. In this case, the unit in encoding device 200 that subtracts the prediction signal (prediction block, prediction sample array) from the input image signal (original block, original sample array) can be called subtractor 231.

[0078] Predictor 220 can perform prediction on the block to be processed (hereinafter referred to as the current block) and generate a prediction block that includes prediction samples of the current block. Predictor 220 can determine whether to apply intra-frame prediction or inter-frame prediction on a per-block or per-unit basis. Predictor 220 can generate various information about the prediction (e.g., prediction mode information) and send it to entropy encoder 240, as described later in the description of the various prediction modes. The information about the prediction can be encoded in entropy encoder 240 and output as a bitstream.

[0079] Intra-predictor 222 can predict the current block by referencing samples within the current image. Depending on the prediction mode, the referenced samples can be located near the current block or positioned at a specific distance away from the current block. In intra-prediction, the prediction mode can include at least one non-directional mode and multiple directional modes. The non-directional mode can include at least one of a DC mode or a planar mode. Depending on the level of detail of the prediction direction, the directional modes can include 33 or 65 directional modes. However, this is just an example; more or fewer directional modes can be used depending on the configuration. Intra-predictor 222 can determine the prediction mode applied to the current block by using prediction modes applied to neighboring blocks.

[0080] Inter-frame predictor 221 can deduce the predicted block of the current block based on a reference block (reference sample array) specified by motion vectors on a reference image. In this case, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation between motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference image indices. Motion information may also include inter-frame prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). For inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing in the current image and temporally neighboring blocks existing in the reference image. The reference image including the reference block and the reference image including the temporally neighboring block may be the same or different. The temporally neighboring block may be referred to as a juxtaposed reference block, a co-located CU (colCU), etc., and the reference image including the temporally neighboring block may be referred to as a juxtaposed image (colPic). For example, inter-frame predictor 221 can configure a motion information candidate list based on neighboring blocks and generate information indicating which candidate to use to deduce the motion vector and / or reference image index of the current block. Inter-frame prediction can be performed based on various prediction modes. For example, in skip mode and merge mode, the inter-frame predictor 221 can use motion information from neighboring blocks as motion information for the current block. In skip mode, unlike merge mode, residual signals may not be sent. In motion vector prediction (MVP) mode, motion vectors from surrounding blocks are used as motion vector predictors, and the motion vector difference is signaled to indicate the motion vector of the current block.

[0081] Predictor 220 can generate a prediction signal based on various prediction methods described later. For example, the predictor can not only apply intra-frame prediction or inter-frame prediction to predict a block, but can also apply both intra-frame and inter-frame prediction simultaneously. This can be referred to as the Inter-intra-frame Combined Prediction (CIIP) mode. Alternatively, the predictor can predict blocks based on the Intra-Block Copy (IBC) prediction mode, or it can predict blocks based on a palette mode. The IBC prediction mode or palette mode can be used for content image / video coding such as Screen Content Coding (SCC) in games, etc. IBC essentially performs prediction within the current frame, but it can be performed similarly to inter-frame prediction because it derives a reference block within the current frame. In other words, IBC can use at least one of the inter-frame prediction techniques described herein. The palette mode can be considered as an example of intra-frame coding or intra-frame prediction. When applying a palette mode, sample values ​​within the image can be signaled based on information about the palette table and palette index. The prediction signal generated by predictor 220 can be used to generate a reconstructed signal or a residual signal.

[0082] Transformer 232 can generate transform coefficients by applying transform techniques to the residual signal. For example, the transform techniques may include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen–Loève Transform (KLT), Graph-Based Transform (GBT), or Conditional Nonlinear Transform (CNT). Here, GBT represents the transform obtained from a graph when the relationship information between pixels is represented as a graph. CNT represents the transform obtained based on generating a prediction signal using all previously reconstructed pixels. Furthermore, the transform processing can be applied to square pixel blocks of the same size, or it can be applied to non-square blocks of variable size.

[0083] Quantizer 233 can quantize the transform coefficients and send them to entropy encoder 240, which can encode the quantized signal (information about the quantized transform coefficients) and output it as a bitstream. This information about the quantized transform coefficients can be called residual information. Quantizer 233 can rearrange the block-form quantized transform coefficients into a one-dimensional vector form based on the coefficient scan order, and can generate information about the quantized transform coefficients based on this one-dimensional vector form.

[0084] The entropy encoder 240 can perform various encoding methods such as Golomb, context-adaptive variable-length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). The entropy encoder 240 can encode information required for video / image reconstruction other than quantization transform coefficients (e.g., values ​​of syntax elements, etc.) together or separately.

[0085] Encoded information (e.g., encoded video / image information) can be transmitted or stored in bitstream form at the Network Abstraction Layer (NAL) unit level. The video / image information may also include information about various parameter sets such as Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). Additionally, the video / image information may include general constraint information. Information and / or syntax elements transmitted from the encoding device / signed to the decoding device may be included in the video / image information. The video / image information can be encoded and included in the bitstream through the encoding process described above. The bitstream can be transmitted over a network or stored in a digital storage medium. Here, the network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitting unit (not shown) for transmitting the signal output from the entropy encoder 240 and / or a storage unit (not shown) for storing the signal may be configured as internal / external components of the encoding device 200, or the transmitting unit may also be included in the entropy encoder 240.

[0086] The quantized transform coefficients output from quantizer 233 can be used to generate a prediction signal. For example, the residual signal (residual block or residual sample) can be reconstructed by applying dequantization and inverse transform to the quantized transform coefficients via dequantizer 234 and inverse transformer 235. Adder 250 can add the reconstructed residual signal to the prediction signal output from inter-frame predictor 221 or intra-frame predictor 222 to generate a reconstructed signal (reconstructed image, reconstructed block, reconstructed sample array). When there is no residual for the block to be processed (similar to when a skip mode is applied), the prediction block can be used as a reconstructed block. Adder 250 can be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed within the current image, and can also be used for inter-frame prediction of the next image by filtering, as described later. Furthermore, luminance mapping and chroma scaling (LMCS) can be applied in image encoding and / or reconstruction processing.

[0087] Filter 260 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 260 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image, and the modified reconstructed image can be stored in memory 270 (specifically, the DPB of memory 270). Various filtering methods can include deblocking filtering, sample adaptive offsetting, adaptive loop filtering, bilateral filtering, etc. Filter 260 can generate various information about the filtering and send it to entropy encoder 240. The information about the filtering can be encoded in entropy encoder 240 and output as a bitstream.

[0088] The modified reconstructed image sent to memory 270 can be used as a reference image in inter-frame predictor 221. When inter-frame prediction is applied through it, the encoding device can avoid prediction mismatch between the encoding device 200 and the decoding device, and can also improve encoding efficiency.

[0089] The DPB of memory 270 can store modified reconstructed images for use as reference images in inter-frame predictor 221. Memory 270 can store motion information of blocks in the current image from which motion information is derived (or encoded) and / or of blocks in the pre-reconstructed image. The stored motion information can be sent to inter-frame predictor 221 as motion information for spatially or temporally neighboring blocks. Memory 270 can store reconstructed samples of reconstructed blocks in the current image and send them to intra-frame predictor 222.

[0090] Figure 3 A rough block diagram of a decoding device that can be implemented using embodiments of the present disclosure and perform decoding of video / image signals is shown.

[0091] Reference Figure 3 The decoding device 300 can be configured to include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-frame predictor 332 and an intra-frame predictor 331. The residual processor 320 may include a dequantizer 321 and an inverse transformer 322.

[0092] According to the implementation, the entropy decoder 310, residual processor 320, predictor 330, adder 340, and filter 350 described above can be configured by a single hardware component (e.g., a decoder chipset or processor). Additionally, the memory 360 may include a decoded image buffer (DPB) and can be configured by a digital storage medium. The hardware component may also include the memory 360 as an internal / external component.

[0093] When the input includes a bitstream containing video / image information, the decoding device 300 can respond to... Figure 2 The decoding device 300 processes video / image information in an encoding device to reconstruct the image. For example, the decoding device 300 can deduce units / blocks based on block partitioning information obtained from the bitstream. The decoding device 300 can perform decoding using processing units applied in the encoding device. Therefore, the decoding processing unit can be an encoding unit, and the encoding unit can be obtained from encoding tree units or larger encoding unit partitions according to a quadtree structure, binary tree structure, and / or ternary tree structure. At least one transform unit can be derived from the encoding unit. Furthermore, the reconstructed image signal decoded and output by the decoding device 300 can be played back by a playback device.

[0094] Decoding device 300 can receive data in bitstream form from... Figure 2The signal output by the encoding device can be decoded by the entropy decoder 310. For example, the entropy decoder 310 can parse the bitstream to derive the information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets such as adaptive parameter sets (APS), picture parameter sets (PPS), sequence parameter sets (SPS), or video parameter sets (VPS). In addition, the video / image information may also include general constraint information. The decoding device can also decode the picture based on the information about the parameter sets and / or general constraint information. The information and / or syntax elements notified / received by signals, as described later herein, can be decoded and obtained from the bitstream through the decoding process. For example, the entropy decoder 310 can decode the information in the bitstream based on encoding methods such as exponential Golomb coding, CAVLC, CABAC, etc., and output the values ​​of the syntax elements required for image reconstruction and the quantized values ​​of the transform coefficients of the residuals. More specifically, the CABAC entropy decoding method can receive bins corresponding to each syntax element from the bitstream, determine a context model using information about the syntax element to be decoded, decoding information of the block to be decoded, information about surrounding blocks and the block to be decoded, or information about symbols / bins decoded in the previous step, perform arithmetic decoding of bins by predicting the occurrence probability of bins based on the determined context model, and generate symbols corresponding to the values ​​of each syntax element. In this case, after determining the context model, the CABAC entropy decoding method can update the context model by using information about the decoded symbols / bins for the context model of the next symbol / bin. Among the information decoded in the entropy decoder 310, information about prediction is provided to the predictors (inter-frame predictor 332 and intra-frame predictor 331), and the residual values ​​(i.e., quantization transform coefficients and related parameter information) from which entropy decoding has been performed in the entropy decoder 310 can be input to the residual processor 320. The residual processor 320 can derive residual signals (residual blocks, residual samples, residual sample arrays). Additionally, filtering information from the information decoded in the entropy decoder 310 can be provided to the filter 350. Furthermore, the receiving unit (not shown) that receives the signal output from the encoding device can be further configured as an internal / external component of the decoding device 300, or the receiving unit can be a component of the entropy decoder 310.

[0095] Furthermore, the decoding device according to this specification may be referred to as a video / image / picture decoding device, and the decoding device may be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include an entropy decoder 310, and the sample decoder may include at least one of a dequantizer 321, an inverse transformer 322, an adder 340, a filter 350, a memory 360, an inter-frame predictor 332, and an intra-frame predictor 331.

[0096] Dequantizer 321 can dequantize the quantized transform coefficients and output the transform coefficients. Dequantizer 321 can rearrange the quantized transform coefficients into two-dimensional blocks. In this case, the rearrangement can be performed based on the coefficient scan order performed in the encoding device. Dequantizer 321 can perform dequantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information) and obtain the transform coefficients.

[0097] The inverse transformer 322 performs an inverse transformation on the transformation coefficients to obtain the residual signal (residual block, residual sample array).

[0098] Predictor 320 can perform prediction on the current block and generate a prediction block that includes prediction samples of the current block. Predictor 320 can determine whether to apply intra-frame prediction or inter-frame prediction to the current block based on the prediction information output from entropy decoder 310, and determine the specific intra-frame / inter-frame prediction mode.

[0099] Predictor 320 can generate prediction signals based on various prediction methods described later. For example, predictor 320 can not only apply intra-frame prediction or inter-frame prediction to predict a block, but can also apply intra-frame prediction and inter-frame prediction simultaneously. This can be referred to as the Inter-Frame Intra-Frame Combined Prediction (CIIP) mode. Alternatively, the predictor can predict blocks based on the Intra-Frame Block Copy (IBC) prediction mode, or it can predict blocks based on a palette mode. The IBC prediction mode or palette mode can be used for content image / video coding such as Screen Content Coding (SCC) in games, etc. IBC essentially performs prediction within the current frame, but it can be performed similarly to inter-frame prediction because it derives a reference block within the current frame. In other words, IBC can use at least one of the inter-frame prediction techniques described herein. The palette mode can be considered an example of intra-frame coding or intra-frame prediction. When a palette mode is applied, information about the palette table and palette index can be included in the video / image information and signaled.

[0100] Intra-predictor 331 can predict the current block by referencing samples within the current image. Depending on the prediction mode, the referenced samples can be located near the current block or at a specific distance away. In intra-prediction, the prediction mode can include at least one non-directional mode and multiple directional modes. Intra-predictor 331 can determine the prediction mode applied to the current block by using prediction modes applied to neighboring blocks.

[0101] Inter-frame predictor 332 can deduce the predicted block of the current block based on a reference block (reference sample array) specified by motion vectors on a reference image. In this case, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation of motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference image indices. Motion information may also include inter-frame prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). For inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing in the current image and temporally neighboring blocks existing in the reference image. For example, inter-frame predictor 332 can configure a motion information candidate list based on neighboring blocks and deduce the motion vector and / or reference image index of the current block based on the received candidate selection information. Inter-frame prediction can be performed based on various prediction modes, and the information about the prediction may include information indicating the inter-frame prediction mode of the current block.

[0102] Adder 340 can add the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the predictor (including inter-frame predictor 332 and / or intra-frame predictor 331) to generate a reconstruction signal (reconstructed image, reconstruction block, reconstruction sample array). When there is no residual for the block to be processed (similar to when a skip mode is applied), the prediction block can be used as a reconstruction block.

[0103] Adder 340 can be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current image, can be output through filtering as described later, or can be used for inter-frame prediction of the next image. Furthermore, Luminance Mapping and Chroma Scaling (LMCS) can be applied in image decoding processing.

[0104] Filter 350 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 350 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image and send the modified reconstructed image to memory 360 (specifically, the DPB of memory 360). Various filtering methods may include deblocking filtering, sample adaptive shifting, adaptive loop filtering, bilateral filtering, etc.

[0105] The (modified) reconstructed image stored in the DPB of memory 360 can be used as a reference image in inter-frame prediction unit 332. Memory 360 can store motion information of blocks from which motion information in the current image is derived (or decoded) and / or motion information of blocks in the pre-reconstructed image. The stored motion information can be sent to inter-frame predictor 332 as motion information for spatially or temporally neighboring blocks. Memory 360 can store reconstructed samples of reconstructed blocks in the current image and send them to intra-frame predictor 331.

[0106] The embodiments described in this document in the filter 260, inter-frame predictor 221 and intra-frame predictor 222 of the encoding device 200 can also be applied equivalently or correspondingly to the filter 350, inter-frame predictor 332 and intra-frame predictor 331 of the decoding device 300, respectively.

[0107] Figure 4 An example is shown of a decoding method performed by a decoding device 300 as an embodiment of the present disclosure.

[0108] Reference Figure 4 It can determine whether to apply the geometric partitioning mode (S400) to the current block.

[0109] In inter-frame prediction based on geometric partitioning patterns, the complexity can increase due to the need to determine suitable geometric partition shapes for prediction. Therefore, this disclosure proposes a method for specifying the size and shape of blocks that maximizes compression efficiency while minimizing the increased complexity when performing inter-frame prediction based on geometric partitioning patterns.

[0110] Whether to apply a geometric partitioning mode to the current block can be determined based on the size of the current block and a predetermined threshold size.

[0111] The size of the current block can be defined as its width (CbWidth) and / or height (CbHeight).

[0112] The threshold size information may include at least one of the following: minimum GPM block size (GEO_MIN_CU_SIZE), maximum GPM block size (GEO_MAX_CU_SIZE), or the ratio between block width and block height (GEO_WH_RATIO).

[0113] The minimum GPM block size indicates the minimum size of a coding unit that can be applied to a geometric partitioning pattern. The maximum GPM block size indicates the maximum size of a coding unit that can be applied to a geometric partitioning pattern. Here, the minimum and / or maximum size of a coding unit can be defined as at least one of the width or height of the corresponding coding unit. The ratio between block width and block height can refer to the ratio between the width and height of a block that can be applied to a geometric partitioning pattern.

[0114] When a predetermined condition between the current block size and the threshold size information is met, it can be determined that a geometric partitioning pattern will be applied to the current block. The predetermined condition may include at least one of the first to sixth conditions described below.

[0115] (First condition) The width of the current block is greater than or equal to the minimum GPM block size.

[0116] (Second condition) The height of the current block is greater than or equal to the minimum GPM block size.

[0117] (Third condition) The width of the current block is less than or equal to the maximum GPM block size.

[0118] (Fourth condition) The height of the current block is less than or equal to the maximum GPM block size.

[0119] (Fifth condition) The width of the current block is less than or equal to the product of the ratio between the width and height of the GPM block and the height of the current block.

[0120] (Sixth condition) The height of the current block is less than or equal to the product of the ratio between the width and height of the GPM block and the width of the current block.

[0121] For example, the minimum GPM block size can be any of 4, 8, or 16, and can be greater than or equal to the minimum coding unit size. Similarly, the maximum GPM block size can be any of 32, 64, 128, or 256, and can be less than or equal to the maximum coding unit size. Furthermore, the ratio between block width and block height can be any of 2, 4, 8, or 16.

[0122] At least one of the minimum GPM block size, the maximum GPM block size, or the ratio between block width and block height can be encoded by the encoding device and signaled to the decoding device via a bitstream. Alternatively, at least one of the minimum GPM block size, the maximum GPM block size, or the ratio between block width and block height can be predefined as the same value in both the encoding and decoding devices.

[0123] The ratio between block width and block height can be expressed as either an N:1 ratio or a 1:N ratio. In other words, in the fifth condition, the ratio between block width and block height can be expressed as an N:1 ratio. Alternatively, in the sixth condition, the ratio between block width and block height can be expressed as a 1:N ratio.

[0124] When applying the geometric partitioning pattern to the current block, the ratio between the block width and block height can be applied differently depending on the size of the current block.

[0125] This section describes the case where 4 is used as the ratio between the block width and block height to support geometric partitioning patterns. In this case, assuming a minimum GPM block size of 8 and a maximum GPM block size of 64, geometric partitioning patterns can be applied to block sizes of 8x8, 8x16, 8x32, 32x8, 16x8, 16x16, 16x32, 16x64, 64x16, 32x16, 32x32, 32x64, 64x32, and 64x64.

[0126] Alternatively, the case where 8 is used as the ratio between the block width and block height to support geometric partitioning patterns will be described. In this case, assuming a minimum GPM block size of 8 and a maximum GPM block size of 64, geometric partitioning patterns can be applied to block sizes of 8x8, 8x16, 8x32, 8x64, 64x8, 32x8, 16x8, 16x16, 16x32, 16x64, 64x16, 32x16, 32x32, 32x64, 64x32, and 64x64.

[0127] As the difference between the minimum GPM block size and the maximum GPM block size increases, the number of additional blocks that can be applied to the geometric partitioning pattern may increase depending on the ratio between the block width and block height supporting the geometric partitioning pattern. Therefore, in terms of the trade-off between complexity and coding efficiency, the ratio between the block width and block height supporting the geometric partitioning pattern can be applied differently depending on the size of the current block.

[0128] For example, in a 128xN or Nx128 block, 4 can be used as the ratio between the block width and the block height, and for the remaining block sizes that are applicable to the geometric partitioning pattern, 8 can be used as the ratio between the block width and the block height.

[0129] Alternatively, in 4xN or Nx4 blocks, 4 can be used as the ratio between block width and block height, and for the remaining block sizes that are applicable to the geometric partitioning pattern, 8 can be used as the ratio between block width and block height.

[0130] In addition, when considering the trade-off between complexity and coding efficiency, the minimum GPM block size, the maximum GPM block size, and the ratio between block width and block height for supporting geometric partitioning modes can be applied differently depending on the size of the image being encoded and decoded and the frame rate.

[0131] For example, when the size of the image being encoded and decoded is greater than or equal to 1920x1080, 8 can be used as the minimum GPM block size, 64 can be used as the maximum GPM block size, and 8 can be used as the ratio between the block width and block height to support geometric partitioning modes.

[0132] On the other hand, when the size of the image being encoded and decoded is less than 1920x1080, 4 can be used as the minimum GPM block size, 32 can be used as the maximum GPM block size, and 4 can be used as the ratio between the block width and block height to support the geometric partitioning mode.

[0133] Threshold size information can be signaled within the high-level syntax of the bitstream. The high-level syntax can include at least one of SPS (Sequence Parameter Set), PPS (Picture Parameter Set), PH (Picture Header), or SH (Slice Header). However, for ease of description, the following assumes that the threshold size information is signaled within the SPS.

[0134] Table 1 shows the syntax table of syntax elements related to the minimum GPM block size, the maximum GPM block size, and the ratio between block width and block height, and Table 2 shows the semantics of the corresponding syntax elements.

[0135] [Table 1]

[0136] [Table 2]

[0137] As shown in Tables 1 and 2, the minimum GPM block size, the maximum GPM block size, and the ratio between block width and block height for geometric partitioning modes can be signaled as a Log2 value. This can be signaled as the remainder obtained by subtracting a predefined value from the Log2 value. In this case, the predefined value can be 2 or 3 for the minimum GPM block size. For the maximum GPM block size, the predefined value can be 5 or 6. For the ratio between block width and block height, the predefined value can be 2.

[0138] log2_gpm_min_cu_size_minus2 can contain information about the minimum GPM block size. The minimum GPM block size can be derived based on the value obtained by adding a predefined value (e.g., 2) to log2_gpm_min_cu_size_minus2.

[0139] log2_gpm_max_cu_size_minus5 can be information about the maximum GPM block size. The maximum GPM block size can be derived based on the value obtained by adding a predefined value (e.g., 5) to log2_gpm_max_cu_size_minus5.

[0140] log2_gpm_wh_cu_ratio_minus2 can be information about the ratio between the block width and block height for a geometric partitioning mode. The ratio between the block width and block height can be derived based on the value obtained by adding a predefined value (e.g., 2) to log2_gpm_wh_cu_ratio_minus2.

[0141] Table 3 shows the syntax table of syntax elements related to the minimum GPM block size, the maximum GPM block size, and the ratio between block width and block height, and Table 4 shows the semantics of the corresponding syntax elements.

[0142] [Table 3]

[0143] [Table 4]

[0144] As shown in Tables 3 and 4, the minimum GPM block size, the maximum GPM block size, and the ratio between the block width and block height for geometric partitioning modes can be signaled as Log2 values.

[0145] Here, in the case of the minimum GPM block size, it can be signaled as the remainder obtained by subtracting a predefined value from the Log2 value. Here, the predefined value can be 2 or 3. The maximum GPM block size can be signaled as the difference between the Log2 value of the maximum GPM block size and the Log2 value of the minimum GPM block size. Unlike Tables 1 and 2, the ratio between block width and block height can be signaled as the Log2 value of the ratio between block width and height, as is.

[0146] log2_gpm_min_cu_size_minus2 can contain information about the minimum GPM block size. The minimum GPM block size can be derived based on the value obtained by adding a predefined value (e.g., 2) to log2_gpm_min_cu_size_minus2.

[0147] log2_gpm_diff_max_min_size can be information about the maximum GPM block size. The maximum GPM block size can be derived based on the value obtained by adding the log2 value of the minimum GPM block size to log2_gpm_diff_max_min_size.

[0148] log2_gpm_wh_cu_ratio can be information about the ratio between the block width and block height for a geometric partitioning mode. The ratio between the block width and block height can be derived based on the value of log2_gpm_wh_cu_ratio.

[0149] Table 3 describes a method for signaling information about the minimum GPM block size before information about the maximum GPM block size, but this is only an example. For instance, information about the maximum GPM block size can be signaled before information about the minimum GPM block size. In this case, the minimum GPM block size can be signaled as the difference between the Log2 value of the maximum GPM block size and the Log2 value of the minimum GPM block size.

[0150] In other words, log2_gpm_diff_max_min_size can be signaled after log2_gpm_max_cu_size_minus5. In this case, the maximum GPM block size (GeoMaxCbSize) can be derived as shown in Equation 1 below, and the minimum GPM block size (GeoMinCbSize) can be derived as shown in Equation 2 below.

[0151] [Formula 1]

[0152] GeoMaxCbLog2Size = log2_gpm_max_cu_size_minus5 + 5

[0153] GeoMaxCbSize = 1 << GeoMaxCbLog2Size

[0154] [Formula 2]

[0155] GeoMinCbLog2Size = GeoMaxCbLog2Size - log2_gpm_diff_max_min_size

[0156] GeoMinCbSize = 1 << GeoMinCbLog2Size

[0157] Table 5 shows the syntax table of syntax elements related to the minimum GPM block size, the maximum GPM block size, and the ratio between block width and block height, and Table 6 shows the semantics of the corresponding syntax elements.

[0158] [Table 5]

[0159] [Table 6]

[0160] As shown in Tables 5 and 6, the minimum GPM block size, the maximum GPM block size, and the ratio between the block width and block height for the geometric partitioning mode can be signaled as a Log2 value. In the case of the minimum GPM block size, it can be signaled as the difference between the Log2 value and the minimum coding unit size supported in the current sequence.

[0161] log2_gpm_min_cu_size_minus2 can contain information about the minimum GPM block size. The minimum GPM block size can be derived from the value obtained by adding the Log2 value of the minimum coding unit size to log2_gpm_min_cu_size_minus2.

[0162] Meanwhile, the signaling methods and derivation methods for the maximum GPM block size and the ratio between block width and block height are described above with reference to Tables 3 and 4.

[0163] Tables 1 through 6 describe the use of SPS for signaling and parsing, but these are merely examples. Signaling and parsing can be performed in any one or two or more of the high-level syntaxes such as PPS, PH, and SH. In this case, values ​​can be stored or used by overwriting the values ​​of the high-level syntax ultimately signaled. The block size and / or the ratio between block width and block height to which the geometric partitioning pattern can be applied can vary depending on the characteristics of the image. Furthermore, when encoding / decoding threshold size information using high-level syntaxes, an optimal block size and an optimal width-to-height ratio can be applied to maximize compression efficiency according to the geometric partitioning pattern.

[0164] Reference Figure 4It can generate a prediction block for the current block based on the geometric partitioning pattern (S410).

[0165] When it is determined that a geometric partitioning pattern should be applied to the current block, the current block can be partitioned into multiple partitions. At least one of the partitions can have a geometric partitioning shape other than a rectangle.

[0166] When applying a geometric partitioning mode, various detailed modes can be applied based on it. Examples of detailed modes include an adaptive blending mode, a GPM intra-mode that applies intra-prediction to at least one of multiple partitions, a GPM MMVD (Merge with Motion Vector Difference) mode that applies MMVD to at least one of multiple partitions, a GPM TM (Template Matching) mode that applies TM to at least one of multiple partitions, and a GPM affine mode that performs affine inter-prediction for each of multiple partitions. At least one of the above detailed modes can also be used for inter-prediction based on the geometric partitioning mode.

[0167] The application of adaptive blending modes can be limited based on the size of the current block. Specifically, depending on the size of the current block and the blending region size to which the geometric partitioning mode is applied, the pixel region operated on by adaptive blending may be limited or exceeded, leading to reduced coding efficiency based on geometric partitioning. An adaptive blending mode can be one that selectively uses any of a plurality of blending region sizes to perform blending on partitions obtained through partitioning.

[0168] As an example, when the current block size is 4xN or Nx4, if the blending region size is greater than or equal to W, prediction and compensation based on geometry partitioning may not work, or prediction performance may be degraded. W can be any integer. For example, W can be 4. Therefore, when the current block size is 4xN or Nx4 or smaller, adaptive blending mode may not be applied. Adaptive blending mode can be applied for other sizes.

[0169] Alternatively, the number of blending regions applied in the adaptive blending mode can be reduced based on the size of the current block.

[0170] As an example, when the current block size is any of 4xN, Nx4, 8xN, or Nx8, the number of applicable blending region sizes can be reduced. For instance, the initial number of blending region sizes can be configured to five, such as {τ / 4, τ / 2, τ, 2τ, 4τ}. In this case, when the current block size is any of 4xN, Nx4, 8xN, or Nx8, the applicable blending region sizes can be reduced to three and configured as {τ / 2, τ, 2τ}.

[0171] Alternatively, an adaptive blending mode can be applied by fixing the number of blending regions to one based on the size of the current block. In this case, the applied blending region size can be τ.

[0172] However, the constraints of the adaptive blending processing mode (e.g., block size condition) and the number of blending region sizes are just examples and can be defined with other conditions and quantities.

[0173] In encoding and decoding devices that execute the adaptive hybrid processing mode according to this disclosure, the signaling and parsing methods for candidate indices may differ based on the size of the current block and the number of hybrid region sizes.

[0174] GPM intra-frame mode can be adaptively applied based on the size of the current block. GPM intra-frame mode can be a mode that uses intra-frame prediction mode to perform encoding / decoding on at least one of multiple partitions.

[0175] Intra-frame modes typically occur more frequently in smaller blocks than in larger blocks. Therefore, according to the method proposed in this disclosure, the application of GPM intra-frame modes can be restricted for larger blocks.

[0176] As an example, the application of GPM intra-frame mode can be restricted when the current block is a 128xN or Nx128 block. In this case, signaling and parsing of the syntax associated with GPM intra-frame mode can be skipped. However, the block size for restricting the application of GPM intra-frame mode is not limited to 128xN and Nx128. For example, the application of GPM intra-frame mode can be restricted when the product of the width and height of the current block is greater than M. Here, the value of M can be a predefined integer and can be selectively defined as any of values ​​such as 1024, 2048, 4096, and 8192.

[0177] The application of GPM MMVD mode or GPM TM mode can be limited based on the size of the current block. GPM MMVD mode can be a method of directly signaling additional refinement values ​​of motion vector information in an MMVD manner when performing prediction using geometric partitioning mode. GPM TM mode can be a method of refining motion vector information in the encoder / decoder in the same way using template matching (TM) when performing prediction using geometric partitioning mode.

[0178] In GPM MMVD mode, since the additional refinement values ​​of motion vector information are directly signaled using MMVD, the signaled bits may act as signaling overhead when the current block size is small. Therefore, the application of GPM MMVD mode can be limited for small blocks based on the current block size.

[0179] As an example, when the current block size is 4xN or Nx4, the application of GPM MMVD mode can be restricted, and its signaling and parsing processes can be skipped. However, the block size for restricting the application of GPM MMVD mode is not limited to 4xN and Nx4. The application of GPM MMVD mode can also be restricted when the product of the width and height of the current block is less than K. In this case, the value of K can be a predefined integer and can be selectively defined as one of values ​​such as 32, 64, and 128. When restricting the application of GPM MMVD mode, the application of GPM TM mode can also be restricted simultaneously.

[0180] The application of GPM affine patterns can be limited based on the size of the current block. A GPM affine pattern can be a pattern that performs predictions by applying an affine pattern to each of multiple partitions based on geometric partitioning. In this case, whether to apply the GPM affine pattern to the current block can be determined based on whether the size of the current block corresponds to a block size to which the affine pattern can be applied.

[0181] As an example, if the affine mode is restricted to be applicable when the width or height of the block is greater than or equal to 8, then the GPM affine mode can be applied when the width or height of the current block is greater than or equal to 8.

[0182] The application of regression-based blending patterns can be limited based on the size of the current block. A regression-based blending pattern is one that performs blending by generating a regression model for the block to which the geometric partitioning pattern is applied, rather than using existing predefined blending values.

[0183] As an example, the application of regression-based blending patterns can be restricted when the width and height of the current block are less than 8. Otherwise, regression-based blending patterns can be applied.

[0184] The above method allows for the limitation or simplification of the application of detailed modes based on geometric partitioning patterns, taking into account the current block size. In this case, the signaling and parsing processes for indicating the application of specific detailed modes that have a relatively small or no impact on compression efficiency improvements to the current block size can be reduced.

[0185] Reference Figure 4 The current block can be reconstructed based on the predicted block of the current block (S430).

[0186] The current block can be reconstructed based on the predicted block and the residual block. Transform coefficients can be derived based on residual information signaled from the bit stream, and residual blocks can be generated by performing at least one of dequantization or inverse transform on the transform coefficients.

[0187] Figure 5An illustrative configuration of a decoding device 300 performing a decoding method according to the present disclosure is shown.

[0188] Reference Figure 5 The decoding device 300 may include a GPM determination unit 500, a prediction block generation unit 510, and a reconstruction unit 520. The GPM determination unit 500 and the prediction block generation unit 510 may be set in... Figure 3 In the inter-frame prediction unit 332.

[0189] The GPM determination unit 500 can perform a determination process regarding whether to apply the geometric partitioning mode according to S400. The prediction block generation unit 510 can perform a prediction block generation process according to S410. The reconstruction unit 520 can perform a reconstruction process for the current block according to S420.

[0190] Figure 6 An encoding method performed by the encoding device 200 as an embodiment of the present disclosure is illustrated.

[0191] Reference Figure 6 This allows us to determine whether to apply a geometric partitioning mode (S400) to the current block. The method for determining whether to apply a geometric partitioning mode is as described above. Figure 4 As described.

[0192] Reference Figure 6 When it is determined that a geometric partitioning pattern should be applied to the current block, a prediction block for the current block can be generated based on the geometric partitioning pattern (S610). In the process of generating a prediction block based on the geometric partitioning pattern, at least one of the detailed patterns based on the geometric partitioning pattern can be additionally applied, or its application can be restricted. This is as described above. Figure 4 As described.

[0193] Reference Figure 6 The transform coefficients of the current block can be derived based on the residual block of the current block (S620). The residual block of the current block can be generated based on the prediction block generated in S610. The transform coefficients can be derived by performing at least one of transformation or quantization on the residual block.

[0194] Reference Figure 6 A bit stream can be generated by encoding the residual information of the transform coefficients of the current block (S630).

[0195] Figure 7 An illustrative configuration of an encoding device 200 performing an encoding method according to the present disclosure is shown.

[0196] Reference Figure 7 The encoding device 200 may include a GPM determination unit 700, a prediction block generation unit 710, a transform coefficient derivation unit 720, and a residual information encoding unit 730.

[0197] The GPM determination unit 700 and the prediction block generation unit 710 can be set in Figure 2 In the inter-frame prediction unit 221, the transform coefficient derivation unit 720 can be set in... Figure 2 The residual processing unit 230 is located within the residual information encoding unit 730. Figure 2 In the entropy coding unit 240.

[0198] The GPM determination unit 700 can perform the determination process regarding whether to apply the geometric partitioning mode according to S600. The prediction block generation unit 710 can perform the prediction block generation process according to S610. The transform coefficient derivation unit 720 can perform the transform coefficient derivation process according to S620. The residual information encoding unit 730 can perform the residual information encoding process according to S630.

[0199] In the above embodiments, the method is described based on a flowchart as a series of steps or blocks. However, the corresponding embodiments are not limited to this order of steps. Some steps may occur simultaneously with other steps or in a different order, as described above. In addition, those skilled in the art will understand that the steps shown in the flowchart are not exclusive. Other steps may be included, or one or more steps in the flowchart may be deleted, without affecting the scope of the embodiments of this disclosure.

[0200] The methods described above according to embodiments of the present disclosure can be implemented in software, and the encoding and / or decoding devices according to the present disclosure can be included in an apparatus for performing image processing, such as a TV, computer, smartphone, set-top box, display device, etc.

[0201] In this disclosure, when the implementation is implemented as software, the above-described method can be implemented as a module (process, function, etc.) performing the above-described functions. The module can be stored in memory and can be executed by a processor. The memory can be internal or external to the processor and can be connected to the processor by various well-known means. The processor may include an application-specific integrated circuit (ASIC), another chipset, logic circuitry, and / or data processing devices. The memory may include read-only memory (ROM), random access memory (RAM), flash memory, memory cards, storage media, and / or other storage devices. In other words, the implementations described herein can be executed by implementation on a processor, microprocessor, controller, or chip. For example, the functional units shown in the various figures can be executed by implementation on a computer, processor, microprocessor, controller, or chip. In this case, information for implementation (e.g., information about instructions) or algorithms can be stored in a digital storage medium.

[0202] Furthermore, decoding and encoding devices employing embodiments of this disclosure can be included in multimedia broadcasting transmitting and receiving devices, mobile communication terminals, home theater video devices, digital cinema video devices, surveillance cameras, video conferencing devices, real-time communication devices similar to video communication, mobile streaming devices, storage media, cameras, devices for providing video-on-demand (VoD) services, OTT (over-the-top) video devices, devices for providing internet streaming services, three-dimensional (3D) video devices, virtual reality (VR) devices, augmented reality (AR) devices, video telephony devices, transportation terminals (e.g., vehicle (including autonomous vehicle) terminals, aircraft terminals, ship terminals, etc.), and medical video devices, and can be used to process video signals or data signals. For example, OTT (over-the-top) video devices can include game consoles, Blu-ray players, internet-connected televisions, home theater systems, smartphones, tablet PCs, digital video recorders (DVRs), etc.

[0203] Furthermore, the processing methods applying the embodiments of this disclosure can be generated in the form of a computer-executable program and can be stored in a computer-readable recording medium. Multimedia data with data structures according to the embodiments of this disclosure can also be stored in a computer-readable recording medium. Computer-readable recording media include all types of storage devices and distributed storage devices that store computer-readable data. Computer-readable recording media can include, for example, Blu-ray discs (BD), Universal Serial Bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disks, and optical media storage devices. Additionally, computer-readable recording media include media implemented in the form of carrier waves (e.g., transmission via the Internet). Furthermore, bitstreams generated by encoding methods can be stored in computer-readable recording media or transmitted via wired / wireless communication networks.

[0204] Furthermore, the embodiments of this disclosure can be implemented by a computer program product using program code, and the program code can be executed on a computer using the embodiments of this disclosure. The program code can be stored on a computer-readable medium.

[0205] Figure 8 Examples of content streaming systems to which embodiments of the present disclosure can be applied are shown.

[0206] Reference Figure 8 A content streaming system that applies embodiments of the present disclosure may generally include an encoding server, a streaming server, a network server, a media storage device, a user device, and a multimedia input device.

[0207] An encoding server generates a bitstream by compressing content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data and then sends it to a streaming server. As another example, when multimedia input devices such as smartphones, cameras, and camcorders directly generate bitstreams, the encoding server can be omitted.

[0208] A bitstream can be generated by an encoding method or bitstream generation method that applies the embodiments of this disclosure, and the streaming server can temporarily store the bitstream during the sending or receiving of the bitstream.

[0209] A streaming server sends multimedia data to a user device via a web server based on a user request, and the web server acts as a medium to inform the user what services are available. When a user requests a desired service from the web server, the web server transmits the request to the streaming server, and the streaming server sends the multimedia data to the user. In this scenario, the content streaming system may include a separate control server, which controls the commands / responses between the various devices in the content streaming system.

[0210] A streaming server can receive content from media storage and / or encoding servers. For example, when receiving content from an encoding server, the content can be received in real time. In this case, to provide a smooth streaming service, the streaming server can store the bitstream for a specific time period.

[0211] Examples of user devices may include mobile phones, smartphones, laptops, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigators, touchscreen PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, head-mounted displays (HMDs)), digital TVs, desktop computers, digital signage, etc.

[0212] In a content streaming system, each server can operate as a distributed server, and in this case, the data received from each server can be distributed and processed.

[0213] The claims set forth herein can be combined in various ways. For example, the technical features of the method claims of this disclosure can be combined and implemented as an apparatus, and the technical features of the apparatus claims of this disclosure can be combined and implemented as a method. Furthermore, the technical features of the method claims and the apparatus claims of this disclosure can be combined and implemented as an apparatus, and the technical features of the method claims and the apparatus claims of this disclosure can be combined and implemented as a method.

Claims

1. A method comprising the following steps: Determine whether to apply a geometric partitioning mode to the current block based on the current block size and a predetermined threshold size. When it is determined that the geometric partitioning pattern should be applied to the current block, a prediction block for the current block is generated based on the geometric partitioning pattern. as well as The current block is reconstructed based on the predicted block.

2. The method according to claim 1, in, The threshold size information includes at least one of the following: the minimum GPM block size with respect to the minimum size of the coding unit that the geometric partitioning pattern can be applied to, the maximum GPM block size with respect to the maximum size of the coding unit that the geometric partitioning pattern can be applied to, or the ratio between the block width and the block height that the geometric partitioning pattern can be applied to.

3. The method according to claim 2, in, The ratio between the block width and the block height is applied differently depending on the size of the current block.

4. The method according to claim 2, in, The bitstream signals at least one of the following: information about the minimum GPM block size, information about the maximum GPM block size, or information about the ratio between the block width and the block height.

5. The method according to claim 4, in, The minimum GPM block size is derived based on a value obtained by adding a first value to a value containing information about the minimum GPM block size, and The maximum GPM block size is derived based on a value obtained by adding a second value to a value containing information about the maximum GPM block size.

6. The method according to claim 4, in, The minimum GPM block size is derived based on a value obtained by adding a first value to a value containing information about the minimum GPM block size, and The maximum GPM block size is derived based on a value obtained by adding the first value to the value of information about the minimum GPM block size and the value of information about the maximum GPM block size.

7. The method according to claim 4, in, The ratio between the block width and the block height is derived based on a value obtained by adding a third value to the value of the information regarding the ratio between the block width and the block height.

8. The method according to claim 1, in, When generating the prediction block of the current block, the number of available mixing region sizes for the current block is determined based on the size of the current block.

9. The method according to claim 1, in, When generating the predicted block of the current block, the application of at least one of a plurality of detailed modes based on the geometric partitioning pattern is restricted according to the size of the current block.

10. The method according to claim 9, in, The multiple detailed modes include at least one of the following: an adaptive blending mode that performs blending by selectively using any of a multiple blending region sizes; a GPM intra-frame mode that applies intra-frame prediction to at least one of a multiple partitions in the current block; a GPMMMVD mode that applies merged MMVD using motion vector differences to at least one of the multiple partitions; a GPM TM mode that applies template matching TM to at least one of the multiple partitions; or a GPM affine mode that performs affine inter-frame prediction for each of the multiple partitions.

11. A method comprising the following steps: Determine whether to apply a geometric partitioning mode to the current block based on the current block size and a predetermined threshold size. When it is determined that the geometric partitioning pattern should be applied to the current block, a prediction block for the current block is generated based on the geometric partitioning pattern. The residual block of the current block is generated based on the predicted block; The transformation coefficients of the current block are derived based on the residual block; as well as The residual information of the transformation coefficients is encoded.

12. A computer-readable storage medium storing a bit stream generated by the method according to claim 11.

13. A method comprising the following steps: Obtain a bitstream of image information, wherein the bitstream is generated based on the following steps: determining whether to apply a geometric partitioning pattern to the current block based on the current block size and a predetermined threshold size; if it is determined that the geometric partitioning pattern should be applied to the current block, generating a prediction block of the current block based on the geometric partitioning pattern; generating a residual block of the current block based on the prediction block; deriving the transform coefficients of the current block based on the residual block; and encoding the residual information of the transform coefficients; and Send data including the bit stream.

Images