Encoding method, encoding device, decoding method, and decoding device for an image

Abstract

Provided are an image decoding method, an image decoding apparatus, an image encoding method, and an image encoding apparatus, the image decoding method including the steps of: acquiring a reference block in a reference image; determining an intra mode for the reference block based on at least one of a plurality of neighboring samples of the reference block and a plurality of internal samples of the reference block; determining a transform kernel for a current block from one or more transform kernels by using the intra mode for the reference block; and performing inverse transform on the current block by using the transform kernel.

Description

Technical Field

[0001] This disclosure relates to the field of image encoding and decoding. More specifically, this disclosure relates to methods and apparatus for encoding and decoding samples in an image. Background Technology

[0002] For image encoding and decoding, the image is divided into blocks, and each block undergoes predictive encoding and predictive decoding through inter-frame prediction or intra-frame prediction.

[0003] Inter-frame prediction is a technique for compressing images by removing temporal redundancy between them. In inter-frame prediction, blocks in the current image are predicted using a reference image. A reference block most similar to the current block is searched within a specific search range in the reference image. The current block is predicted based on the reference block, and the predicted block is subtracted from the current block to generate a residual block.

[0004] Intra-frame prediction is a technique for compressing images by removing spatial redundancy. It generates prediction blocks based on the neighboring pixels of the current block according to a prediction pattern. Residual blocks are then generated by subtracting the prediction blocks from the current block.

[0005] The residual blocks generated through inter-frame or intra-frame prediction undergo transform and quantization before being sent to the decoder. The decoder performs inverse quantization and inverse transform on the residual blocks and reconstructs the current block by combining the predicted blocks with the residual blocks. The decoder can then filter the reconstructed current block to remove artifacts. Summary of the Invention

[0006] Solution to the problem According to embodiments of this disclosure, an image decoding method is provided. The image decoding method may include: obtaining a reference block in a reference image. The image decoding method may include: determining an intra-frame mode for the reference block based on at least one of a plurality of neighboring samples and a plurality of internal samples of the reference block. The image decoding method may include: determining a transform kernel for a current block from one or more transform kernels using the intra-frame mode for the reference block. The image decoding method may include: performing an inverse transform on the current block using the transform kernel.

[0007] According to embodiments of this disclosure, an image decoding apparatus is provided. The image decoding apparatus may include a memory storing one or more instructions and at least one processor. The at least one processor may execute one or more instructions to obtain a reference block in a reference image. The at least one processor may execute one or more instructions to determine an intra-frame mode for the reference block based on at least one of a plurality of neighboring samples and a plurality of internal samples of the reference block. The at least one processor may execute one or more instructions to determine a transform kernel for a current block from one or more transform kernels using the intra-frame mode for the reference block. The at least one processor may execute one or more instructions to perform an inverse transform on the current block using the transform kernel.

[0008] According to embodiments of this disclosure, an image encoding method is provided. The image encoding method may include: obtaining a reference block in a reference image. The image encoding method may include: determining an intra-frame mode for the reference block based on at least one of a plurality of neighboring samples and a plurality of internal samples of the reference block. The image encoding method may include: determining a transform kernel for a current block from one or more transform kernels by using the intra-frame mode for the reference block. The image encoding method may include: performing a transform on the current block using the transform kernel.

[0009] According to embodiments of this disclosure, an image encoding apparatus is provided. The image encoding apparatus may include a memory storing one or more instructions and at least one processor. The at least one processor may execute one or more instructions to obtain a reference block in a reference image. The at least one processor may execute one or more instructions to determine an intra-frame mode for the reference block based on at least one of a plurality of neighboring samples and a plurality of internal samples of the reference block. The at least one processor may execute one or more instructions to determine a transform kernel for a current block using the intra-frame mode for the reference block from one or more transform kernels. The at least one processor may execute one or more instructions to perform a transform on the current block using the transform kernel.

[0010] According to embodiments of the present disclosure, a computer-readable storage medium is provided for storing a bitstream encoded by an image encoding method. Attached Figure Description

[0011] Figure 1 This is a block diagram of an image decoding apparatus according to an embodiment of the present disclosure.

[0012] Figure 2 This is a block diagram of an image encoding apparatus according to an embodiment of the present disclosure.

[0013] Figure 3 The present disclosure illustrates a process for determining at least one coding unit by dividing the current coding unit, according to an embodiment of the present disclosure.

[0014] Figure 4 A process for determining at least one coding unit by dividing coding units of a non-square shape, according to an embodiment of the present disclosure, is illustrated.

[0015] Figure 5 This illustrates a process for dividing coding units based on at least one of block-form information and partition-form pattern information, according to embodiments of the present disclosure.

[0016] Figure 6 A method for determining a particular coding unit from an odd number of coding units according to embodiments of the present disclosure is illustrated.

[0017] Figure 7 The following describes the order in which multiple coding units are processed when multiple coding units are determined by dividing the current coding unit, according to an embodiment of the present disclosure.

[0018] Figure 8 The present disclosure illustrates a process for determining that a current coding unit is divided into an odd number of coding units when it is impossible to process coding units in a specific order.

[0019] Figure 9 The present disclosure illustrates a process for determining at least one coding unit by dividing a first coding unit, according to an embodiment of the present disclosure.

[0020] Figure 10 The embodiments of the present disclosure show that when the second coding unit meets certain conditions, the form in which the second coding unit, which is a non-square shape determined by dividing the first coding unit, can be divided is limited.

[0021] Figure 11 The process of dividing the coding units into square shapes is illustrated according to an embodiment of the present disclosure when the partitioning pattern information does not indicate that the coding units are divided into four square shapes.

[0022] Figure 12 The processing order among multiple coding units according to embodiments of the present disclosure is shown to vary depending on the process used to divide the coding units.

[0023] Figure 13 The present disclosure illustrates a process for determining the depth of a coding unit as the shape and size of the coding unit varies when multiple coding units are determined by recursively dividing coding units.

[0024] Figure 14 The diagram illustrates a depth that can be determined based on the form and size of the coding unit and a partial index (PID) used to classify the coding unit, according to an embodiment of the present disclosure.

[0025] Figure 15 The illustration shows how multiple coding units are determined based on multiple specific data units included in the image, according to an embodiment of the present disclosure.

[0026] Figure 16 The present disclosure illustrates encoding units that can be determined for each frame when the encoding units can be divided into different combinations for each frame.

[0027] Figure 17 Various forms of coding units, which can be determined based on partition-form pattern information represented in binary code, are shown according to embodiments of the present disclosure.

[0028] Figure 18 Other forms of coding units that can be determined based on partition-form pattern information represented in binary code, according to embodiments of the present disclosure, are shown.

[0029] Figure 19 This is a block diagram of an image encoding and decoding system that performs loop filtering according to embodiments of the present disclosure.

[0030] Figure 20 This is a block diagram illustrating the configuration of an image decoding apparatus according to an embodiment of the present disclosure.

[0031] Figure 21 This is a flowchart of an image decoding method according to an embodiment of the present disclosure.

[0032] Figure 22 This is a flowchart of an image decoding method according to an embodiment of the present disclosure.

[0033] Figure 23 The process of determining the intra-frame mode for a reference block by using neighboring blocks of the reference block, according to an embodiment of the present disclosure, is illustrated.

[0034] Figure 24 The process of determining the intra-frame mode for the reference block by using neighboring samples of the reference block according to an embodiment of the present disclosure is illustrated.

[0035] Figure 25 This is a flowchart of an image decoding method according to an embodiment of the present disclosure.

[0036] Figure 26 The process of determining the intra-frame mode for the reference block by using a reference block is illustrated according to an embodiment of the present disclosure.

[0037] Figure 27 The process of determining the intra-frame mode for the reference block by using a reference block is illustrated according to an embodiment of the present disclosure.

[0038] Figure 28The process of determining the intra-frame mode for the reference block by using a reference block is illustrated according to an embodiment of the present disclosure.

[0039] Figure 29 This is a flowchart of an image decoding method according to an embodiment of the present disclosure.

[0040] Figure 30 The process of determining the intra-frame mode for the reference block by using neighboring samples of the reference block according to an embodiment of the present disclosure is illustrated.

[0041] Figure 31 This is a flowchart of an image decoding method according to an embodiment of the present disclosure.

[0042] Figure 32 The process of determining the intra-frame mode for a reference block by using neighboring samples of the current block, according to an embodiment of the present disclosure, is illustrated.

[0043] Figure 33 This is a flowchart of an image decoding method according to an embodiment of the present disclosure.

[0044] Figure 34 The process of determining the intra-frame mode for the reference block by using a reference block is illustrated according to an embodiment of the present disclosure.

[0045] Figure 35 It is a diagram used to describe the form of a template according to embodiments of the present disclosure.

[0046] Figure 36 This is a block diagram illustrating the configuration of an image encoding apparatus according to an embodiment of the present disclosure.

[0047] Figure 37 This is a flowchart of an image encoding method according to an embodiment of the present disclosure.

[0048] Figure 38 This is a flowchart of an image encoding method according to an embodiment of the present disclosure. Detailed Implementation

[0049] Throughout this disclosure, the expression "at least one of a, b or c" means only a, only b, only c, both a and b, both a and c, both b and c, all of a, b and c, or variations thereof.

[0050] In this disclosure, the expression "a, b and / or c" may be replaced by "at least one of a, b or c". In other words, the expression "a, b and / or c" may indicate only a, only b, only c, both a and b, both a and c, both b and c, all of a, b and c, or variations thereof.

[0051] Various modifications can be made to the embodiments of this disclosure, which will be described more fully below with reference to the accompanying drawings. This disclosure is not limited to the specific embodiments, but may include all modifications, equivalents, and substitutions that fall within the technical scope and concept of this disclosure.

[0052] In consideration of the principles of this disclosure, the terminology used herein has been chosen to be commonly used and widely accepted; however, this may depend on the intent of a person skilled in the art, judicial precedent, the emergence of new technologies, etc. Some terms used herein were chosen based on the applicant's judgment, in which case the meaning of the term will be understood through the relevant description. Therefore, terms should be defined based on their meaning and description throughout this disclosure.

[0053] In this disclosure, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well. Some related well-known techniques that may obscure this disclosure will not be described. Ordinal numbers (e.g., first, second, etc.) used herein are for the purpose of distinguishing components from one another.

[0054] When the terms “connection” or “coupling” are used, it means that a component can be directly connected to or coupled to another component. However, unless otherwise defined, it should also be understood that the component can be indirectly connected to or coupled to another component via a new component.

[0055] In this disclosure, the terms "comprising (or including)" or "including (or containing)" are inclusive or open-ended and do not exclude additional, unlisted elements or method steps. In this disclosure, a component referred to as "~unit," "~module," etc., can be a combination of two or more components, or can be divided into two or more sub-components. Furthermore, each component can perform its main function and further perform some or all of the functions provided by another component. In this way, a portion of the main function served by each component can be dedicated to and performed by another component.

[0056] In this disclosure, the expression “configured as” as used herein may be used interchangeably with “suitable for,” “capable of,” “designed for,” “suitable for,” “manufactured as,” or “capable,” depending on the context. The expression “configured as” may not necessarily mean “specifically designed for” in terms of hardware. Alternatively, in some cases, the expression “system configured to do something” may refer to “an entity capable of cooperating with another device or component to do something.” For example, “processor configured to perform functions A, B, and C” may include a dedicated processor (e.g., an embedded processor for performing functions A, B, and C) or a general-purpose processor (e.g., a central processing unit (CPU) or application processor that can perform functions A, B, and C by executing one or more software programs stored in memory).

[0057] At least one processor according to embodiments of the present disclosure may include various processing circuits and / or multiple processors. For example, the term "processor" as used in the present disclosure including the appended claims may include various processing circuits comprising at least one processor, and one or more of the at least one processor are configured to perform the various functions described herein individually and / or collectively in a distributed manner. As used in the specification, the terms "processor," "at least one processor," and "one or more processors," when described as being configured to perform various functions, may include, for example, but not limited to, a single processor performing the functions. In some cases, among the referenced functions, some functions are performed by a processor while others are performed by other processors, or a single processor may perform all the referenced functions. Furthermore, at least one processor may include, for example, a combination of processors performing the various enumerated / disclosed functions in a distributed manner. At least one processor may execute program instructions to implement or perform various functions.

[0058] In this disclosure, an image may include a picture, a still image, a frame, a moving image composed of multiple consecutive still images, or a video.

[0059] In this disclosure, a sample point can refer to data assigned to a sampling location in an image and may include data to be processed. For example, a sample point may comprise pixels in a frame in the spatial domain. A block can refer to a unit that includes multiple sample points.

[0060] Now refer to Figures 1 to 19 This disclosure describes an image encoding method and apparatus based on tree-structured encoding and transformation units, as well as an image decoding method and apparatus.

[0061] Figure 1 A block diagram of an image decoding apparatus 100 according to an embodiment of the present disclosure is shown.

[0062] The image decoding apparatus 100 may include a bitstream acquirer 110 and a decoder 120. The bitstream acquirer 110 and the decoder 120 may include at least one processor. The bitstream acquirer 110 and the decoder 120 may also include memory for storing instructions to be executed by the at least one processor.

[0063] Bitstream acquirer 110 can receive bitstreams. The bitstream includes information about an image encoded by image encoding device 200, as described later. The bitstream can be transmitted from image encoding device 200. Image encoding device 200 and image decoding device 100 can be connected to each other via cable or wirelessly, and bitstream acquirer 110 can receive bitstreams via cable or wirelessly. Bitstream acquirer 110 can receive bitstreams from storage media such as optical media or hard disks. Decoder 120 can reconstruct the image based on information obtained from the received bitstream. Decoder 120 can obtain syntax elements for reconstructing the image from the bitstream. Decoder 120 can reconstruct the image based on the syntax elements.

[0064] In the detailed operation of the image decoding device 100, the bitstream receiver 110 can receive bitstreams.

[0065] The image decoding apparatus 100 can perform the operation of obtaining a binary bit string corresponding to the partitioning pattern of the coding unit from the bit stream. The image decoding apparatus 100 can perform the operation of determining partitioning rules for the coding unit. Furthermore, the image decoding apparatus 100 can perform the operation of dividing the coding unit into multiple coding units based on at least one of the binary bit string corresponding to the partitioning pattern and the partitioning rules. The image decoding apparatus 100 can determine a first permissible size range of the coding unit based on the ratio between the width and height of the coding unit to determine the partitioning rules. The image decoding apparatus 100 can determine a second permissible size range of the coding unit based on the partitioning pattern of the coding unit to determine the partitioning rules.

[0066] The division of coding units will now be described in detail according to embodiments of this disclosure.

[0067] First, the frame can be divided into one or more stripes or one or more parallel blocks. A stripe or parallel block can be a sequence of one or more maximum coding units (code tree units (CTUs)). In an embodiment, a stripe may include one or more parallel blocks and one or more maximum coding units. Stripes comprising one or more parallel blocks can be identified within the frame.

[0068] In contrast to the Maximum Coding Unit (CTU), there exists a Maximum Coding Block (CTB). A Maximum Coding Block (CTB) is an N×N block comprising N×N samples, where N is an integer. Each color component can be divided into one or more Maximum Coding Blocks.

[0069] When the image has three sample arrays (sample arrays for the Y, Cr, and Cb components respectively), the maximum coding unit (CTU) is a unit that includes a maximum coding block for the luma sample, two maximum coding units for the corresponding chroma sample, and a syntax structure for encoding the luma and chroma samples. When the image is monochrome, the maximum coding unit is a unit that includes a maximum coding block for the monochrome sample and a syntax structure for encoding the monochrome sample. When the image is encoded to a color plane separated according to each color component, the maximum coding unit is a unit that includes a syntax structure for encoding the samples of the image and the picture.

[0070] A maximum coding block can be divided into coding blocks of size M×N, each coding block containing M×N samples, where M and N are integers.

[0071] When the image has sample arrays for each of the Y, Cr, and Cb components, the encoding unit (CU) is a unit comprising an encoding block for luma samples and two encoding blocks for corresponding chroma samples, as well as a syntax structure for encoding the luma and chroma samples. When the image is monochrome, the encoding unit comprises an encoding block (CB) for monochrome samples and a syntax structure for encoding the monochrome samples. When the image is encoded onto a color plane separated according to each color component, the encoding unit comprises a syntax structure for encoding the samples of the image and the picture.

[0072] As stated above, the maximum coding block and the maximum coding unit are different concepts, and the coding block and the coding unit are also different concepts. In other words, the (maximum) coding unit refers to the data structure that includes the (maximum) coding block, which includes the associated samples and the corresponding syntax structure. However, as will be understood by those skilled in the art, the (maximum) coding unit or (maximum) coding block refers to a block of a specific size that includes a specific number of samples, and unless otherwise stated, the maximum coding block and the maximum coding unit, or the coding block and the coding unit, will now be used interchangeably.

[0073] An image can be divided into maximum coding units (or coding tree units (CTUs)). The size of a maximum coding unit can be determined based on information obtained from the bitstream. A maximum coding unit can have a square shape of uniform size. It is not limited to this.

[0074] For example, information about the maximum size of a luminance-coded block can be obtained from the bitstream. For example, the maximum size of a luminance-coded block, indicated by the information about the maximum size of the luminance-coded block, can be one of 4×4, 8×8, 16×16, 32×32, 64×64, 128×128, and 256×256.

[0075] For example, information about the maximum size of the luma coding block that can be divided into two and the difference between the luma block sizes can be obtained from the bitstream. The information about the difference in luma block sizes indicates the difference in size between the maximum luma coding unit and the maximum luma coding block that can be divided into two. Therefore, the size of the maximum luma coding unit can be determined by combining the information about the maximum size of the luma coding block that can be divided into two and the information about the difference in luma block sizes obtained from the bitstream. The size of the maximum chroma coding unit can also be determined by using the size of the maximum luma coding unit. For example, when the Y:Cb:Cr ratio is 4:2:0 according to the color format, the size of the chroma block can be half the size of the luma block, and similarly, the size of the maximum chroma coding unit can be half the size of the maximum luma coding unit.

[0076] In an embodiment, when information about the maximum size of a binary-divisible luminance coding block is obtained from the bitstream, the maximum size of the binary-divisible luminance coding block can be variably determined. Optionally, the maximum size of a ternary-divisible luminance coding block can be fixed. For example, the maximum size of a ternary-divisible luminance coding block in image I can be 32×32, and the maximum size of a ternary-divisible luminance coding block in image P or image B can be 64×64.

[0077] Furthermore, the largest coding unit can be hierarchically divided into coding units based on the partitioning pattern information obtained from the bitstream. As partitioning pattern information, at least one of the following can be obtained from the bitstream: information indicating whether quaternary partitioning is possible, information indicating whether multi-partitioning is possible, partitioning direction information, and partitioning type information.

[0078] For example, information indicating whether a code unit can be quaternized can indicate whether the current coding unit can be quaternized.

[0079] When the current coding unit is not divisible by four elements, the information indicating whether the current coding unit is divisible by three elements can indicate whether the current coding unit will no longer be divisible or will be divisible by two elements or three elements.

[0080] When the current coding unit is divisible by a binary or a ternary partition, the partitioning direction information indicates that the current coding unit is partitioned according to either the horizontal or vertical direction.

[0081] When the current coding unit is divided horizontally or vertically, the partition type information indicates whether the current coding unit is divided by a binary partition or a ternary partition.

[0082] The partitioning mode of the current coding unit can be determined based on the partitioning direction information and partitioning type information. For the case where the current coding unit can be partitioned horizontally in a binary manner, the partitioning mode can be determined as horizontally binary (SPLIT_BT_HOR); for the case where it can be partitioned horizontally in a ternary manner, the partitioning mode can be determined as horizontally ternary (SPLIT_TT_HOR); for the case where it can be partitioned vertically in a binary manner, the partitioning mode can be determined as vertically binary (SPLIT_BT_VER); and for the case where it can be partitioned vertically in a ternary manner, the partitioning mode can be determined as vertically ternary (SPLIT_TT_VER).

[0083] Image decoding device 100 can obtain partitioning pattern information from binary bit strings in a bit stream. The form of the bit stream received by image decoding device 100 may include fixed-length binary codes, unary codes, truncated unary codes, predetermined binary codes, etc. The binary bit string represents information as a sequence of binary bit words. The binary bit string can consist of at least one bit. Image decoding device 100 can obtain partitioning pattern information corresponding to the binary bit string based on partitioning rules. Image decoding device 100 can determine whether to perform quaternion partitioning of coding units, or determine the partitioning direction and partitioning type, based on a binary bit string.

[0084] A coding unit can be equal to or smaller than the maximum coding unit. For example, the maximum coding unit is also one of the coding units with the largest size. When the partitioning pattern information for the maximum coding unit indicates no partitioning, the coding units determined from the maximum coding unit have the same size as the maximum coding unit. When the partitioning pattern information for the maximum coding unit indicates partitioning, the maximum coding unit can be partitioned into coding units. Furthermore, when the partitioning pattern information for the coding units indicates partitioning, the coding units can be partitioned into coding units of smaller size. However, image partitioning is not limited to this, and the maximum coding unit and coding units may not be distinguished. (Refer to...) Figures 3 to 16 A more detailed description of the partitioning of the coding unit.

[0085] Furthermore, one or more prediction blocks for prediction can be determined from the coding unit. The prediction blocks may be equal to or smaller than the coding unit. Furthermore, one or more transform blocks for transform can be determined from the coding unit. The transform blocks may be equal to or smaller than the coding unit.

[0086] The shape and size of the transform block can be independent of the shape and size of the prediction block.

[0087] In another embodiment, the encoding unit is a prediction block that can be predicted by using the encoding unit. Furthermore, the encoding unit is a transformation block that can be transformed by using the encoding unit.

[0088] Reference Figures 3 to 16 The partitioning of coding units is described in more detail. In this disclosure, the current block and neighboring blocks may represent one of a maximum coding unit, a coding unit, a prediction block, and a transform block. Furthermore, the current block and the current coding unit are the blocks currently being decoded or encoded, or the blocks currently being partitioned. Neighboring blocks may be blocks reconstructed prior to the current block. Neighboring blocks may be spatially or temporally adjacent to the current block. Neighboring blocks may be located to the lower left, left, top, upper right, right, and lower right of the current block.

[0089] The foregoing embodiments described operations related to the image decoding method performed by the image decoding apparatus 100. The operation of the image decoding apparatus 200, which performs an image encoding method corresponding to the reverse process of the image decoding method, will be described through embodiments of this disclosure.

[0090] Figure 2 This is a block diagram of an image encoding apparatus according to embodiments of the present disclosure, capable of encoding an image based on at least one of block-form information and partition-form pattern information.

[0091] Image encoding apparatus 200 may include encoder 220 and bitstream generator 210. Encoder 220 may receive and encode an input image. Encoder 220 may obtain at least one syntax element by encoding the input image. Syntax element may include at least one of skip flag, prediction mode, motion vector difference, motion vector prediction method (or index), transform quantization coefficient, coding block mode, coding block flag, intra-prediction mode, direct flag, merge flag, incremental QP, reference index, prediction direction, and transform index. Encoder 220 may determine a context model based on block-form information including at least one of the shape, orientation, aspect ratio, or size of coding units.

[0092] Bitstream generator 210 can generate a bitstream based on an encoded input image. For example, bitstream generator 210 can generate a bitstream by entropy encoding of syntax elements based on a context model. Furthermore, image encoding device 200 can send the bitstream to image decoding device 100.

[0093] In embodiments of this disclosure, the encoder 220 of the image encoding apparatus 200 can determine the form of the encoding unit. For example, the encoding unit may have a square or non-square form, and information indicating the form may be included in the block form information.

[0094] In embodiments of this disclosure, encoder 220 may determine the form into which the encoding unit will be divided. Encoder 220 may determine the form of at least one encoding unit included in the encoding unit, and bitstream generator 210 may generate a bitstream including partition form pattern information, wherein the partition form pattern information includes information about the form of the encoding unit.

[0095] In embodiments of this disclosure, encoder 220 can determine whether a coding unit will be divided. When encoder 220 determines that a coding unit comprises only one coding unit or that a coding unit will not be divided, bitstream generator 210 can generate a bitstream including partitioned pattern information indicating that the coding unit will not be divided. Furthermore, encoder 220 can divide a coding unit into multiple coding units, and bitstream generator 210 can generate a bitstream including partitioned pattern information indicating that a coding unit will be divided into multiple coding units.

[0096] In embodiments of this disclosure, information indicating how many coding units a coding unit will be divided into or along which direction the coding units will be divided may be included in the partitioning pattern information. For example, the partitioning pattern information may indicate whether partitioning is performed or not in at least one of the vertical or horizontal directions.

[0097] Image coding apparatus 200 determines information about the partitioning pattern of the coding units based on the partitioning pattern of the coding units. Image coding apparatus 200 determines a context model based on at least one of the shape, orientation, aspect ratio, or size of the coding units. Image coding apparatus 200 also generates a bitstream based on the context model of the partitioning pattern used to divide the coding units.

[0098] To determine the context model, the image encoding apparatus 200 may obtain an array for matching at least one of the shape, orientation, aspect ratio, or size of the coding unit with the index of the context model. The image encoding apparatus 200 may obtain from the array the index of the context model based on at least one of the shape, orientation, aspect ratio, or size of the coding unit. The image encoding apparatus 200 may determine the context model based on the index of the context model.

[0099] To determine the context model, the image coding apparatus 200 may further determine the context model based on block shape information including at least one of the shape, orientation, aspect ratio, or size of neighboring coding units adjacent to the coding unit. Furthermore, neighboring coding units may include at least one coding unit located to the lower left, left, upper left, upper, upper right, right, or lower right of the coding unit.

[0100] The image coding apparatus 200 can compare the width of the upper neighboring coding unit with the width of the coding unit to determine the context model. The image coding apparatus 200 can also compare the heights of the left and right neighboring coding units with the height of the coding unit. The image coding apparatus 200 can determine the context model based on the comparison results.

[0101] The operation of the image encoding device 200 includes combining Figures 3 to 19The operation of the video decoding device 100 described herein is similar, therefore its detailed description will be omitted.

[0102] Figure 3 The process of determining at least one coding unit by dividing the current coding unit is illustrated in an image decoding apparatus 100 according to an embodiment of the present disclosure.

[0103] The block format may include 4N×4N, 4N×2N, 2N×4N, 4N×N, N×4N, 32N×N, N×32N, 16N×N, N×16N, 8N×N, or N×8N. N can be a positive integer. The block format information indicates at least one of the shape, orientation, aspect ratio, or size of the coding unit.

[0104] The shape of the encoding unit can be square or non-square. When the width and height of the encoding unit have the same length (i.e., the block shape of the encoding unit corresponds to 4N×4N), the image decoding device 100 can determine the block shape information of the encoding unit as square. The image decoding device 100 can also determine the shape of the encoding unit as non-square.

[0105] When the width and height of the coding unit differ in length (i.e., the block shape of the coding unit corresponds to 4N×2N, 2N×4N, 4N×N, N×4N, 32N×N, N×32N, 16N×N, N×16N, 8N×N, or N×8N), the image decoding device 100 can determine that the block shape information of the coding unit is not square. When the shape of the coding unit is not square, the image decoding device 100 can determine the aspect ratio in the block shape information of the coding unit to be at least one of 1:2, 2:1, 1:4, 4:1, 1:8, 8:1, 1:16, 16:1, 1:32, and 32:1. Furthermore, based on the width and height of the coding unit, the image decoding device 100 can determine whether the coding unit is horizontal or vertical. Furthermore, based on at least one of the width, height, or area of ​​the coding unit, the image decoding device 100 can determine the size of the coding unit.

[0106] In embodiments of this disclosure, the image decoding apparatus 100 can determine the form of the coding unit based on block format information and determine which form the coding unit will be divided into based on partition format pattern information. In other words, based on the block format indicated by the block format information used by the image decoding apparatus 100, a method for dividing the coding unit indicated by the partition format pattern information can be determined.

[0107] Image decoding apparatus 100 can obtain partitioning pattern information from the bitstream. However, it is not limited to this, and image decoding apparatus 100 and image encoding apparatus 200 can determine pre-arranged partitioning pattern information based on block format information. Image decoding apparatus 100 can determine pre-arranged partitioning pattern information for the largest or smallest coding unit. For example, image decoding apparatus 100 can determine that the partitioning pattern information for the largest coding unit is quaternizable. Furthermore, image decoding apparatus 100 can determine that the partitioning pattern information for the smallest coding unit is "not partitioned". Specifically, image decoding apparatus 100 can determine the size of the largest coding unit to be 256×256. Image decoding apparatus 100 can determine that the pre-arranged partitioning pattern information is quaternizable. "Quaternizable" is a partitioning pattern in which the width and height of the coding unit are both divided into two halves. Image decoding apparatus 100 can obtain coding units with a size of 128×128 each from the largest coding unit with a size of 256×256 based on the partitioning pattern information. Furthermore, the image decoding device 100 can determine the size of the smallest coding unit to be 4×4. The image decoding device 100 can obtain partitioning pattern information indicating "no partitioning" for the smallest coding unit.

[0108] In embodiments of this disclosure, the image decoding apparatus 100 may use block shape information indicating that the current encoding unit has a square shape. For example, the image decoding apparatus 100 may determine, based on partitioning pattern information, whether the square-shaped encoding unit will not be divided, will be vertically divided, will be horizontally divided, or will be divided into four encoding units, etc. (Refer to...) Figure 3 When the block form information of the current coding unit 300 indicates a square, the decoder 120 may not divide the coding unit 310a with the same size as the current coding unit 300 according to the partition form pattern information indicating "no division", or may determine coding units 310b, 310c, 310d, 310e, 310f, etc. based on the partition form pattern information indicating a specific partitioning method.

[0109] Reference Figure 3The image decoding device 100 can determine two coding units 310b obtained by dividing the current coding unit 300 in the vertical direction based on partitioning pattern information indicating "vertical division". The image decoding device 100 can determine two coding units 310c obtained by dividing the current coding unit 300 in the horizontal direction based on partitioning pattern information indicating "horizontal division". The image decoding device 100 can determine four coding units 310d obtained by dividing the current coding unit 300 in both the vertical and horizontal directions based on partitioning pattern information indicating "vertical ternary division". The image decoding device 100 can determine three coding units 310e obtained by dividing the current coding unit 300 in the vertical direction based on partitioning pattern information indicating "vertical ternary division". The image decoding device 100 can determine three coding units 310f obtained by dividing the current coding unit 300 in the horizontal direction based on partitioning pattern information indicating "horizontal ternary division". However, the partitioning pattern into which the square-shaped coding unit can be divided is not limited to the aforementioned forms, and various forms indicated by the partitioning pattern information may exist. The specific partitioning pattern into which the square-shaped coding unit is divided will now be described in detail according to embodiments of the present disclosure.

[0110] Figure 4 The process of determining at least one coding unit by dividing a non-square-shaped coding unit is illustrated in an image decoding apparatus 100 according to an embodiment of the present disclosure.

[0111] In embodiments of this disclosure, the image decoding apparatus 100 may use block-form information indicating that the current coding unit has a non-square shape. The image decoding apparatus 100 may determine, based on the partitioning shape pattern information, whether the current non-square coding unit will not be partitioned or will be partitioned in a specific way. (Refer to...) Figure 4 When the block format information of the current encoding unit 400 or 450 indicates a non-square shape, the image decoding device 100 can determine encoding units 410 or 460 with the same size as the current encoding unit 400 or 450 based on the partition format pattern information indicating "no partitioning," or it can determine encoding units 420a, 420b, 430a, 430b, 430c, 470a, 470b, 480a, 480b, 480c, etc., based on the partition format pattern information indicating a specific partitioning method. Certain partitioning methods for encoding units with non-square shapes will be described in detail according to embodiments of this disclosure.

[0112] In embodiments of this disclosure, the image decoding apparatus 100 can determine the partitioning pattern of the coding units based on block format pattern information, in which case the partitioning pattern information can indicate the number of at least one coding unit generated by dividing the coding units. (Refer to...) Figure 4When the partitioning pattern information indicates that the current coding unit 400 or 450 will be divided into two coding units, the image decoding device 100 can determine the two coding units 420a and 420b or 470a and 470b to be included in the current coding unit by dividing the current coding unit 400 or 450 based on the partitioning pattern information.

[0113] In embodiments of this disclosure, when the image decoding device 100 divides a non-square-shaped current coding unit 400 or 450 based on partitioning pattern information, the image decoding device 100 can divide the current coding unit by considering the position of the long side of the current non-square-shaped coding unit 400 or 450. For example, the image decoding device 100 can determine multiple coding units by dividing the current coding unit 400 or 450 according to the direction of the long side of the current coding unit 400 or 450, considering the form of the current coding unit 400 or 450.

[0114] In embodiments of this disclosure, when the partitioning pattern information indicates that the coding unit will be divided into an odd number of blocks (ternary partitioning), the image decoding device 100 can determine the odd number of coding units to be included in the current coding unit 400 or 450. For example, when the partitioning pattern information indicates that the current coding unit 400 or 450 will be divided into three coding units, the image decoding device 100 can divide the current coding unit 400 or 450 into three coding units 430a, 430b, 430c, 480a, 480b, and 480c.

[0115] In embodiments of this disclosure, the aspect ratio of the current encoding unit 400 or 450 can be 4:1 or 1:4. When the aspect ratio is 4:1, it means the width is longer than the height, so the block shape information can correspond to the horizontal direction. When the aspect ratio is 1:4, it means the width is shorter than the height, so the block shape information can correspond to the vertical direction. The image decoding device 100 can determine whether to divide the current encoding unit into an odd number of blocks based on the partitioning pattern information. Furthermore, the image decoding device 100 can determine the partitioning direction of the current encoding unit 400 or 450 based on the block shape information of the current encoding unit 400 or 450. For example, when the current encoding unit 400 is in the vertical direction, the image decoding device 100 can determine encoding units 430a, 430b, and 430c by dividing the current encoding unit 400 in the horizontal direction. Furthermore, when the current encoding unit 450 is in the horizontal direction, the image decoding device 100 can determine encoding units 480a, 480b, and 480c by dividing the current encoding unit 450 in the vertical direction.

[0116] In embodiments of this disclosure, the image decoding apparatus 100 may determine an odd number of coding units to be included in the current coding unit 400 or 450, and the determined coding units may not all have the same size. For example, a particular coding unit 430b or 480b among the odd number of determined coding units 430a, 430b, 430c, 480a, 480b, and 480c may have a different size than the other coding units 430a, 430c, 480a, and 480c. In other words, the coding units determined by dividing the current coding unit 400 or 450 may have different sizes, and in some cases, each of the odd number of coding units 430a, 430b, 430c, 480a, 480b, and 480c may have a different size.

[0117] In embodiments of this disclosure, when the partitioning pattern information indicates that the coding unit will be divided into an odd number of blocks, the image decoding device 100 can determine the odd number of coding units to be included in the current coding unit 400 or 450. Furthermore, the image decoding device 100 can impose specific restrictions on at least one of the odd number of coding units generated through partitioning. (Refer to...) Figure 4 The image decoding apparatus 100 can perform different decoding processes on the intermediate encoding unit 430b or 480b located among the three encoding units 430a, 430b, and 430c or 480a, 480b, and 480c generated by dividing the current encoding unit 400 or 450, compared to other encoding units 430a and 430b or 480a and 480c. For example, unlike other encoding units 430a and 430c or 480a and 480, the image decoding apparatus 100 can restrict the intermediate encoding unit 430b or 480b from being further divided, or restrict the encoding unit 430b or 480b to be divided only a specific number of times.

[0118] Figure 5 This illustrates the process by which an image decoding apparatus 100 according to an embodiment of the present disclosure divides coding units based on at least one of block-form information and partition-form pattern information.

[0119] In embodiments of this disclosure, the image decoding apparatus 100 may determine whether to divide the square-shaped first coding unit 500 into coding units or not, based on at least one of block format information and partition format pattern information. In embodiments of this disclosure, when the partition format pattern information indicates that the first coding unit 500 will be divided in the horizontal direction, the image decoding apparatus 100 may determine the second coding unit 510 by dividing the first coding unit 500 in the horizontal direction. In embodiments of this disclosure, the terms first coding unit, second coding unit, and third coding unit are terms used to understand the sequential relationship of the division between coding units. For example, the second coding unit may be determined by dividing the first coding unit, and the third coding unit may be determined by dividing the second coding unit. The relationship between the first coding unit, second coding unit, and third coding unit, as used below, can be understood based on the foregoing characteristics.

[0120] In embodiments of this disclosure, the image decoding apparatus 100 may determine, based on partitioning pattern information, whether to divide the determined second coding unit 510 into coding units or not to divide the determined second coding unit 510. (Refer to...) Figure 5 The image decoding device 100 can divide the non-square second coding unit 510, determined by dividing the first coding unit 500, into at least one third coding unit 520a, 520b, 520c, 520d, etc., based on partitioning pattern information, or it may not divide the second coding unit 510. The image decoding device 100 can obtain partitioning pattern information, and the image decoding device 100 can divide the first coding unit 500 into multiple second coding units (e.g., 510) of various shapes based on the obtained partitioning pattern information, and divide the second coding unit 510 according to the method of dividing the first coding unit 500 based on the partitioning pattern information. In the embodiments of this disclosure, when the first coding unit 500 is divided into the second coding unit 510 based on the partitioning pattern information of the first coding unit 500, the second coding unit 510 can also be divided into third coding units (e.g., 520a, 520b, 520c, 520d, etc.) based on the partitioning pattern information of the second coding unit 510. In other words, coding units can be recursively partitioned based on the partitioning pattern information associated with each coding unit. Therefore, square-shaped coding units can be determined from non-square-shaped coding units, and square-shaped coding units can be recursively partitioned so that non-square-shaped coding units can be determined.

[0121] Reference Figure 5A specific coding unit (e.g., the middle coding unit or a square coding unit) among an odd number of third coding units 520b, 520c, and 520d, determined by dividing a non-square-shaped second coding unit 510, can be recursively partitioned. In embodiments of this disclosure, for one of the odd number of third coding units 520b, 520c, and 520d, the non-square-shaped third coding unit 520b can be horizontally divided into a plurality of fourth coding units. One of the plurality of fourth coding units 530a, 530b, 530c, and 530d, or the non-square-shaped fourth coding unit 530b or 530d, can be further divided into a plurality of coding units. For example, the non-square-shaped fourth coding unit 530b or 530d can be further divided into an odd number of coding units. A method for recursive partitioning of coding units will be described later with reference to embodiments of this disclosure.

[0122] In embodiments of this disclosure, the image decoding apparatus 100 may divide each of the third coding units 520a, 520b, 520c, 520d, etc., into coding units based on partitioning pattern information. Furthermore, the image decoding apparatus 100 may determine, based on the partitioning pattern information, not to divide the second coding unit 510. According to embodiments of this disclosure, the image decoding apparatus 100 may divide the non-square second coding unit 510 into an odd number of third coding units 520b, 520c, and 520d. The image decoding apparatus 100 may impose specific restrictions on certain third coding units among the odd number of third coding units 520b, 520c, and 520d. For example, the image decoding apparatus 100 may restrict the coding unit 520c located in the middle of the odd number of third coding units 520b, 520c, and 520d from being further divided, or restrict the number of times the coding unit 520c can be divided.

[0123] Reference Figure 5 The image decoding apparatus 100 may restrict the middle encoding unit 520c, located among an odd number of third encoding units 520b, 520c, and 520d included in the non-square second encoding unit 510, such that it is not further divided, but is divided into a specific partition form (e.g., only four encoding units or forms corresponding to the form into which the second encoding unit 510 is divided), or it is divided only a specific number of times (e.g., n times, where n>0). However, the restriction on the middle encoding unit 520c is merely an illustrative example and should not be construed as being limited to the foregoing example, but rather as encompassing various restrictions under which the middle encoding unit 520c can be decoded differently from the other encoding units 520b and 520d.

[0124] In embodiments of this disclosure, the image decoding device 100 can obtain partitioning pattern information for dividing the current coding unit from a specific location in the current coding unit.

[0125] Figure 6 This invention illustrates a method by which an image decoding apparatus 100, according to an embodiment of the present disclosure, determines a specific coding unit from an odd number of coding units.

[0126] Reference Figure 6 The partition pattern information of the current coding unit 600 or 650 can be obtained from a sample at a specific position among multiple samples included in the current coding unit 600 or 650 (e.g., sample 640 or 690 located in the middle). However, a position in the current coding unit 600 from which at least one piece of partition pattern information can be obtained should not be interpreted as limited to Figure 6 The middle position shown should be interpreted as encompassing various positions that may be included in the current coding unit 600 (e.g., top, bottom, left, right, upper left, lower left, upper right, or lower right). The image decoding device 100 can obtain partitioning pattern information from a specific position and determine whether to divide the current coding unit into coding units of various forms and sizes.

[0127] In embodiments of this disclosure, when a current coding unit is divided into a specific number of coding units, the image decoding apparatus 100 may select one of them. Various methods may exist for selecting one coding unit from a plurality of coding units, and these will be described below through embodiments of this disclosure.

[0128] In embodiments of this disclosure, the image decoding device 100 may divide the current encoding unit into multiple encoding units and determine the encoding unit at a specific location.

[0129] In embodiments of this disclosure, the image decoding apparatus 100 can use information indicating the position of each of the odd-numbered coding units to determine the middle coding unit among the odd-numbered coding units. (Refer to...) Figure 6The image decoding device 100 can determine an odd number of coding units 620a, 620b, and 620c, or 660a, 660b, and 660c, by dividing the current coding unit 600 or 650. The image decoding device 100 can determine the center coding unit 620b or 660b by using information about the positions of the odd number of coding units 620a, 620b, and 620c, or 660a, 660b, and 660c. For example, the image decoding device 100 can determine the middle coding unit 620b by determining the positions of coding units 620a, 620b, and 620c based on information indicating the positions of specific samples included in coding units 620a, 620b, and 620c. Specifically, the image decoding device 100 can determine the position of the encoding units 620a, 620b and 620c based on the information indicating the position of the upper left sample points 630a, 630b and 630c of the encoding units 620a, 620b and 620c, and determine the encoding unit 620b located in the middle.

[0130] In embodiments of this disclosure, information indicating the positions of the top-left samples 630a, 630b, and 630c respectively included in encoding units 620a, 620b, and 620c may include information about the positions or coordinates of the encoding units 620a, 620b, and 620c in the frame. In embodiments of this disclosure, information indicating the positions of the top-left samples 630a, 630b, and 630c included in the respective encoding units 620a, 620b, and 620c may include information indicating the width or height of the encoding units 620a, 620b, and 620c included in the current encoding unit 600, and the width or height may correspond to information indicating the difference between the coordinates of the encoding units 620a, 620b, and 620c in the frame. In other words, the image decoding apparatus 100 can determine the middle encoding unit 620b by directly using information about the positions or coordinates of the encoding units 620a, 620b, and 620c in the frame or by using information about the width or height of the encoding unit corresponding to the difference between the coordinates.

[0131] In embodiments of this disclosure, information indicating the position of the upper left sample 630a of the upper encoding unit 620a can indicate coordinates (xa, ya), information indicating the position of the upper left sample 630b of the center encoding unit 620b can indicate coordinates (xb, yb), and information indicating the position of the upper left sample 630c of the lower encoding unit 620c can indicate coordinates (xc, yc). The image decoding apparatus 100 can determine the center encoding unit 620b by using the coordinates of the upper left samples 630a, 630b, and 630c included in the respective encoding units 620a, 620b, and 620c. For example, when the coordinates of the upper left samples 630a, 630b, and 630c are arranged in ascending or descending order, the encoding unit 620b including the coordinates (xb, yb) of the middle sample 630b can be determined as the middle encoding unit among the encoding units 620a, 620b, and 620c determined by dividing the current encoding unit 600. The coordinates indicating the positions of the upper left samples 630a, 630b, and 630c can be coordinates indicating their absolute positions within the frame. Furthermore, based on the position of the upper left sample 630a of the upper encoding unit 620a, coordinates (dxb, dyb) can be used as information indicating the relative position of the upper left sample 630b of the center encoding unit 620b, and coordinates (dxc, dyc) can be used as information indicating the relative position of the upper left sample 630c of the lower encoding unit 620c. Moreover, the method of determining the encoding unit at a specific position by using the coordinates of samples as information indicating the positions of samples included in the encoding unit should not be construed as limited to the aforementioned method, but should be interpreted using various arithmetic methods that can utilize the coordinates of samples.

[0132] In embodiments of this disclosure, the image decoding device 100 may divide the current encoding unit 600 into a plurality of encoding units 620a, 620b, and 620c, and select encoding units from the encoding units 620a, 620b, and 620c based on specific criteria. For example, the image decoding device 100 may select encoding unit 620b with different sizes from the encoding units 620a, 620b, and 620c.

[0133] In embodiments of this disclosure, the image decoding apparatus 100 can determine the width or height of each of the encoding units 620a, 620b, and 620c by using coordinates (xa, ya) indicating the position of the upper left sample point 630a of the upper encoding unit 620a, coordinates (xb, yb) indicating the position of the upper left sample point 630b of the center encoding unit 620b, and coordinates (xc, yc) indicating the position of the upper left sample point 630c of the lower encoding unit 620c. The image decoding apparatus 100 can also determine the dimensions of each encoding unit 620a, 620b, and 620c by using coordinates (xa, ya), (xb, yb), and (xc, yc) indicating the positions of the encoding units 620a, 620b, and 620c. In embodiments of this disclosure, the image decoding apparatus 100 can determine the width of the upper encoding unit 620a as the width of the current encoding unit 600. The image decoding apparatus 100 can determine the height of the upper encoding unit 620a as (yb-ya). In embodiments of this disclosure, the image decoding apparatus 100 can determine the width of the center encoding unit 620b as the width of the current encoding unit 600. The image decoding apparatus 100 can determine the height of the center encoding unit 620b as (yc-yb). In embodiments of this disclosure, the image decoding apparatus 100 can determine the width or height of the lower encoding unit by using the width or height of the current encoding unit and the width or height of the upper encoding unit 620a and the center encoding unit 620b. The image decoding apparatus 100 can determine encoding units with different sizes from other encoding units based on the determined widths and heights of the encoding units 620a, 620b, and 620c. (Refer to...) Figure 6 The image decoding apparatus 100 can determine a central encoding unit 620b, whose size differs from that of the upper encoding unit 620a and the lower encoding unit 620c, as an encoding unit at a specific location. However, the process by which the image decoding apparatus 100 determines an encoding unit with a size different from other encoding units is merely an illustrative example of determining an encoding unit at a specific location using the size of the encoding unit determined based on sample point coordinates, and various processes may exist for determining an encoding unit at a specific location by comparing the size of the encoding unit determined according to specific sample point coordinates.

[0134] The image decoding apparatus 100 can determine the width or height of each of the encoding units 660a, 660b, and 660c by using coordinates (xd, yd) indicating the position of the upper left sample 670a of the left encoding unit 660a, coordinates (xe, ye) indicating the position of the upper left sample 670b of the center encoding unit 660b, and coordinates (xf, yf) indicating the position of the upper left sample 670c of the right encoding unit 660c. The image decoding apparatus 100 can also determine the dimensions of each encoding unit 660a, 660b, and 660c by using coordinates (xd, yd), (xe, ye), and (xf, yf) indicating the positions of the encoding units 660a, 660b, and 660c.

[0135] In embodiments of this disclosure, the image decoding apparatus 100 may determine the width of the left encoding unit 660a as (xe-xd). The image decoding apparatus 100 may determine the height of the left encoding unit 660a as the height of the current encoding unit 650. In embodiments of this disclosure, the image decoding apparatus 100 may determine the width of the center encoding unit 660b as (xf-xe). The image decoding apparatus 100 may determine the height of the center encoding unit 660b as the height of the current encoding unit 600. In embodiments of this disclosure, the image decoding apparatus 100 may determine the width or height of the right encoding unit 660c by using the width or height of the current encoding unit 650 and the width and height of the left encoding unit 660a and the center encoding unit 660b. The image decoding apparatus 100 may determine encoding units with different dimensions from other encoding units based on the determined widths and heights of the encoding units 660a, 660b, and 660c. (Refer to...) Figure 6 The image decoding apparatus 100 can determine a central encoding unit 660b, whose size differs from that of the left encoding unit 660a and the right encoding unit 660c, as an encoding unit at a specific location. However, the process by which the image decoding apparatus 100 determines an encoding unit with a size different from other encoding units is merely an illustrative example of determining an encoding unit at a specific location using the size of the encoding unit determined based on sample point coordinates, and various processes may exist for determining an encoding unit at a specific location by comparing the size of the encoding unit determined according to specific sample point coordinates.

[0136] The position of the sample considered in order to determine the position of the coding unit should not be interpreted as limited to the aforementioned upper left position, but can be interpreted as being determined by using information about the position of any sample included in the coding unit.

[0137] In embodiments of this disclosure, the image decoding apparatus 100 may consider the form of the current coding unit and select a coding unit at a specific position from an odd number of coding units determined by dividing the current coding unit. For example, when the current coding unit has a non-square form with a width greater than its height, the image decoding apparatus 100 may determine a coding unit at a specific position in the horizontal direction. In other words, the image decoding apparatus 100 may determine one coding unit among coding units whose position varies in the horizontal direction and impose restrictions on that coding unit. When the current coding unit has a non-square form with a height greater than its width, the image decoding apparatus 100 may determine a coding unit at a specific position in the vertical direction. In other words, the image decoding apparatus 100 may determine one coding unit among coding units whose position varies in the vertical direction and impose restrictions on that coding unit.

[0138] In embodiments of this disclosure, the image decoding apparatus 100 can determine a coding unit at a specific position from an even number of coding units using information indicating the position of each of the even number of coding units. The image decoding apparatus 100 can determine the even number of coding units by partitioning the current coding unit (binary partitioning), and determine the coding unit at a specific position by using information about the positions of the even number of coding units. The detailed process used here can correspond to that described above. Figure 6 The aforementioned process for determining a coding unit at a specific position (e.g., in the middle) from an odd number of coding units, as described in the document, will therefore be omitted from the description.

[0139] In embodiments of this disclosure, when a non-square current coding unit is divided into multiple coding units, certain information about coding units at a specific location can be used during the partitioning process to determine the coding unit at that specific location from among the multiple coding units. For example, the image decoding apparatus 100 can use at least one of block-form information and partition-form pattern information stored in samples included in the central coding unit during the partitioning process to determine the coding unit located in the middle of the multiple coding units divided from the current coding unit.

[0140] Reference Figure 6The image decoding apparatus 100 can divide the current coding unit 600 into multiple coding units 620a, 620b, and 620c based on partition format pattern information, and determine the coding unit 620b located in the middle of the multiple coding units 620a, 620b, and 620c. Furthermore, the image decoding apparatus 100 can determine the middle coding unit 620b by considering the position where the partition format pattern information is obtained. Specifically, the partition format pattern information of the current coding unit 600 can be obtained from a sample point 640 located in the middle of the current coding unit 600, and when the current coding unit 600 is divided into multiple coding units 620a, 620b, and 620c based on the partition format pattern information, the coding unit 620b including the sample point 640 can be determined as the middle coding unit. However, the information used to determine a coding unit as the middle coding unit should not be interpreted as limited to partition format pattern information, but various types of information can be used in the process of determining the middle coding unit.

[0141] In embodiments of this disclosure, specific information for identifying a specific location of the coding unit can be obtained from specific samples contained in the coding unit to be determined. (Refer to...) Figure 6 The image decoding apparatus 100 can use partitioned pattern information obtained from samples at specific locations in the current coding unit 600 (e.g., samples located at the center of the current coding unit 600) to determine coding units at specific locations (e.g., coding units located in the middle of the plurality of coding units) among the plurality of coding units 620a, 620b, and 620c determined by dividing the current coding unit 600. In other words, the image decoding apparatus 100 can determine samples at specific locations by considering the block shape of the current coding unit 600, and can determine and impose specific constraints on coding unit 620b among the plurality of coding units 620a, 620b, and 620c determined by dividing the current coding unit 600 that includes samples from which specific information (e.g., partitioned pattern information) can be obtained. (Refer to...) Figure 6 In embodiments of this disclosure, the image decoding apparatus 100 may determine a sample 640 located in the middle of the current encoding unit 600 as a sample from which specific information can be obtained, and may impose specific restrictions on the encoding unit 620b including the sample 640 during the decoding process. However, the location of the sample from which specific information can be obtained should not be interpreted as limited to the aforementioned location, but can be interpreted as including samples at any location in the encoding unit 620b to which restrictions will be imposed.

[0142] In embodiments of this disclosure, the location of a sample from which specific information can be obtained can be determined based on the shape of the current encoding unit 600. In embodiments of this disclosure, block shape information can determine whether the shape of the current encoding unit is square or non-square, and the location of a sample from which specific information can be obtained can be determined based on the shape. For example, the image decoding apparatus 100 can use at least one of the width and height information of the current encoding unit to determine a sample located on a boundary that bisects at least one of the width and height of the current encoding unit as a sample from which specific information can be obtained. In another example, when the block shape information associated with the current encoding unit indicates a non-square shape, the image decoding apparatus 100 can determine one of the sample points adjacent to the boundary that bisects the long side of the current encoding unit as a sample from which specific information can be obtained.

[0143] In embodiments of this disclosure, when a current coding unit is divided into multiple coding units, the image decoding device 100 can use partitioning pattern information to determine the coding unit at a specific location among the multiple coding units. In embodiments of this disclosure, the image decoding device 100 can obtain partitioning pattern information from samples at specific locations included in the coding unit, and the image decoding device 100 can use the partitioning pattern information obtained from samples at specific locations included in each of the multiple coding units to divide the multiple coding units generated by dividing the current coding unit. That is, the coding units can be recursively divided based on the partitioning pattern information obtained from samples at specific locations included in each coding unit. Combined with... Figure 5 The recursive partitioning process for the coding unit has been described, so its detailed description will not be repeated.

[0144] In embodiments of this disclosure, the image decoding apparatus 100 can determine at least one coding unit by dividing the current coding unit, and determine the order in which the at least one coding unit is decoded based on a specific block (e.g., the current coding unit).

[0145] Figure 7 The following diagram illustrates the order in which the image decoding apparatus 100 processes the plurality of coding units when determining the plurality of coding units by dividing the current coding unit, according to an embodiment of the present disclosure.

[0146] In embodiments of this disclosure, based on partitioning pattern information, the image decoding device 100 can determine the second coding units 710a and 710b by dividing the first coding unit 700 in the vertical direction, determine the second coding units 730a and 730b by dividing the first coding unit 700 in the horizontal direction, or determine the second coding units 750a, 750b, 750c and 750d by dividing the first coding unit 700 in both the vertical and horizontal directions.

[0147] Reference Figure 7 The image decoding apparatus 100 can determine the order in which second coding units 710a and 710b, determined by dividing the first coding unit 700 in the vertical direction, will be processed along the horizontal direction 710c. The image decoding apparatus 100 can determine the processing order of second coding units 730a and 730b, determined by dividing the first coding unit 700 in the horizontal direction, as being along the vertical direction 730c. The image decoding apparatus 100 can determine second coding units 750a, 750b, 750c, and 750d, determined by dividing the first coding unit 700 in both the vertical and horizontal directions according to a specific order (e.g., raster scan order or z-scan order 750e) where coding units in one row are processed and coding units in the next row are processed.

[0148] In embodiments of this disclosure, the image decoding apparatus 100 may recursively divide encoding units. (Refer to...) Figure 7 The image decoding apparatus 100 can determine a plurality of coding units 710a, 710b, 730a, 730b, 750a, 750b, 750c, and 750d by dividing a first coding unit 700, and recursively divide each of the determined plurality of coding units 710a, 710b, 730a, 730b, 750a, 750b, 750c, and 750d. The method of dividing the plurality of coding units 710a, 710b, 730a, 730b, 750a, 750b, 750c, and 750d can correspond to the method of dividing the first coding unit 700. Therefore, each of the plurality of coding units 710a, 710b, 730a, 730b, 750a, 750b, 750c, and 750d can be independently divided into a plurality of coding units. (Refer to...) Figure 7 The image decoding device 100 can determine the second coding units 710a and 710b by dividing the first coding unit 700 in the vertical direction, and also determines whether to divide each of the second coding units 710a and 710b independently or not to divide each of the second coding units 710a and 710b.

[0149] In embodiments of this disclosure, the image decoding apparatus 100 may divide the second encoding unit 710a on the left side in the horizontal direction into third encoding units 720a and 720b, and may not divide the second encoding unit 710b on the right side.

[0150] In embodiments of this disclosure, the processing order of coding units can be determined based on the coding unit partitioning process. In other words, the processing order of the partitioned coding units can be determined based on the processing order of the coding units just before the coding units were partitioned. The image decoding apparatus 100 can determine the processing order of the third coding units 720a and 720b, which are determined by separating the second coding unit 710a on the left and the second coding unit 710b on the right. Since the third coding units 720a and 720b are determined by partitioning the second coding unit 710a on the left in the horizontal direction, the third coding units 720a and 720b can be processed in the vertical direction 720c. Furthermore, since the processing order of the second coding unit 710a on the left and the second coding unit 710b on the right corresponds to the horizontal direction 710c, the coding unit 710b on the right can be processed after the third coding units 720a and 720b included in the second coding unit 710a on the left are processed along the vertical direction 720c. Since the foregoing description pertains to a process for determining the order in which coding units are processed based on coding units before they are divided, it should not be construed as limited to the foregoing embodiments, but rather as encompassing the use of various methods by which coding units, determined by being divided into various forms, are processed individually in a specific order.

[0151] Figure 8 An embodiment of the present disclosure illustrates the process by which an image decoding apparatus 100 determines that the current coding unit will be divided into an odd number of coding units when coding units cannot be processed in a specific order.

[0152] In embodiments of this disclosure, the image decoding apparatus 100 may determine, based on obtained partitioning pattern information, to divide the current coding unit into an odd number of coding units. (Refer to...) Figure 8 The square-shaped first encoding unit 800 can be divided into non-square-shaped second encoding units 810a and 810b, and the second encoding units 810a and 810b can be individually divided into third encoding units 820a and 820b or 820c, 820d and 820e. In embodiments of this disclosure, the image decoding device 100 can determine a plurality of third encoding units 820a and 820b by horizontally dividing the left encoding unit 810a in the second encoding unit, and divide the right encoding unit 810b into an odd number of third encoding units 820c, 820d and 820e.

[0153] In embodiments of this disclosure, the image decoding apparatus 100 can determine whether there is an odd number of partitioned coding units by determining whether the third coding units 820a, 820b, 820c, 820d, and 820e can be processed in a specific order. (See also...) Figure 8The image decoding device 100 can determine the third coding units 820a, 820b, 820c, 820d, and 820e by recursively dividing the first coding unit 800. Based on at least one of block-form information and partition-form pattern information, the image decoding device 100 can determine whether the first coding unit 800, the second coding units 810a and 810b, or the third coding units 820a, 820b, 820c, 820d, and 820e are divided into an odd number of coding units. For example, the rightmost coding unit in the second coding units 810a and 810b can be divided into an odd number of third coding units 820c, 820d, and 820e. Multiple coding units included in the first coding unit 800 can be processed in a specific order (e.g., a zigzag scanning order) 830, and the image decoding device 100 can determine whether the conditions are met for processing the third coding units 820c, 820d, and 820e determined by dividing the second coding unit 810b on the right side according to the specific order.

[0154] In embodiments of this disclosure, the image decoding apparatus 100 can determine whether a condition is met that the third encoding units 820a, 820b, 820c, 820d, and 820e included in the first encoding unit 800 can be processed in a specific order. This condition relates to whether at least one of the width and height of the second encoding unit 810a or 810b is halved according to the boundary of the third encoding units 820a, 820b, 820c, 820d, and 820e. For example, the third encoding units 820a and 820b, determined by halving the height of the non-square second encoding unit 810a on the left, can satisfy this condition. Since the boundary of the third encoding units 820c, 820d, and 820e, determined by dividing the second encoding unit 810b on the right into three encoding units, fails to halve the width or height of the second encoding unit 810b on the right, it can be determined that the third encoding units 820c, 820d, and 820e do not satisfy the condition. The image decoding apparatus 100 may determine that the failure to meet the conditions is due to a discontinuous scanning order, and based on the determination, determine that the second coding unit 810b on the right is divided into an odd number of coding units. In embodiments of this disclosure, the image decoding apparatus 100 may impose specific restrictions on coding units at specific positions within the odd number of divided coding units, and such restrictions or specific positions have been described in the foregoing embodiments of this disclosure, and therefore will not be repeated in detail.

[0155] Figure 9 The process of determining at least one coding unit by dividing a first coding unit 900 is illustrated in an image decoding apparatus 100 according to an embodiment of the present disclosure.

[0156] In embodiments of this disclosure, the image decoding apparatus 100 may divide the first coding unit 900 based on partitioned pattern information obtained by the bitstream acquirer 110. The square-shaped first coding unit 900 may be divided into four square-shaped coding units or multiple non-square-shaped coding units. For example, referring to… Figure 9 When the first coding unit 900 is square and the partitioning pattern information indicates that the partition is a non-square coding unit, the image decoding device 100 can divide the first coding unit 900 into multiple non-square coding units. Specifically, when the partitioning pattern information indicates that an odd number of coding units are determined by horizontal or vertical partitioning of the first coding unit 900, the image decoding device 100 can divide the square-shaped first coding unit 900 into an odd number of coding units: second coding units 910a, 910b, and 910c determined by vertical partitioning or second coding units 920a, 920b, and 920c determined by horizontal partitioning.

[0157] In embodiments of this disclosure, the image decoding apparatus 100 can determine whether a condition is met that the second encoding units 910a, 910b, 910c, 920a, 920b, and 920c included in the first encoding unit 900 can be processed in a specific order. This condition relates to whether at least one of the width and height of the first encoding unit 900 is divided in half according to the boundaries of the second encoding units 910a, 910b, 910c, 920a, 920b, and 920c. (Refer to...) Figure 9 Since the boundaries of the second encoding units 910a, 910b, and 910c, determined by vertically dividing the square-shaped first encoding unit 900, make it impossible to halve the width of the first encoding unit 900, the first encoding unit 900 can be determined as not satisfying the condition of being processed in a specific order. Since the boundaries of the second encoding units 920a, 920b, and 920c, determined by horizontally dividing the square-shaped first encoding unit 900, make it impossible to halve the height of the first encoding unit 900, the first encoding unit 900 can be determined as not satisfying the condition of being processed in a specific order. The image decoding apparatus 100 can determine that the failure to meet the conditions is due to a discontinuous scanning order, and based on the determination, determines that the first encoding unit 900 is divided into an odd number of encoding units. In embodiments of this disclosure, the image decoding apparatus 100 can impose specific restrictions on encoding units at specific positions within the odd number of divided encoding units, and such restrictions or specific positions have been described in the foregoing embodiments of this disclosure, therefore their detailed description will not be repeated.

[0158] In embodiments of this disclosure, the image decoding apparatus 100 can determine various forms of coding units by dividing a first coding unit.

[0159] Reference Figure 9 The image decoding device 100 can divide the square-shaped first encoding unit 900 and the non-square-shaped first encoding unit 930 or 950 into encoding units of various shapes.

[0160] Figure 10 The image decoding apparatus 100 restricts the form in which the second coding unit can be divided when a non-square second coding unit, determined by dividing the first coding unit 1000, satisfies certain conditions, according to an embodiment of the present disclosure.

[0161] In embodiments of this disclosure, the image decoding apparatus 100 may determine, based on partitioning pattern information obtained by the bitstream acquirer 110, to divide a square-shaped first coding unit 1000 into non-square-shaped second coding units 1010a, 1010b, 1020a, and 1020b. The second coding units 1010a, 1010b, 1020a, and 1020b may be divided independently. Therefore, the image decoding apparatus 100 may determine, based on partitioning pattern information associated with each of the second coding units 1010a, 1010b, 1020a, and 1020b, whether to divide each of them or not. In embodiments of this disclosure, the image decoding apparatus 100 can determine third encoding units 1012a and 1012b by horizontally dividing the left-side non-square-shaped second encoding unit 1010a, wherein the left-side non-square-shaped second encoding unit 1010a is determined by vertically dividing the first encoding unit 1000. On the other hand, when the image decoding apparatus 100 horizontally divides the left-side second encoding unit 1010a, it can restrict the right-side second encoding unit 1010b to be divided in the same horizontal direction as the left-side second encoding unit 1010a. In the case where the third encoding units 1014a and 1014b are determined by dividing the right-side second encoding unit 1010b in the same direction, the third encoding units 1012a, 1012b, 1014a, and 1014b can be determined by horizontally dividing the left-side second encoding unit 1010a and the right-side second encoding unit 1010b, respectively. However, this is the same result as when the image decoding device 100 divides the first encoding unit 1000 into four square-shaped second encoding units 1030a, 1030b, 1030c and 1030d, which may be inefficient from the perspective of image decoding.

[0162] In embodiments of this disclosure, the image decoding apparatus 100 can determine third encoding units 1022a, 1022b, 1024a, and 1024b by vertically dividing a second encoding unit 1020a or 1020b of a non-square shape determined by horizontally dividing the first encoding unit 1000. On the other hand, when the image decoding apparatus 100 vertically divides one of the second encoding units (e.g., the upper second encoding unit 1020a), for the reasons described above, the image decoding apparatus 100 can restrict another second encoding unit (e.g., the lower second encoding unit 1020b) to be divided in the same vertical direction as the direction in which the upper second encoding unit 1020a is divided.

[0163] Figure 11 The process of dividing the image decoding apparatus 100 into square-shaped encoding units is illustrated according to an embodiment of the present disclosure when the partitioning pattern information does not indicate that the encoding units are divided into four square shapes.

[0164] In embodiments of this disclosure, the image decoding apparatus 100 can determine second coding units 1110a, 1110b, 1120a, and 1120b by dividing the first coding unit 1100 based on partitioning pattern information. The partitioning pattern information may include information about the various forms in which the coding unit can be divided, but may not include information about dividing a coding unit into four square-shaped coding units. Based on the partitioning pattern information, the image decoding apparatus 100 cannot divide the square-shaped first coding unit 1100 into four square-shaped second coding units 1130a, 1130b, 1130c, and 1130d. Based on the partitioning pattern information, the image decoding apparatus 100 can determine non-square-shaped second coding units 1110a, 1110b, 1120a, 1120b, etc.

[0165] In embodiments of this disclosure, the image decoding device 100 can independently divide each of the non-square-shaped second encoding units 1110a, 1110b, 1120a, 1120b, etc. Each of the second encoding units 1110a, 1110b, 1120a, 1120b, etc., can be divided in a specific order using a recursive method, wherein the recursive method can correspond to the method of dividing the first encoding unit 1100 based on partitioning pattern information.

[0166] For example, the image decoding device 100 can determine square-shaped third coding units 1112a and 1112b by horizontally dividing the second coding unit 1110a on the left, and square-shaped third coding units 1114a and 1114b by horizontally dividing the second coding unit 1110b on the right. Furthermore, the image decoding device 100 can determine square-shaped third coding units 1116a, 1116b, 1116c, and 1116d by horizontally dividing both the second coding unit 1110a on the left and the second coding unit 1110b on the right. In this case, the coding units can be determined in the same manner as when the first coding unit 1100 is divided into four square-shaped second coding units 1130a, 1130b, 1130c, and 1130d.

[0167] In another example, the image decoding device 100 can determine square-shaped third coding units 1122a and 1112b by vertically dividing the upper second coding unit 1120a, and square-shaped third coding units 1124a and 1124b by vertically dividing the lower second coding unit 1120b. Furthermore, the image decoding device 100 can determine square-shaped third coding units 1126a, 1126b, 1126a, and 1126b by vertically dividing both the upper and lower second coding units 1120a and 1120b. In this case, the coding units can be determined in the same manner as when the first coding unit 1100 is divided into four square-shaped second coding units 1130a, 1130b, 1130c, and 1130d.

[0168] Figure 12 The processing order among multiple coding units according to embodiments of the present disclosure is shown to vary depending on the process used to divide the coding units.

[0169] In embodiments of this disclosure, the image decoding apparatus 100 may divide the first coding unit 1200 based on partitioning pattern information. When the block shape is square and the partitioning pattern information indicates that the first coding unit 1200 is divided according to at least one of the horizontal and vertical directions, the image decoding apparatus 100 may determine the second coding unit (e.g., 1210a, 1210b, 1220a, 1220b, etc.) by dividing the first coding unit 1200. (See also...) Figure 12The non-square-shaped second coding units 1210a, 1210b, 1220a, and 1220b, determined by dividing the first coding unit 1200 only in the horizontal or vertical direction, can be individually partitioned based on the partitioning pattern information of each second coding unit. For example, the image decoding device 100 can determine third coding units 1216a, 1216b, 1216c, and 1216d by horizontally partitioning each of the second coding units 1210a and 1210b generated by vertically partitioning the first coding unit 1200, and determine third coding units 1226a, 1226b, 1226c, and 1226d by vertically partitioning each of the second coding units 1220a and 1220b generated by horizontally partitioning the first coding unit 1200. This has already been combined with... Figure 11 The process for dividing the second coding units 1210a, 1210b, 1220a and 1220b is described, and therefore its detailed description will be omitted.

[0170] In embodiments of this disclosure, the image decoding apparatus 100 can process the encoding units in a specific order. (Already combined with...) Figure 7 The characteristics of processing coding units in a specific order are described, therefore their detailed description will be omitted. (See reference...) Figure 12 The image decoding device 100 can determine the processing order of four square-shaped third encoding units 1216a, 1216b, 1216c, and 1216d, or 1226a, 1226b, 1226c, and 1226d, by dividing the first encoding unit 1200 into square shapes. In embodiments of this disclosure, the image decoding device 100 can determine the processing order of the third encoding units 1216a, 1216b, 1216c, and 1216d, or 1226a, 1226b, 1226c, and 1226d, according to the form in which the first encoding unit 1200 is divided.

[0171] In embodiments of this disclosure, the image decoding device 100 can determine the third encoding units 1216a, 1216b, 1216c, and 1216d by horizontally dividing each of the second encoding units 1210a and 1210b generated by vertical division. The image decoding device 100 can process the third encoding units 1216a, 1216b, 1216c, and 1216d in sequence 1217, wherein the third encoding units 1216a and 1216c included in the second encoding unit 1210a on the left side are processed first in the vertical direction, and the third encoding units 1216b and 1216d included in the second encoding unit 1210b on the right side are processed in the vertical direction.

[0172] In embodiments of this disclosure, the image decoding device 100 can determine third encoding units 1226a, 1226b, 1226c, and 1226d by vertically dividing each of the second encoding units 1220a and 1220b generated by horizontal division. The image decoding device 100 can process the third encoding units 1226a, 1226b, 1226c, and 1226d in sequence 1227, wherein the third encoding units 1226a and 1226b included in the upper second encoding unit 1220a are processed first in the horizontal direction, and the third encoding units 1226c and 1226d included in the lower second encoding unit 1220b are processed in the horizontal direction.

[0173] Reference Figure 12 The square-shaped third coding units 1216a, 1216b, 1216c, and 1216d or 1226a, 1226b, 1226c, and 1226d can be determined by dividing each of the second coding units 1210a, 1210b, 1220a, and 1220b. The second coding units 1210a and 1210b determined by vertical division and the second coding units 1220a and 1220b determined by horizontal division are divided into different forms, but as can be seen from the subsequently determined third coding units 1216a, 1216b, 1216c, and 1216d and 1226a, 1226b, 1226c, and 1226d, the first coding unit 1200 is ultimately divided into coding units of the same form. Therefore, although the same type of coding unit is ultimately determined by recursively dividing the coding unit through different processes based on the partition pattern information, the image decoding device 100 can process multiple coding units that are determined to have the same type in different orders.

[0174] Figure 13 The present disclosure illustrates a process for determining the depth of a coding unit as the shape and size of the coding unit varies when multiple coding units are determined by recursively dividing coding units.

[0175] In embodiments of this disclosure, the image decoding apparatus 100 can determine the depth of the coding unit according to a specific criterion. For example, the specific criterion could be the length of the long side of the coding unit. When the length of the long side of the current coding unit is obtained by dividing the length of the long side of the coding unit before partitioning by 2n (where n>0), the image decoding apparatus 100 can determine that the depth of the current coding unit has increased by n compared to the depth of the coding unit before partitioning. The coding unit with increased depth is referred to as a lower depth coding unit.

[0176] Reference Figure 13In embodiments of this disclosure, based on block shape information indicating a square shape (e.g., block shape information indicating "0: SQUARE"), the image decoding device 100 can divide a square-shaped first encoding unit 1300, such that a second encoding unit 1302, a third encoding unit 1304, etc., can be determined. Assuming the size of the square-shaped first encoding unit 1300 is 2N×2N, the width and height of the first encoding unit 1300 can be divided into half of said size, such that the determined second encoding unit 1302 can have a size of N×N. Furthermore, the third encoding unit 1304, determined by dividing the width and height of the second encoding unit 1302 into half of its size, can have a size of N / 2xN / 2. In this case, the width and height of the third encoding unit 1304 correspond to one-quarter of the size of the first encoding unit 1300. When the depth of the first coding unit 1300 is D, the depth of the second coding unit 1302, which has half the width and height of the first coding unit 1300, can be D+1, and the depth of the third coding unit 1304, which has one-quarter the width and height of the first coding unit 1300, can be D+2.

[0177] In embodiments of this disclosure, based on block form information indicating a non-square shape (e.g., block form information may indicate "1: NS_VER" or "2: NS_HOR", where "1: NS_VER" indicates a non-square shape whose height is greater than its width, and "2: NS_HOR" indicates a non-square shape whose width is greater than its height), the image decoding device 100 can divide a non-square shape into a first encoding unit 1310 or 1320, thereby determining a second encoding unit 1312 or 1322, a third encoding unit 1314 or 1324, etc.

[0178] The image decoding device 100 can determine a second coding unit (e.g., 1302, 1312, 1322, etc.) by dividing the width and height of a first coding unit 1310 with a size of N×2N. Specifically, the image decoding device 100 can horizontally divide the first coding unit 1310 to determine a second coding unit 1302 with a size of N×N or a second coding unit 1322 with a size of N×N / 2, and can horizontally and vertically divide the first coding unit 1310 to determine a second coding unit 1312 with a size of N / 2×N.

[0179] In embodiments of this disclosure, the image decoding apparatus 100 can determine a second encoding unit (e.g., 1302, 1312, 1322, etc.) by dividing the width and height of a first encoding unit 1320 with a size of 2N×N. Specifically, the image decoding apparatus 100 can vertically divide the first encoding unit 1320 to determine a second encoding unit 1302 with a size of N×N or a second encoding unit 1312 with a size of N / 2×N, and can horizontally and vertically divide the first encoding unit 1320 to determine a second encoding unit 1322 with a size of N×N / 2.

[0180] In embodiments of this disclosure, the image decoding apparatus 100 can determine a third encoding unit (e.g., 1304, 1314, 1324, etc.) by dividing the width and height of a second encoding unit 1302 with a size of N×N. Specifically, the image decoding apparatus 100 can divide the second encoding unit 1302 vertically and horizontally, thereby determining a third encoding unit 1304 with a size of N / 2×N / 2, a third encoding unit 1314 with a size of N / 4×N / 2, or a third encoding unit 1324 with a size of N / 2×N / 4.

[0181] In embodiments of this disclosure, the image decoding apparatus 100 can determine a third encoding unit (e.g., 1304, 1314, 1324, etc.) by dividing the width and height of a second encoding unit 1312 with a size of N / 2×N. Specifically, the image decoding apparatus 100 can horizontally divide the second encoding unit 1312 to determine a third encoding unit 1304 with a size of N / 2×N / 2 or a third encoding unit 1324 with a size of N / 2×N / 4, and can vertically and horizontally divide the second encoding unit 1312 to determine a third encoding unit 1314 with a size of N / 4×N / 2.

[0182] In embodiments of this disclosure, the image decoding apparatus 100 can determine a third encoding unit (e.g., 1304, 1314, 1324, etc.) by dividing the width and height of a second encoding unit 1322 with a size of N×N / 2. Specifically, the image decoding apparatus 100 can vertically divide the second encoding unit 1322 to determine a third encoding unit 1304 with a size of N / 2×N / 2 or a third encoding unit 1314 with a size of N / 4×N / 2, or it can vertically and horizontally divide the second encoding unit 1322 to determine a third encoding unit 1324 with a size of N / 2×N / 4.

[0183] In embodiments of this disclosure, the image decoding apparatus 100 may horizontally or vertically divide square-shaped encoding units (e.g., 1300, 1302, or 1304). For example, a first encoding unit 1300 with a size of 2N×2N may be vertically divided to determine a first encoding unit 1310 with a size of N×2N, or a first encoding unit 1300 with a size of 2N×2N may be horizontally divided to determine a first encoding unit 1320 with a size of 2N×N. In embodiments of this disclosure, when the depth is determined based on the length of the longest side of the encoding unit, the depth of the encoding unit determined by horizontally or vertically dividing the first encoding unit 1300 with a size of 2N×2N may be equal to the depth of the first encoding unit 1300.

[0184] In embodiments of this disclosure, the width and height of the third encoding unit 1314 or 1324 may correspond to one-quarter of the width and height of the first encoding unit 1310 or 1320. When the depth of the first encoding unit 1310 or 1320 is D, the depth of the second encoding unit 1312 or 1322, which has half the width and height of the first encoding unit 1310 or 1320, may be D+1, and the depth of the third encoding unit 1314 or 1324, which has one-quarter the width and height of the first encoding unit 1310 or 1320, may be D+2.

[0185] Figure 14 The diagram illustrates a depth that can be determined based on the form and size of the coding unit and an index (partial index (PID)) used to classify the coding units according to embodiments of the present disclosure.

[0186] In embodiments of this disclosure, the image decoding apparatus 100 can determine second encoding units of various shapes by dividing a first encoding unit 1400 into square shapes. (Refer to...) Figure 14 The image decoding apparatus 100 can determine the second coding units 1402a, 1402b, 1404a, 1404b, 1406a, 1406b, 1406c, and 1406d by dividing the first coding unit 1400 in at least one direction, either vertical or horizontal, according to partitioning pattern information. In other words, the image decoding apparatus 100 can determine the second coding units 1402a, 1402b, 1404a, 1404b, 1406a, 1406b, 1406c, and 1406d based on the partitioning pattern information for the first coding unit 1400.

[0187] In embodiments of this disclosure, the depths of the second coding units 1402a, 1402b, 1404a, 1404b, 1406a, 1406b, 1406c, and 1406d, determined based on the partitioning pattern information of the square-shaped first coding unit 1400, can be determined based on the length of the long side of the second coding unit. For example, since the length of the side of the square-shaped first coding unit 1400 is equal to the length of the long side of the non-square-shaped second coding units 1402a, 1402b, 1404a, and 1404b, the depths of the first coding unit 1400 and the non-square-shaped second coding units 1402a, 1402b, 1404a, and 1404b are considered to be the same as D. On the other hand, when the image decoding device 100 divides the first encoding unit 1400 into four square-shaped second encoding units 1406a, 1406b, 1406c and 1406d based on the partition pattern information, the side length of the square-shaped second encoding units (1406a, 1406b, 1406c or 1406d) is half the side length of the first encoding unit 1400. Therefore, the depth of the second encoding units 1406a, 1406b, 1406c or 1406d can be D+1, which is one level lower than the depth D of the first encoding unit 1400.

[0188] In embodiments of this disclosure, the image decoding device 100 can horizontally divide a first encoding unit 1410, whose height is greater than its width, into a plurality of second encoding units 1412a, 1412b, 1414a, 1414b, and 1414c according to partitioning pattern information. In embodiments of this disclosure, the image decoding device 100 can vertically divide a first encoding unit 1420, whose width is greater than its height, into a plurality of second encoding units 1422a, 1422b, 1424a, 1424b, and 1424c according to partitioning pattern information.

[0189] In embodiments of this disclosure, the depth of a second coding unit 1412a, 1412b, 1414a, 1414b, 1414c, 1422a, 1422b, 1424a, 1424b, or 1424c, determined based on the partitioning pattern information of a non-square-shaped first coding unit 1410 or 1420, can be determined based on the length of its longer side. For example, since the length of one side of a square-shaped second coding unit 1412a or 1412b is half the length of one side of a non-square-shaped first coding unit 1410 whose height is greater than its width, the depth of the square-shaped second coding unit 1412a or 1412b is D+1, which is one level lower than the depth D of the non-square-shaped first coding unit 1410.

[0190] Furthermore, the image decoding device 100 can divide the non-square-shaped first coding unit 1410 into an odd number of second coding units 1414a, 1414b, and 1414c based on partitioning pattern information. The odd number of second coding units 1414a, 1414b, and 1414c may include the non-square-shaped second coding units 1414a and 1414c and the square-shaped second coding unit 1414b. In this case, since the length of the longer side of the non-square second coding unit 1414a or 1414c and the length of one side of the square second coding unit 1414b are half the length of one side of the first coding unit 1410, the depth of the second coding units 1414a, 1414b, or 1414c can be D+1, which is one level lower than the depth D of the first coding unit 1410. The image decoding device 100 can determine the depth of the coding unit associated with the non-square first coding unit 1420 whose width is greater than its height in a manner corresponding to the method described above for determining the depth of the coding unit associated with the first coding unit 1410.

[0191] In embodiments of this disclosure, when determining the index (PID) for distinguishing the coding units of a partition, if an odd number of partitions have different coding units, the image decoding apparatus 100 may determine the index based on the size ratio between the coding units. (See also...) Figure 14 The middle coding unit 1414b, located among the odd-numbered coding units 1414a, 1414b, and 1414c, may have the same width as the other coding units 1414a and 1414c, and twice the height of the other coding units 1414a and 1414c. In this case, the middle coding unit 1414b may contain the other two coding units 1414a and 1414c. Therefore, when the index (PID) of the middle coding unit 1414b in the scanning order is "1", the coding unit 1414c in the subsequent position may have an index 3 increased by 2. That is, there is a discontinuity in the index values. In embodiments of this disclosure, the image decoding apparatus 100 may determine whether the odd-numbered coding units have the same size based on whether there is a discontinuity in the index used to distinguish between the divided coding units.

[0192] In embodiments of this disclosure, the image decoding apparatus 100 can determine whether the current coding unit is divided into a specific partitioning pattern based on the value of the index used to distinguish among multiple coding units determined by dividing the current coding unit. (Refer to...) Figure 14The image decoding apparatus 100 can determine an even number of coding units 1412a and 1412b or an odd number of coding units 1414a, 1414b, and 1414c by dividing a first coding unit 1410 into shapes such as rectangles whose height is longer than their width. To distinguish the multiple coding units from each other, the image decoding apparatus 100 can use an index (PID) indicating each coding unit. In embodiments of this disclosure, the PID can be obtained from a sample point at a specific location within each coding unit (e.g., the top-left sample point).

[0193] In embodiments of this disclosure, indexes are used to distinguish coding units, and the image decoding apparatus 100 can determine a coding unit at a specific location from the coding units determined by partitioning. In embodiments of this disclosure, when the partitioning pattern information for a first coding unit 1410 of a rectangular shape with a height greater than its width indicates partitioning into three coding units, the image decoding apparatus 100 can partition the first coding unit 1410 into three coding units 1414a, 1414b, and 1414c. The image decoding apparatus 100 can assign an index to each of the three coding units 1414a, 1414b, and 1414c. The image decoding apparatus 100 can compare the indices of the coding units to determine the middle coding unit among an odd number of partitioned coding units. Based on the index of the coding unit, the image decoding apparatus 100 can determine the coding unit 1414b, which has an index corresponding to the center value in the index, as the middle coding unit located in the coding units determined by partitioning the first coding unit 1410. In embodiments of this disclosure, when determining an index for distinguishing between divided coding units, the image decoding apparatus 100 may determine the index based on the size ratio between coding units when the coding units have different sizes. (See also...) Figure 14The coding unit 1414b generated by dividing the first coding unit 1410 may have the same width as the other coding units 1414a and 1414c, and twice the height of the other coding units 1414a and 1414c. In this case, when the index (PID) of the middle coding unit 1414b is "1", the coding unit 1414c at a subsequent position may have an index 3 increased by 2. In this case where the index increases uniformly and then changes incrementally, the image decoding device 100 may determine to perform the division of a plurality of coding units including coding units with different sizes than the other coding units. In embodiments of this disclosure, when the partitioning pattern information indicates a division into an odd number of coding units, the image decoding device 100 may divide the current coding unit in a form such that the coding unit at a specific position in the odd number of coding units (e.g., the middle coding unit) has a different size than the other coding units. In this case, the image decoding device 100 may use the index (PID) of the coding unit to determine the center coding unit with a different size. The index and size or position of the coding unit in the specific location to be determined are specified to describe the embodiment, and should not be construed as limiting thereto. Rather, this disclosure should be interpreted as using various indexes and various positions and sizes of coding units.

[0194] In embodiments of this disclosure, the image decoding apparatus 100 may use a specific data unit of the encoding unit, and recursively divide the data starting from that specific data unit.

[0195] Figure 15 The illustration shows how multiple coding units are determined based on multiple specific data units included in the image, according to an embodiment of the present disclosure.

[0196] In embodiments of this disclosure, a specific data unit can be defined as a data unit from which coding units are recursively divided based on partitioning pattern information. That is, it may correspond to the coding unit with the highest depth used when determining the multiple coding units for dividing the current frame. For ease of explanation, the specific data unit will now be referred to as a reference data unit.

[0197] In embodiments of this disclosure, the reference data unit may represent a specific size and form. In embodiments of this disclosure, the reference data unit may include M×N samples. Here, M and N may be the same and may be integers represented by powers of 2. Specifically, the reference data unit may represent a square or non-square form and may subsequently be divided into an integer number of encoding units.

[0198] In embodiments of this disclosure, the image decoding device 100 can divide the current frame into multiple reference data units. In embodiments of this disclosure, the image decoding device 100 can divide the current frame into multiple reference data units based on partitioning pattern information for each reference data unit. The process of dividing the reference data units can correspond to a partitioning process using a quadtree structure.

[0199] In embodiments of this disclosure, the image decoding apparatus 100 may predetermine the possible minimum size of reference data units included in the current frame. Therefore, the image decoding apparatus 100 may determine reference data units of various sizes equal to or greater than the minimum size, and determine at least one encoding unit based on the determined reference data units using partitioned pattern information.

[0200] Reference Figure 15 The image decoding apparatus 100 may use a square-shaped reference coding unit 1500 or a non-square-shaped reference coding unit 1502. In embodiments of this disclosure, the form and size of the reference coding unit may be determined based on various data units (e.g., sequences, frames, stripes, strip segments, parallel blocks, parallel block groups, maximum coding units, etc.) that may include at least one reference coding unit.

[0201] In embodiments of this disclosure, the bitstream acquirer 110 of the image decoding apparatus 100 can obtain, for each of various data units, at least one of information about the form of a reference coding unit and information about the size of the reference coding unit from the bitstream. (This has been verified...) Figure 3 The partitioning process of the current coding unit 300 describes the process of determining at least one coding unit included in the square reference coding unit 1500, which has been... Figure 4 The process of partitioning the current coding unit 400 or 450 describes the process of determining at least one coding unit included in the non-square reference coding unit 1502, and therefore its detailed description will not be repeated.

[0202] In embodiments of this disclosure, the image decoding apparatus 100 may use an index to identify the size and form of a reference coding unit to determine the size and form of the reference coding unit based on some predetermined data units based on specific conditions. Specifically, the bitstream acquirer 110 may obtain only an index from the bitstream to identify the size and form of the reference coding unit for each slice, slice fragment, parallel block, parallel block group, maximum coding unit, etc., as data units (e.g., data units with a size equal to or smaller than the slice) that satisfy specific conditions among various data units (e.g., sequences, frames, slices, slice fragments, parallel blocks, parallel block groups, maximum coding units, etc.). The image decoding apparatus 100 may determine the size and form of the reference data unit for each data unit that satisfies specific conditions by using the index. Using the bitstream may be inefficient when obtaining and using information about the form and size of the reference coding unit from the bitstream for each relatively small data unit; therefore, it is preferable to obtain and use only the index, rather than directly obtaining information about the form and size of the reference coding unit. In this case, at least one of the size and shape of the reference coding unit corresponding to the index indicating the size and shape of the reference coding unit may be predetermined. The image decoding device 100 can determine at least one of the dimensions and shapes of the reference coding unit included in the data unit that forms the basis for obtaining the index by selecting at least one of the dimensions and shapes of the predetermined reference coding unit according to the index.

[0203] In embodiments of this disclosure, the image decoding apparatus 100 may use at least one reference coding unit included in the maximum coding unit 1510. Specifically, the maximum coding unit 1510 for dividing the image may include at least one reference coding unit, and the coding unit may be determined through a recursive partitioning process for each reference coding unit. In embodiments of this disclosure, at least one of the width and height of the maximum coding unit 1510 may correspond to an integer multiple of at least one of the width and height of the reference coding unit. In embodiments of this disclosure, the size of the reference coding unit may be obtained by partitioning the maximum coding unit 1510 n times according to a quadtree structure. Specifically, the image decoding apparatus 100 may determine the reference coding unit by partitioning the maximum coding unit 1510 n times according to a quadtree structure, and in embodiments of this disclosure, the reference coding unit may be partitioned based on at least one of block-form information and partition-form pattern information.

[0204] In embodiments of this disclosure, the image decoding apparatus 100 may obtain and use block format information indicating the shape of the current coding unit and partition format pattern information indicating the method of dividing the current coding unit from the bitstream. The partition format pattern information may be included in the bitstream associated with various data units. For example, the image decoding apparatus 100 may use partition format pattern information included in sequence parameter sets, picture parameter sets, video parameter sets, strip headers, strip segment headers, parallel block headers, or parallel block group headers. Furthermore, the image decoding apparatus 100 may obtain and use syntax elements corresponding to the block format information or partition format pattern information from the bitstream for each reference coding unit or maximum coding unit.

[0205] The method for determining the partitioning rules will now be described in detail based on embodiments of the present disclosure.

[0206] Image decoding device 100 can determine partitioning rules for an image. Partitioning rules can be predetermined between image decoding device 100 and image encoding device 200. Image decoding device 100 can determine partitioning rules for an image based on information obtained from the bitstream. Image decoding device 100 can determine partitioning rules based on information obtained from at least one of sequence parameter set, picture parameter set, video parameter set, strip header, strip segment header, parallel block header, or parallel block group header. Image decoding device 100 can determine different partitioning rules based on frames, stripes, parallel blocks, time layers, maximum coding units, or coding units.

[0207] Image decoding apparatus 100 may determine partitioning rules based on the block format of the coding units. The block format may include the size, shape, aspect ratio, or orientation of the coding units. Image encoding apparatus 200 and image decoding apparatus 100 may agree in advance to determine partitioning rules based on the block format of the coding units. However, the invention is not limited thereto. Image decoding apparatus 100 may also determine partitioning rules based on information obtained from the bitstream received from image encoding apparatus 200.

[0208] The shape of the encoding unit may include a square and a non-square shape. When the width and height of the encoding unit are equal in length, the image decoding device 100 may determine the shape of the encoding unit as a square. Furthermore, when the width and height of the encoding unit are not equal in length, the image decoding device 100 may determine the shape of the encoding unit as a non-square shape.

[0209] The size of the encoding unit can include various sizes: 4×4, 8×4, 4×8, 8×8, 16×4, 16×8, ..., 256×256. The size of the encoding unit can be classified according to the length of its long side, the length of its short side, or its area. The image decoding apparatus 100 can apply the same partitioning rules to encoding units classified into the same group. For example, the image decoding apparatus 100 can classify encoding units with the same long side length as having the same size. The image decoding apparatus 100 can also apply the same partitioning rules to encoding units with the same long side length.

[0210] The aspect ratio of the coding unit may include 1:2, 2:1, 1:4, 4:1, 1:8, 8:1, 1:16, 16:1, 32:1, or 1:32. The orientation of the coding unit may include horizontal and vertical directions. The horizontal orientation may correspond to the case where the width of the coding unit is longer than its height. The vertical orientation may correspond to the case where the width of the coding unit is shorter than its height.

[0211] The image decoding apparatus 100 can adaptively determine partitioning rules based on the size of the coding unit. The image decoding apparatus 100 can determine different permitted partitioning patterns based on the size of the coding unit. For example, the image decoding apparatus 100 can determine whether partitioning is permitted based on the size of the coding unit. The image decoding apparatus 100 can determine the partitioning direction based on the size of the coding unit. The image decoding apparatus 100 can determine the permitted partitioning type based on the size of the coding unit.

[0212] The partitioning rule can be determined based on the size of the coding unit, according to a pre-determined partitioning rule between the image encoding device 200 and the image decoding device 100. Furthermore, the image decoding device 100 can determine the partitioning rule based on information obtained from the bitstream.

[0213] The image decoding apparatus 100 can adaptively determine partitioning rules based on the position of the encoding units. The image decoding apparatus 100 can adaptively determine partitioning rules based on the position occupied by the encoding units in the image.

[0214] Furthermore, the image decoding apparatus 100 can determine partitioning rules to prevent coding units generated from different partitioning paths from having the same block form. However, it is not limited to this, and coding units generated from different partitioning paths can have the same block form. Coding units generated from different partitioning paths can have different decoding processing orders. (Already combined...) Figure 12 The decoding process order is described, so its detailed description will be omitted.

[0215] Figure 16 The present disclosure illustrates encoding units that can be determined for each frame when the encoding units can be divided into different combinations for each frame.

[0216] Reference Figure 16 The image decoding apparatus 100 can determine different combinations of partitioning formats into which the encoding units can be divided for each frame. For example, the image decoding apparatus 100 can decode an image by using a frame 1600, which can be divided into four encoding units, a frame 1610, which can be divided into two or four encoding units, and a frame 1620, which can be divided into two, three, or four encoding units, included in at least one frame in the image. To divide the frame 1600 into multiple encoding units, the image decoding apparatus 100 can use only partitioning format information indicating that the encoding units are divided into four square shapes. To divide the frame 1610, the image decoding apparatus 100 can use only partitioning format information indicating that the frame is divided into two or four encoding units. To divide the frame 1620, the image decoding apparatus 100 can use only partitioning format information indicating that the frame is divided into two, three, or four encoding units. Since the above combinations of partitioning formats are merely examples for describing the operation of the image decoding apparatus 100, they should not be construed as limiting to these examples, and this disclosure should be interpreted as using various combinations of partitioning formats for each data unit.

[0217] In embodiments of this disclosure, the bitstream acquirer 110 of the image decoding apparatus 100 can acquire a bitstream including indices that indicate combinations of partitioning information for each data unit (e.g., sequence, picture, slice, strip segment, parallel block, or parallel block group). For example, the bitstream acquirer 110 can acquire indices indicating combinations of partitioning information from a sequence parameter set, picture parameter set, strip header, parallel block header, or parallel block group header. The image decoding apparatus 100 can determine combinations of partitioning formats into which the encoded unit can be divided for each given data unit by using the acquired indices; therefore, different combinations of partitioning formats can be used for each data unit.

[0218] Figure 17 Various forms of coding units, which can be determined based on partition-form pattern information that can be represented in binary code, are shown according to embodiments of the present disclosure.

[0219] In embodiments of this disclosure, the image decoding apparatus 100 can divide encoding units into various forms using block-form information and partition-form pattern information obtained through the bitstream acquirer 110. Various forms in which encoding units can be divided are possible, including those described above in the foregoing embodiments.

[0220] Reference Figure 17The image decoding device 100 can divide square-shaped coding units in at least one direction, either horizontal or vertical, based on partition pattern information, and divide non-square-shaped coding units along either the horizontal or vertical direction.

[0221] In embodiments of this disclosure, when the image decoding apparatus 100 is able to divide a square-shaped coding unit horizontally and vertically into four square-shaped coding units, there may be four partitioning patterns that can be indicated by the partitioning pattern information of the square-shaped coding units. In embodiments of this disclosure, the partitioning pattern information can be represented by a two-bit binary code, and each partitioning pattern can be assigned a binary code. For example, when the coding unit is not divided, the partitioning pattern information can be represented as (00)b; when the coding unit is divided horizontally and vertically, the partitioning pattern information can be represented as (01)b; when the coding unit is divided horizontally, the partitioning pattern information can be represented as (10)b; when the coding unit is divided vertically, the partitioning pattern information can be represented as (11)b.

[0222] In embodiments of this disclosure, when the image decoding apparatus 100 horizontally or vertically divides non-square-shaped coding units, the type of partitioning pattern, which can be indicated by partitioning pattern information, can be determined based on the number of coding units generated by the division. (Refer to...) Figure 17 In embodiments of this disclosure, the image decoding apparatus 100 may divide a non-square-shaped coding unit into up to three units. The image decoding apparatus 100 may divide the coding unit into two coding units, in which case the partitioned pattern information may be represented by (10)b. The image decoding apparatus 100 may divide the coding unit into three coding units, in which case the partitioned pattern information may be represented by (11)b. The image decoding apparatus 100 may determine not to divide the coding unit, in which case the partitioned pattern information may be represented by (0)b. Specifically, the image decoding apparatus 100 may use variable-length coding (VLC) instead of fixed-length coding (FLC) to use binary codes representing the partitioned pattern information.

[0223] In embodiments of this disclosure, reference is made to Figure 17 The binary code indicating that the coding unit will not divide the partition pattern information can be represented as (0)b. When the binary code indicating that the coding unit will not divide the partition pattern information is set to (00)b, although there is actually no partition pattern information set to (01)b, the entire binary code of the 2-bit partition pattern information is required. On the other hand, as... Figure 17As shown, when using three partitioning schemes for non-square-shaped coding units, the image decoding apparatus 100 can efficiently use the bitstream by determining that the coding unit will not be partitioned simply by using a 1-bit binary code (0)b as partitioning scheme pattern information. However, the partitioning scheme of the non-square-shaped coding unit indicated by the partitioning scheme pattern information should not be interpreted as limited to the following: Figure 17 The three forms shown should be interpreted as having various other forms.

[0224] Figure 18 Other forms of coding units that can be determined based on partition-form pattern information that can be represented in binary code, according to embodiments of the present disclosure, are shown.

[0225] Reference Figure 18 The image decoding apparatus 100 can divide square-shaped coding units in the horizontal or vertical direction, and divide non-square-shaped coding units in the horizontal or vertical direction, based on partition pattern information. In other words, partition pattern information can indicate that square-shaped coding units will be divided in one direction. In this case, the binary code of the partition pattern information indicating that square-shaped coding units will not be divided can be represented as (0)b. When the binary code of the partition pattern information indicating that the coding units will not be divided is set to (00)b, although there is no partition pattern information actually set to (01)b, the entire binary code of the 2-bit partition pattern information is required. On the other hand, as Figure 18 As shown, when using three partitioning schemes of square-shaped coding units, the image decoding apparatus 100 can efficiently use the bitstream by determining that the coding unit will not be partitioned simply by using a 1-bit binary code (0)b as partitioning scheme pattern information. However, the partitioning scheme of the square-shaped coding unit indicated by the partitioning scheme pattern information should not be interpreted as limited to the following: Figure 18 The three forms shown should be interpreted as having various other forms.

[0226] In embodiments of this disclosure, block-form information or partition-form pattern information can be represented using binary codes, and this information can be directly generated as a bitstream. Furthermore, the block-form information or partition-form pattern information that can be represented by binary codes may not be directly generated into the bitstream, but can be used as binary code input in Context Adaptive Binary Arithmetic Coding (CABAC).

[0227] In embodiments of this disclosure, the process by which the image decoding apparatus 100 obtains block-form information or partition-form pattern information via CABAC will be described. A bitstream including binary codes for the syntax can be obtained by a bitstream acquirer 110. The image decoding apparatus 100 can detect syntax elements representing block-form information or partition-form pattern information by debinarizing the binary bit strings included in the obtained bitstream. In embodiments of this disclosure, the image decoding apparatus 100 can obtain a set of binary bit strings corresponding to the syntax elements to be decoded, and decode each binary bit using probability information. The image decoding apparatus 100 can repeat this operation until the binary bit string composed of the decoded binary bits becomes the same as one of the previously obtained binary bit strings. The image decoding apparatus 100 can determine the syntax elements by performing debinarization of the binary bit strings.

[0228] In embodiments of this disclosure, the image decoding apparatus 100 can determine the syntax of the binary bit string by performing decoding processing of adaptive binary arithmetic encoding, and the image decoding apparatus 100 can update the probability model of the binary bits obtained by the bit stream acquirer 110. (Refer to...) Figure 17 In embodiments of this disclosure, the bitstream acquirer 110 of the image decoding apparatus 100 can acquire a bitstream representing binary codes that represent partitioned pattern information. Using the acquired binary codes, which are either 1 bit or 2 bits in size, the image decoding apparatus 100 can determine the syntax of the partitioned pattern information. To determine the syntax of the partitioned pattern information, the image decoding apparatus 100 can update the probability of each bit of the 2-bit binary code. Specifically, based on whether the first binary bit of the 2-bit binary code has a value of "0" or "1", the image decoding apparatus 100 can update the probability of having a value of "0" or "1" when decoding the next binary bit.

[0229] In embodiments of this disclosure, the image decoding apparatus 100 may update the probability of the binary bits used in the process of decoding the binary bit string of the grammar during the process of determining the grammar, and the image decoding apparatus 100 may determine that a certain bit of the binary bit string has the same probability without updating the probability.

[0230] Reference Figure 17In the process of determining the syntax using a binary bit string of partition form pattern information representing non-square-shaped coding units, when non-square-shaped coding units are not divided, the image decoding device 100 can determine the syntax of the partition form pattern information by using a binary bit with a value of "0". Specifically, when the block form information indicates that the current coding unit has a non-square shape, the first binary bit of the binary bit string of the partition form pattern information can be "0" when the non-square-shaped coding unit is not divided, and can be "1" when the non-square-shaped coding unit is divided into two or three coding units. Therefore, the probability that the first binary bit of the binary bit string of the partition form pattern information for non-square-shaped coding units is "0" can be 1 / 3, and the probability that the first binary bit is "1" can be 2 / 3. As described above, since the partition form pattern information indicating that non-square-shaped coding units are not divided can be represented by only a 1-bit binary bit string with a value of "0", the image decoding device 100 can determine the syntax of the partition form pattern information by determining whether the second binary bit is "0" or "1" only when the first binary bit of the partition form pattern information is "1". In embodiments of this disclosure, when the first binary bit of the partition format pattern information is "1", the image decoding device 100 may consider the probability that the second binary bit is "0" or "1" to be the same, and decode the binary bit.

[0231] In embodiments of this disclosure, the image decoding apparatus 100 may use various probabilities for each bit in the process of determining the bits of the binary bit string for partitioned pattern information. In embodiments of this disclosure, the image decoding apparatus 100 may determine different probabilities for the bits of partitioned pattern information based on the orientation of a non-square block. In embodiments of this disclosure, the image decoding apparatus 100 may determine different probabilities for the bits of partitioned pattern information based on the area of ​​the current encoding unit or the length of its long side. In embodiments of this disclosure, the image decoding apparatus 100 may determine different probabilities for the bits of partitioned pattern information based on at least one of the shape of the current encoding unit and the length of its long side.

[0232] In embodiments of this disclosure, the image decoding apparatus 100 can determine that the binary bits of partitioned pattern information about coding units of at least a certain size have the same probability. For example, it can be determined that coding units with a length of at least 64 sample points about the long side of the coding unit have the same probability of having the same binary bits of partitioned pattern information.

[0233] In embodiments of this disclosure, the image decoding apparatus 100 may determine the initial probability of the bits constituting the binary bit string of partitioned pattern information based on the stripe type (e.g., I stripe, P stripe, or B stripe).

[0234] Figure 19 This is a block diagram of an image encoding and decoding system that performs loop filtering.

[0235] The encoding stage 1910 of the image encoding and decoding system 1900 transmits the encoded bitstream of the image, and the decoding stage 1950 outputs the reconstructed image by receiving the bitstream and decoding it. The encoding stage 1910 may have a configuration similar to that of the image encoding apparatus 200, which will be described later, and the decoding stage 1950 may have a configuration similar to that of the image decoding apparatus 100.

[0236] In the coding stage 1910, the predictive encoder 1915 outputs predicted data through inter-frame and intra-frame prediction, and the transformer and quantizer 1920 outputs quantized transform coefficients of the residual data between the predicted data and the current input image. For example, transform coefficients can be generated using a transform kernel including at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), or Wavelet Transform. The residual data can be transformed to compress information. For example, the residual data can undergo a transform and therefore can be represented using a small number of frequencies. The entropy encoder 1925 encodes and transforms the quantized transform coefficients and outputs the results in a bitstream. The quantized transform coefficients can be dequantized and inversely transformed by 1930 to recover spatial domain data. The dequantizer and inverse transform 1930 can perform dequantization on the quantized transform coefficients and determine the residual data by applying a transform kernel. The recovered spatial domain data is then passed through a deblocking filter 1935 and a loop filter 1940 to be output as a reconstructed image. The reconstructed image can be generated by predictive encoder 1915 and then used as a reference image for subsequent images.

[0237] The encoded image data in the bitstream received by the decoder stage 1950 is reconstructed into spatial domain residual data by passing through the entropy decoder 1955, dequantizer, and inverse transformer 1960. The predicted data and residual data output from the predictor decoder 1975 can be combined into spatial image data, and the deblocking filter 1965 and loop filter 1970 can filter the spatial image data and output a reconstructed image of the current original image. The reconstructed image can be used by the predictor decoder 1975 as a reference image for subsequent original images.

[0238] The loop filter 1940 of the encoding stage 1910 performs loop filtering by using input filter information based on user input or system settings. The filter information used by the loop filter 1940 is output to the entropy encoder 1925 and transmitted to the decoding stage 1950 along with the encoded image data. The loop filter 1970 of the decoding stage 1950 can perform loop filtering based on the filter information input from the decoding stage 1950.

[0239] Figure 20 This is a block diagram illustrating the configuration of an image decoding apparatus according to an embodiment of the present disclosure.

[0240] Reference Figure 20 The image decoding device 2000 may include an acquirer 2010 and a predictive decoder 2020.

[0241] In embodiments of this disclosure, the acquirer 2010 and the predictive decoder 2020 may be implemented by at least one processor. In embodiments of this disclosure, the image decoding apparatus 2000 may include a memory for storing at least one of the instructions or input / output data of the acquirer 2010 and the predictive decoder 2020. The acquirer 2010 and the predictive decoder 2020 may operate according to the instructions stored in the memory. In embodiments of this disclosure, the image decoding apparatus 2000 may include a memory controller for controlling the input / output of data to / from the memory.

[0242] In embodiments of this disclosure, the acquirer 2010 may correspond to Figure 19 The entropy decoder 1955 is shown. In embodiments of this disclosure, the prediction decoder 2020 may correspond to... Figure 19 The predictive decoder shown is from 1975.

[0243] The acquirer 2010 can acquire a bitstream generated as a result of image encoding. The bitstream may include the result of encoding the current block. In embodiments of this disclosure, the acquirer 2010 can receive the bitstream from an image encoding device via a network. In embodiments of this disclosure, the acquirer 2010 can acquire the bitstream from a data storage medium, including at least one of magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; or magneto-optical media such as floppy disks.

[0244] The acquirer 2010 can obtain syntax elements for image decoding from the bitstream. Values ​​corresponding to the syntax elements can be included in the bitstream according to the hierarchical structure of the image. In embodiments of this disclosure, the acquirer 2010 can obtain the syntax elements by entropy decoding of the binary bits included in the bitstream.

[0245] In embodiments of this disclosure, the bitstream may include information about the prediction mode of the current block in the current image. The current block may include at least one of a maximum coding unit, coding unit, transform unit, or prediction unit partitioned from the current image. In embodiments of this disclosure, the prediction mode of the current block may include at least one of an intra-frame mode, an inter-frame mode, a combined mode, a geometric partitioning mode, a block copy mode, or a template matching prediction mode.

[0246] In embodiments of this disclosure, intra-frame modes may include non-directional intra-planar mode (intra_planar) 0, non-directional intra-DC mode (intra_DC) 1, intra-angular mode (intra_angular) 2 to 66, and intra-wide angular mode (intra_wide_angular) -14 to -1 and 67 to 80. In embodiments of this disclosure, the intra-planar mode may refer to a mode used to determine predicted samples based on a weighted average of the distances to the left, top, bottom left, and top right reference samples of the current block. In embodiments of this disclosure, the intra-DC mode may refer to a mode used to determine the average value of reference samples as predicted samples. In embodiments of this disclosure, in the intra-angular mode, the position of reference samples can be identified by considering the direction indicated by the intra-angular mode to generate predicted samples for samples in the current block. For example, in mode 34, a reference sample based on the sample in the current block being located in the top-left 45-degree direction can be identified. The intra-wide angular mode can be used to identify reference samples for samples in a non-square current block. In embodiments of this disclosure, the predictive decoder 2020 can determine one of the intra-wide orientation modes as the intra-prediction mode for a non-square current block. In embodiments of this disclosure, the predictive decoder 2020 can differently set the number and type of intra-prediction modes available in the intra-mode.

[0247] In embodiments of this disclosure, the prediction decoder 2020 can determine the intra-prediction mode based on the most probable mode (MPM). The prediction decoder 2020 can determine whether an MPM is used. The acquirer 2010 can obtain information from the bitstream regarding whether an MPM is used. The prediction decoder 2020 can determine a candidate mode list. The prediction decoder 2020 can determine the candidate mode list based on the intra-prediction modes of the blocks above and to the left of the current block. The prediction decoder 2020 can determine one of the candidate modes in the list as the intra-prediction mode for the current block. The prediction decoder 2020 can obtain information from the bitstream indicating the intra-prediction mode for the current block in the candidate mode list.

[0248] In embodiments of this disclosure, the prediction decoder 2020 can determine an intra-prediction mode based on a template. The prediction decoder 2020 can determine a template for the current block. The template for the current block may include left-side samples, top-left samples, and / or top samples of the current block. The prediction decoder 2020 can determine neighboring samples of the template for the current block. The neighboring samples of the template may include left-side samples, top-left samples, and / or top samples of the template. The prediction decoder 2020 can perform prediction on the template using the neighboring samples as reference samples. The prediction decoder 2020 can determine an intra-prediction mode for a reference block by comparing the predicted template with the reconstructed template of the current block. In embodiments of this disclosure, the intra-prediction mode for the reference block may include an intra-prediction mode for the reference block or a virtual intra-prediction mode. The virtual intra-prediction mode for the reference block may be an intra-prediction mode determined based on at least one of samples (internal samples) in the reference block or neighboring samples of the reference block. The predictive decoder 2020 can determine the intra-mode with the minimum error between the predicted template and the reconstructed template of the current block as the intra-mode for the reference block. In embodiments of this disclosure, the process by which the predictive decoder 2020 determines the intra-predictive mode by performing prediction on the template can be referred to as Template-Based Intra-Mode Derivation (TIMD).

[0249] In embodiments of this disclosure, the prediction decoder 2020 can infer the intra-prediction mode of the current block by using neighboring samples of the current block. The prediction decoder 2020 can determine the gradient by using neighboring samples of the current block. The prediction decoder 2020 can determine multiple 3×3 blocks adjacent to the current block. The prediction decoder 2020 can obtain the horizontal and vertical variations of samples included in each determined 3×3 block. The prediction decoder 2020 can determine the gradient based on the horizontal and vertical variations. The prediction decoder 2020 can determine the horizontal and vertical variations by using a Sobel filter. The prediction decoder 2020 can determine the intra-prediction mode corresponding to the gradient. The prediction decoder 2020 can determine the intra-prediction mode of the current block based on multiple intra-prediction modes determined for multiple 3×3 blocks. In embodiments of this disclosure, the prediction decoder 2020 can determine the most frequently determined intra-prediction mode as the intra-prediction mode of the current block. In embodiments of this disclosure, the prediction decoder 2020 can determine the magnitude based on the horizontal and vertical variations. The predictive decoder 2020 can determine the intra-prediction mode of the current block based on amplitude. The predictive decoder 2020 can determine the weights of the intra-prediction mode corresponding to the gradient as amplitude. For example, the predictive decoder 2020 can increase the weights of the gradient-determined intra-prediction mode as horizontal and vertical variations increase. The predictive decoder 2020 can determine the intra-prediction mode of the current block based on the results reflecting the amplitude-determined weights. In embodiments of this disclosure, the process by which the predictive decoder 2020 determines the intra-prediction mode based on gradients can be referred to as decoder-side intra-mode derivation (DIMD).

[0250] In embodiments of this disclosure, the block copying mode may include an intra-block copying mode. In embodiments, the block copying mode may include an intra-block copying mode. In embodiments, the intra-block copying mode may be a sub-mode of an intra-mode, but is not limited to this, and may indicate a mode separate from the intra-mode. In embodiments, the template matching prediction mode may include a template matching intra-prediction mode. Combination modes may include a combined inter-frame intra-prediction (CIIP) mode, wherein predictions from an intra-mode and predictions from an inter-mode are combined to perform prediction. Geometric partitioning modes may include modes for dividing blocks to include orientation within the blocks. Geometric partitioning modes may perform prediction for each partitioned region of a block using inter-frame prediction or intra-frame prediction.

[0251] The Predictive Decoder 2020 can reconstruct the current block by performing a prediction on the current block based on the prediction pattern used for the current block.

[0252] In embodiments of this disclosure, the acquirer 2010 may obtain information about the prediction mode of the current block from the bit stream. For example, the acquirer 2010 may obtain index information indicating the prediction mode of the current block from the bit stream.

[0253] In embodiments of this disclosure, when the prediction mode of the current block is CIIP mode, the prediction decoder 2020 can combine inter-frame prediction and intra-frame prediction to reconstruct the current block. For example, the prediction decoder 2020 can perform intra-frame prediction according to a planar mode. For example, the prediction decoder 2020 can perform inter-frame prediction by using motion vectors (MV). The prediction decoder 2020 can reconstruct the current block by using a weighted sum of predicted blocks from inter-frame prediction and predicted blocks from intra-frame prediction. The weights can be determined based on whether neighboring blocks of the current block have been intra-frame predicted.

[0254] In embodiments of this disclosure, when the prediction mode of the current block is geometric partitioning, the prediction decoder 2020 can partition the current block and perform prediction. The prediction decoder 2020 can obtain partition angles and partition distances relative to the boundaries where partitioning is performed in the current block. The prediction decoder 2020 can partition the current block based on the partition angles and partition distances. The prediction decoder 2020 can reconstruct the current block by performing inter-frame prediction or intra-frame prediction on each partition region in the current block. The prediction decoder 2020 can (i) perform intra-frame prediction on two partition regions, (ii) perform inter-frame prediction on one region and intra-frame prediction on the other region, or (iii) perform inter-frame prediction on two partition regions.

[0255] In embodiments of this disclosure, when the prediction mode for the current block is block copy mode, the prediction decoder 2020 can reconstruct the current block based on reference blocks included in the current image. In embodiments of this disclosure, when the prediction mode for the current block is block copy mode, the prediction decoder 2020 can determine information about the block vector (BV) of a reference block. In embodiments of this disclosure, the prediction decoder 2020 can determine the prediction block based on a reference block. For example, the prediction decoder 2020 can determine that the prediction block is the same as the reference block or determine the prediction block by filtering the reference block.

[0256] In embodiments of this disclosure, when the prediction mode for the current block is template matching prediction mode, the prediction decoder 2020 can reconstruct the current block using a reference block. The acquirer 2010 can acquire information related to whether to use the template matching prediction mode. The prediction decoder 2020 can determine whether to use the template matching prediction mode based on the acquired information. The reference block can be determined based on at least one of a region included in the current image or a region included in a previously decoded image. In embodiments of this disclosure, the prediction decoder 2020 can determine the reference block based on a cost function. The cost function can include at least one of Sum of Absolute Differences (SAD), Sum of Squared Differences (SSD), Sum of Absolute Transform Differences (SATD), Sum of Squared Errors (SSE), or Mean Removal SAD (MR-SAD). For example, the prediction decoder 2020 can determine the error between templates based on at least one of SAD, SATD, SSE, and MR-SAD. When the cost function is SAD, the prediction decoder 2020 can determine the error based on the sum of the absolute values ​​of the differences between the samples of the template of the current block and the samples of the template of the candidate block. When the cost function is SSD, the prediction decoder 2020 can determine the error based on the sum of the squares of the differences between each sample of the template of the current block and each sample of the template of the candidate block. The cost function may include a function indicating the number of identical samples. The prediction decoder 2020 can determine a block with small error from the candidate blocks as a reference block. The prediction decoder 2020 can determine the prediction block by performing template matching intra-frame prediction on the current image. In this disclosure, the process for determining the reference block of the current block based on the template can be referred to as template matching (TM). In this disclosure, the prediction performed on the current block based on template matching can be referred to as template matching prediction (TMP) or intra-frame template matching prediction.

[0257] The prediction decoder 2020 can generate a reconstructed current block based on the prediction block. In embodiments of this disclosure, the prediction decoder 2020 can determine the prediction block as the reconstructed current block. In one embodiment, the prediction decoder 2020 can combine residual data obtained from the bitstream by the acquirer 2010 with the prediction block to generate the reconstructed current block. The reconstructed current block can be used as a reference block for subsequent blocks.

[0258] In a prediction mode (e.g., intra-frame mode) that uses reference samples included in the current image, assuming continuity between neighboring samples of the current block and samples within the current block, a predicted block for the current block can be generated based on the neighboring samples of the current block according to the prediction mode. In embodiments of this disclosure, the prediction decoder 2020 can perform intra-frame prediction not only using neighboring samples of the current block included in the current image but also using spatial reference samples included in the current image. When using samples reconstructed earlier than the current block, the size of the residual data can be reduced by predicting samples of the current block not only based on samples directly adjacent to the current block but also based on samples far from the current block. In embodiments of this disclosure, the image decoding apparatus 2000 can perform intra-frame prediction based on reference blocks including unreconstructed samples, thereby increasing the range of regions that can be identified as reference blocks. In embodiments of this disclosure, the image decoding apparatus 2000 can improve compression efficiency by improving the efficiency of intra-frame prediction.

[0259] In a prediction mode (e.g., an inter-frame mode) that includes reference samples in a reference image instead of the current image, assuming continuity between the current image and the reference image, a predicted block for the current block can be generated based on a reference block (or reference samples) in the reference image according to the prediction mode. In embodiments of this disclosure, the image decoding apparatus 2000 can improve compression efficiency by increasing the efficiency of intra-frame prediction.

[0260] Image decoding apparatus 2000 can improve prediction accuracy by considering both reference blocks (or reference samples) included in the current image and reference blocks (or reference samples) included in images other than the current image. In embodiments of this disclosure, image decoding apparatus 2000 can improve prediction accuracy by considering both the current image and all images other than the current image.

[0261] The Predictive Decoder 2020 can perform deblocking filtering. The deblocking filter can improve image quality by smoothing the edges between blocks.

[0262] The Predictive Decoder 2020 can perform filtering on samples in the current block that have already undergone past block filtering by using a Sample Adaptive Offset (SAO) filter and / or a Bilateral Filter (BIF). The SAO filter and BIF can improve image quality by reducing the error between the reconstructed image and the original image. The SAO filter and BIF can perform filtering based on samples.

[0263] The Predictive Decoder 2020 can perform filtering using an Adaptive Loop Filter (ALF). ALF improves image quality by reducing the error between the reconstructed image and the original image. ALF can perform filtering on a block-by-block basis.

[0264] Figure 21This is a flowchart of an image decoding method according to an embodiment of the present disclosure.

[0265] In embodiments of this disclosure, the image decoding method may be executed by an image decoding apparatus 2000. For example, the processor of the image decoding apparatus 2000 may execute at least one instruction included in memory to cause the image decoding apparatus 2000 to perform each operation of the image decoding method.

[0266] In operation S2110, the image decoding apparatus 2000 can perform prediction on the current block based on inter-frame prediction. The image decoding apparatus 2000 can obtain a reference block included in the reference image. The image decoding apparatus 2000 can obtain the reference block based on the motion vector of the current block. The image decoding apparatus 2000 can obtain the prediction samples of the current block by performing inter-frame prediction based on the reference block.

[0267] Image decoding apparatus 2000 can obtain one or more reference blocks corresponding to one or more motion vectors of the current block. In embodiments of this disclosure, the current block may be a block included in a bidirectional prediction (B) frame (or B-strip). A block included in a B frame (or B-strip) can undergo inter-frame prediction based on up to two motion vectors and a reference index. When the current block is included in a B frame or B-strip, image decoding apparatus 2000 can obtain two reference blocks corresponding to two motion vectors of the current block. In embodiments of this disclosure, the current block may be a block included in a prediction (P) frame (or P-strip). A block included in a P frame (or P-strip) can undergo inter-frame prediction based on up to one motion vector and a reference index. When the current block is included in a P frame or P-strip, image decoding apparatus 2000 can obtain one reference block corresponding to one motion vector of the current block.

[0268] In operation S2120, the image decoding apparatus 2000 may determine an intra-frame mode for the reference block based on at least one of neighboring samples of the reference block, internal samples of the reference block, or neighboring samples of the current block. In embodiments of this disclosure, the intra-frame mode for the reference block may include an intra-frame mode of the reference block or a virtual intra-frame mode. The virtual intra-frame mode of the reference block may be an intra-frame mode determined based on at least one of samples (or internal samples) in the reference block or neighboring samples of the reference block. The image decoding apparatus 2000 may determine the intra-frame mode for the reference block based on temporal information or spatial information of the current block. Temporal information may include information about images other than the current image including the current block. Spatial information may include information about the current image including the current block.

[0269] In embodiments of this disclosure, the reference block may include a block used for inter-frame prediction of the current block. In embodiments, neighboring samples of the reference block (or the current block) may include adjacent and / or non-adjacent samples of the reference block (or the current block). Neighboring samples of the reference block may include samples to the left, top-left, and top of the reference block.

[0270] In embodiments of this disclosure, the image decoding apparatus 2000 may determine an intra-frame mode for a reference block based on neighboring samples and internal samples of the reference block. The image decoding apparatus 2000 may also determine the intra-frame mode for the reference block based on the result of predicting internal samples of the reference block using neighboring samples. In embodiments of this disclosure, the reference... Figures 22 to 24 as well as Figure 29 and Figure 30 The process by which the image decoding apparatus 2000 determines the intra-frame mode for the reference block based on neighboring samples and internal samples of the reference block is described in detail.

[0271] In embodiments of this disclosure, the image decoding apparatus 2000 may determine an intra-frame mode for a reference block based on internal samples of the reference block. The image decoding apparatus 2000 may determine the intra-frame mode for the reference block based on multiple gradients determined for the internal samples of the reference block. In embodiments of this disclosure, the reference... Figures 25 to 28 as well as Figure 33 and Figure 34 The process by which the image decoding apparatus 2000 determines the intra-frame mode for the reference block based on the internal samples of the reference block is described in detail.

[0272] In embodiments of this disclosure, the image decoding apparatus 2000 can determine the intra-frame mode for a reference block by using neighboring samples of the current sample. The neighboring samples of the current block can be samples included in the template region of the current block. In embodiments of this disclosure, the reference... Figure 33 and Figure 34 The process by which the image decoding apparatus 2000 determines the intra-frame mode for a reference block based on neighboring samples of the current block is described in detail.

[0273] In embodiments of this disclosure, the image decoding apparatus 2000 may determine one of two intra-frame modes obtained from two reference blocks as the intra-frame mode for the reference blocks. The image decoding apparatus 2000 may determine two intra-frame modes from corresponding two reference blocks and designate one of the determined intra-frame modes as the intra-frame mode for the reference blocks. Optionally, the image decoding apparatus 2000 may refer to both reference blocks to determine a virtual intra-frame mode.

[0274] In operation S2130, the image decoding apparatus 2000 can determine the transform kernel by using the intra-frame mode for the reference block. The image decoding apparatus 2000 can determine a transform kernel set including multiple transform kernels corresponding to the intra-frame mode for the reference block. The image decoding apparatus 2000 can determine the type of transform kernel set according to the intra-frame mode for the reference block. For example, the range of the direction index of the intra-frame mode for the reference block may correspond to one of the multiple transform kernel sets. A transform kernel set may include multiple transform kernels. For example, a first transform kernel set may include a first transform kernel and a second transform kernel, and a second transform kernel set may include a third transform kernel and a fourth transform kernel. In embodiments, depending on the type of transform, there are one or more (e.g., N, where N is 1 or greater) transform kernel sets, and each transform kernel set may include one or more (e.g., M, where M is 1 or greater) transform kernels.

[0275] In embodiments of this disclosure, the number of transform cores included in each transform core set can be determined based on the sum of the transform coefficients. For example, when the sum of the transform coefficients is less than a predetermined first value, the number of transform cores can be N1 (e.g., N1=1); when the sum of the transform coefficients is greater than a predetermined first value and less than a predetermined second value, the number of transform cores can be N2 (e.g., N1=4); when the sum of the transform coefficients is greater than a predetermined second value, the number of transform cores can be N3 (e.g., N1=6).

[0276] In embodiments of this disclosure, the image decoding apparatus 2000 can determine a transform kernel set based on the size of the current block and the intra-frame mode of the reference block. The transform kernel set can be predetermined based on the size of the current block. For example, M transform kernel sets can be predetermined for each size of the current block.

[0277] The image decoding apparatus 2000 can select one of a plurality of transform kernels as the transform kernel for the current block. The image decoding apparatus 2000 can determine one of a plurality of transform kernels included in the transform kernel set. In embodiments of this disclosure, the image decoding apparatus 2000 can obtain index information from the bitstream indicating the transform kernel of the current block among the plurality of transform kernels.

[0278] In embodiments of this disclosure, the transform core may involve at least one of a core for a first transform, a core for a second transform, or an inseparable first transform core that combines the first and second transforms.

[0279] In embodiments of this disclosure, the image decoding apparatus 2000 can determine a transform kernel for a first transform by using an intra-frame mode for a reference block. The transform kernel for the first transform may include a Discrete Cosine Transform (DCT) kernel and / or a Discrete Sine Transform (DST) kernel. The first transform can be applied in the vertical and / or horizontal directions of the block. The image decoding apparatus 2000 can obtain information from the bitstream indicating whether it is a vertical or horizontal transform. The first transform may be referred to as Multiple Transform Selection (MTS). The transform kernel for the first transform may be referred to as a core transform kernel.

[0280] In embodiments of this disclosure, the image decoding apparatus 2000 can determine the transform kernel for the second transform by using an intra-frame mode for a reference block. The second transform can be applied only to a portion of the residual signal. For example, the second transform can be performed only on the left or upper left region of the block. The image decoding apparatus 2000 can determine whether to perform the second transform for each block. The second transform is performed without separating the vertical and horizontal directions. The second transform may be referred to as the Low Frequency Inseparable Transform (LFNST).

[0281] In embodiments of this disclosure, the image decoding apparatus 2000 can determine the transform kernel for the inseparable first transform by using an intra-frame mode for a reference block. The inseparable first transform can take the form of a combination of a first transform and a second transform. For example, the transform kernel for the inseparable first transform can be a combination of a transform kernel for the first transform and a transform kernel for the second transform. The inseparable first transform is performed without separating the vertical and horizontal directions.

[0282] In operation S2140, the image decoding device 2000 can perform an inverse transform on the current block using a transform kernel. The image decoding device 2000 can perform an inverse transform by applying a selected transform kernel to the current block. The image decoding device 2000 can obtain a residual signal (or residual data) by performing an inverse transform based on the transform kernel. The image decoding device 2000 can obtain a residual signal by applying a transform kernel to the transform coefficients.

[0283] The image decoding apparatus 2000 can perform an inverse transform by applying a first transform to the residual signal (or residual data). The image decoding apparatus 2000 can generate transform coefficients by applying a transform kernel for a second transform to the residual signal.

[0284] The image decoding apparatus 2000 can perform an inverse transform by applying a second transform to the residual signal. The image decoding apparatus 2000 can generate transform coefficients by applying a transform kernel used for the second transform to the residual signal. The image decoding apparatus 2000 can perform the inverse transform of the second transform and also the inverse transform of the first transform. The inverse transform of the first transform has already been described above, and therefore will not be repeated.

[0285] The image decoding apparatus 2000 can generate a residual signal by applying a transform kernel used for the inseparable first transform to the transform coefficients. The image decoding apparatus 2000 may not perform additional first and / or second transforms on blocks that have already undergone the inseparable first transform.

[0286] When the prediction mode of the current block is inter-frame mode, the image decoding apparatus 2000 can determine the transform kernel set for the current block by determining the intra-frame mode used for the reference block. The image decoding apparatus 2000 can derive the intra-frame mode representing the internal direction for the reference block using the reconstructed reference block, and even when the current block is in inter-frame mode, it can improve coding efficiency (or transform efficiency) by applying the derived intra-frame mode for the reference block to the current block. In other words, by determining the intra-frame mode for the reference block, the image decoding apparatus 2000 can select a suitable transform kernel from various types of transform kernels for the current block, even in inter-frame mode, just as it would in intra-frame mode, thereby improving video compression efficiency. In embodiments of this disclosure, even when the prediction mode of the current block is inter-frame mode, a second transform and an inseparable first transform can be applied, and various transform kernels for the first transform can be applied. By applying the same direction-dependent transform kernel as the second transform, the residual at the block boundary can be effectively compressed. Such advantages are merely examples, and the advantages and objectives of this disclosure are not limited to the foregoing.

[0287] Figure 22 This is a flowchart of an image decoding method according to an embodiment of the present disclosure.

[0288] In embodiments of this disclosure, the image decoding method may be executed by an image decoding apparatus 2000. For example, the processor of the image decoding apparatus 2000 may execute at least one instruction included in memory to cause the image decoding apparatus 2000 to perform each operation of the image decoding method.

[0289] In operation S2210, the image decoding apparatus 2000 can obtain a reference block in the reference image. In embodiments of this disclosure, the reference block can be determined based on the motion vector of the current block. The reference block may include a block used for inter-frame prediction of the current block.

[0290] In operation S2220, the image decoding device 2000 can determine the intra-frame mode for the reference block based on at least one of a plurality of neighboring samples and a plurality of internal samples of the reference block.

[0291] In embodiments of this disclosure, neighboring samples of the reference block may include samples located within a specific distance from the reference block. Neighboring samples of the reference block may include at least one of adjacent and non-adjacent samples of the reference block. Neighboring samples of the reference block may include samples located above the reference block, samples located to the left of the reference block, and / or samples located at the upper left corner of the reference block.

[0292] In embodiments of this disclosure, the image decoding apparatus 2000 can predict internal samples of a reference block using multiple neighboring samples. For example, the image decoding apparatus 2000 can determine predicted samples based on neighboring samples of the reference block, which serve as reference samples, according to multiple intra-frame modes (e.g., intra-frame direction prediction modes 2 to 66 or all available intra-frame prediction modes). The image decoding apparatus 2000 can determine an error by comparing the predicted samples with internal samples of the reference block. The image decoding apparatus 2000 can determine an intra-frame mode for the reference block based on the error. For example, the image decoding apparatus 2000 can determine the intra-frame mode with the smallest error among multiple intra-frame modes as the intra-frame mode for the reference block.

[0293] In operation S2230, the image decoding apparatus 2000 may determine the transform kernel of the current block from one or more transform kernels based on the intra-frame mode of the reference block. In embodiments of this disclosure, operation S2230 may correspond to operation S2130.

[0294] In operation S2240, the image decoding device 2000 can perform an inverse transform on the current block using a transform kernel. In embodiments of this disclosure, operation S2240 may correspond to operation S2140.

[0295] Figure 23 The process of determining the intra-frame mode for a reference block by using neighboring blocks of the reference block, according to an embodiment of the present disclosure, is illustrated.

[0296] In embodiments of this disclosure, the image decoding apparatus 2000 may determine the intra-frame mode for a reference block for the current block 2310 based on time information. The image decoding apparatus 2000 may determine the intra-frame mode for the reference block by using neighboring samples 2330 and internal samples of the reference block 2320.

[0297] The image decoding device 2000 can determine the reference block C for the current block C 2310. pred2320. In embodiments of this disclosure, the current block 2310 may include blocks included in a P-frame (or P-strip). The reference block 2320 may be a block included in a reference image other than the current image, for inter-frame prediction of the current block 2310. The image decoding apparatus 2000 may determine the reference block 2320 included in the reference image (or reference strip) by using the motion vector of the current block 2310. In embodiments of this disclosure, the neighboring samples 2330 of the reference block 2320 may be referred to as temporally neighboring samples.

[0298] The image decoding apparatus 2000 can perform intra-frame prediction using multiple neighboring samples according to one or more intra-frame prediction modes. The image decoding apparatus 2000 can obtain the prediction sample R by performing prediction on the internal samples of the reference block 2320 using the neighboring samples 2330 of the reference block 2320. pred The image decoding apparatus 2000 can obtain the prediction sample of the reference block 2320 by using the neighboring sample 2330 of the reference block 2320 as reference sample according to each of the plurality of intra prediction modes. For example, the image decoding apparatus 2000 can perform prediction for all intra prediction modes, but is not limited thereto, and can perform prediction for a predetermined set of intra prediction modes among the intra prediction modes.

[0299] Image decoding apparatus 2000 can determine the error by comparing predicted samples of reference block 2320 with neighboring samples 2330. In embodiments of this disclosure, image decoding apparatus 2000 can determine the error by using a cost function. For example, image decoding apparatus 2000 can determine the error by using at least one of SAD, SATD, SSE, and MR-SAD. When the cost function is SAD, image decoding apparatus 2000 can determine the error based on the sum of the absolute values ​​of the differences between predicted samples of reference block 2320 and neighboring samples 2330. The error can be replaced by the term "cost". For example, image decoding apparatus 2000 can determine the error between predicted samples of reference block 2320 and neighboring samples 2330 by using Sum of Absolute Differences (SAD).

[0300] The image decoding apparatus 2000 can determine the intra-frame mode for a reference block based on errors. The image decoding apparatus 2000 can determine the intra-frame mode for the reference block based on the error between intra-frame prediction samples and internal samples of the reference block. The image decoding apparatus 2000 can determine the intra-frame prediction mode with the minimum error as the intra-frame mode for the reference block.

[0301] The image decoding apparatus 2000 can determine the transform kernel by using a determined intra-frame mode for a reference block. Here, the transform kernel can refer to a kernel applied to a residual block to perform an inverse transform for at least one of a first transform, a second transform, or an inseparable first transform. The image decoding apparatus 2000 can determine the transform kernel for at least one of the first transform, the second transform, or the inseparable first transform by using a determined intra-frame mode for a reference block.

[0302] Figure 24 The present disclosure illustrates a process for determining an intra-frame mode for a reference block using neighboring samples of the reference block.

[0303] refer to Figure 24 The current block can have multiple reference blocks. In embodiments of this disclosure, the image decoding apparatus 2000 can determine the intra-frame mode of the reference blocks for the current block 2410 based on time information. The image decoding apparatus 2000 can determine the intra-frame mode for the reference blocks by using neighboring samples 2430 or 2450 of the reference blocks 2420 or 2440 and internal samples of the reference blocks 2420 or 2440.

[0304] The image decoding device 2000 can determine the first reference block C for the current block C 2410. pred0 2420 and the second reference block C pred1 2440. In embodiments of this disclosure, the current block 2410 may include blocks included in a B-frame (or B-strip). Reference blocks 2420 or 2440 may be blocks used for inter-frame prediction of the current block 2410 and included in each reference image other than the current image. The image decoding apparatus 2000 can determine the reference block 2420 or 2440 included in the reference image (or reference strip) by using the motion vector of the current block 2410. In embodiments of this disclosure, the neighboring samples 2430 of the reference blocks 2420 or 2440 may be referred to as temporally neighboring samples.

[0305] The image decoding apparatus 2000 can obtain the predicted sample R by performing prediction on the internal samples of the reference block 2420 using the neighboring samples 2430 or 2450 of the reference block 2420 or 2440. pred0 Or R pred1 The image decoding apparatus 2000 can obtain the prediction sample of the reference block 2420 or 2440 by using the neighboring sample 2430 or 2450 of the reference block 2420 or 2440 as the reference sample, according to each of the multiple intra-frame prediction modes.

[0306] Image decoding apparatus 2000 can determine the error by comparing the predicted samples of reference block 2420 or 2440 with neighboring samples 2430 or 2450. Image decoding apparatus 2000 can also determine the error by using a cost function. For example, image decoding apparatus 2000 can determine the error by using at least one of SAD, SATD, SSE, and MR-SAD cost functions.

[0307] In embodiments of this disclosure, the image decoding apparatus 2000 can determine a first candidate intra-frame mode based on at least one of a plurality of neighboring samples 2430 of a first reference block 2420 and a plurality of internal samples included in the reference block 2420. The image decoding apparatus 2000 can determine a second candidate intra-frame mode based on at least one of a plurality of neighboring samples 2450 of a second reference block 2440 of the current block 2410 and a plurality of internal samples of the second reference block 2440. The image decoding apparatus 2000 can determine a first candidate intra-frame mode and a second candidate intra-frame mode with minimum error from the first reference block 2420 and the second reference block 2440, respectively. The image decoding apparatus 2000 can select one of the first candidate intra-frame mode and the second candidate intra-frame mode as the intra-frame mode for the reference block. For example, when... Figure 23 When a single reference block exists, the image decoding apparatus 2000 can determine the intra-frame mode for the reference block without the need for a process of determining candidate intra-frame modes.

[0308] In embodiments of this disclosure, the image decoding apparatus 2000 may determine an intra-frame mode for a reference block based on an error. The image decoding apparatus 2000 may determine a first error by performing prediction on a plurality of internal samples of the first reference block 2420 based on a plurality of neighboring samples 2430 of the first reference block 2420 according to a first candidate intra-frame mode. The image decoding apparatus 2000 may determine a second error by performing prediction on a plurality of internal samples of the second reference block 2440 based on a plurality of neighboring samples 2450 of the second candidate intra-frame mode. The image decoding apparatus 2000 may determine an intra-frame mode for the reference block from the first candidate intra-frame mode and the second candidate intra-frame mode based on the first error and the second error. The image decoding apparatus 2000 may determine the candidate intra-frame mode with the smaller error from the first candidate intra-frame mode and the second candidate intra-frame mode as the intra-frame mode for the reference block.

[0309] In embodiments of this disclosure, the image decoding apparatus 2000 may determine an intra-frame mode for a reference block based on quantization parameters (QPs) of a strip including a reference block. In embodiments, the quantization parameters may refer to variables used in the processing of decoding transform coefficients. The image decoding apparatus 2000 may perform inverse quantization based on the quantization parameters. The image decoding apparatus 2000 may obtain a first quantization parameter for a first reference block 2420 and a second quantization parameter for a second reference block 2440. The image decoding apparatus 2000 may determine a candidate intra-frame mode for a reference block having a smaller quantization parameter as the intra-frame mode for the reference block. When the first quantization parameter is equal to or greater than the second quantization parameter, the image decoding apparatus 2000 may select the second candidate intra-frame mode as the intra-frame mode for the reference block. When the first quantization parameter is less than the second quantization parameter, the image decoding apparatus 2000 may select the first candidate intra-frame mode as the intra-frame mode for the reference block.

[0310] In embodiments of this disclosure, the image decoding apparatus 2000 can determine the intra-frame mode for a reference block based on the frame sequence count (POC) value of a reference image. In embodiments, the POC value may refer to a variable related to the image display order. The image decoding apparatus 2000 can obtain a first POC value for a first reference block 2420 and a second POC value for a second reference block 2440. The image decoding apparatus 2000 can determine the candidate intra-frame mode of the reference image with a smaller difference between the current image and its POC value as the intra-frame mode for the reference block. When the difference between the POC value of the current image 2410 and the first POC value is less than the difference between the POC value of the current image 2410 and the second POC value, the image decoding apparatus 2000 can select the first candidate intra-frame mode as the intra-frame mode for the reference block. When the difference between the POC value of the current image 2410 and the first POC value is equal to or greater than the difference between the POC value of the current image 2410 and the second POC value, the image decoding apparatus 2000 can select the second candidate intra-frame mode as the intra-frame mode for the reference block.

[0311] In embodiments of this disclosure, the image decoding apparatus 2000 can determine the intra-frame mode for a reference block based on bidirectional prediction weights. The image decoding apparatus 2000 can obtain the prediction samples of the current block based on a weighted sum among multiple reference blocks. The image decoding apparatus 2000 can obtain bidirectional prediction weights for a first reference block 2420 and a second reference block 2440. When the weight for the first reference block 2420 is equal to or greater than the weight for the second reference block 2440, the image decoding apparatus 2000 can select a first candidate intra-frame mode as the intra-frame mode for the reference block. When the weight for the first reference block 2420 is less than the weight for the second reference block, the image decoding apparatus 2000 can select a second candidate intra-frame mode as the intra-frame mode for the reference block.

[0312] In embodiments of this disclosure, the image decoding apparatus 2000 may determine the intra-frame mode for the reference block based on the stripe type of the reference block. When the type of the stripe including the reference block is intra-frame (I) stripe, the image decoding apparatus 2000 may determine the candidate intra-frame modes of the reference block as the intra-frame mode for the reference block. When the type of the stripe including the first reference block 2420 is I stripe, the image decoding apparatus 2000 may select the first candidate intra-frame mode as the intra-frame mode for the reference block. When the type of the stripe including the second reference block 2440 is I stripe, the image decoding apparatus 2000 may select the second candidate intra-frame mode as the intra-frame mode for the reference block.

[0313] Image decoding apparatus 2000 can determine a transform kernel by using an intra-frame mode for a determined reference block. Herein, a transform kernel can refer to a kernel applied to a residual block to perform an inverse transform of at least one of a first transform, a second transform, or an inseparable first transform. Image decoding apparatus 2000 can determine a transform kernel for at least one of a first transform, a second transform, or an inseparable first transform by using a determined intra-frame mode for a reference block.

[0314] Figure 25 This is a flowchart of an image decoding method according to an embodiment of the present disclosure.

[0315] In embodiments of this disclosure, the image decoding method may be executed by an image decoding apparatus 2000. For example, the processor of the image decoding apparatus 2000 may execute at least one instruction included in memory to cause the image decoding apparatus 2000 to perform each operation of the image decoding method.

[0316] In operation S2510, the image decoding apparatus 2000 can determine multiple gradients using multiple internal samples of a reference block. The image decoding apparatus 2000 can determine horizontal and vertical variations using multiple internal samples of the reference block. For example, the image decoding apparatus 2000 can determine horizontal and vertical variations using a Sobel filter. The image decoding apparatus 2000 can determine an N×N block (or region) included in the reference block, for example, a 3×3 block. The N×N block (or region) in the reference block can be determined at specific intervals. For example, when the interval is 1, the top-left coordinate of the first N×N block can be (x1, y1), and the top-left coordinate of the second N×N block can be (x1+1, y1+1). Depending on the interval, at least some of the N×N blocks in the reference block can have the same samples. The image decoding apparatus 2000 can obtain the horizontal and vertical variations of the samples included in the determined N×N blocks. The image decoding apparatus 2000 can determine gradients based on the horizontal and vertical variations. The gradient can be determined based on the ratio between horizontal and vertical changes.

[0317] In operation S2520, the image decoding device 2000 can determine multiple directional intra-frame modes corresponding to the plurality of gradients. The image decoding device 2000 can determine the intra-frame prediction mode corresponding to the gradient. For example, when the gradient is 0 (i.e., horizontal), the image decoding device 2000 can determine the horizontal mode (e.g., intra-frame prediction mode 18) as the directional intra-frame mode corresponding to the gradient.

[0318] Image decoding apparatus 2000 can generate a gradient histogram (HOG) using multiple gradients. Image decoding apparatus 2000 can determine the HOG based on multiple intra-prediction modes determined for multiple N×N blocks. The HOG can be referred to as gradient statistics or gradient information. In embodiments of this disclosure, the HOG may include information indicating the frequency (or magnitude) of directional intra-prediction modes determined based on the gradients. In embodiments of this disclosure, weights can be applied to the frequencies of directional intra-prediction modes in the HOG. Image decoding apparatus 2000 can determine the weights of the intra-prediction modes corresponding to the gradients. As the horizontal and vertical variability increases, image decoding apparatus 2000 can increase the weights of the intra-prediction modes determined based on the gradients. For example, the weights can be determined based on the sum of the absolute values ​​of the horizontal and vertical variability.

[0319] In operation S2530, the image decoding apparatus 2000 can select an intra-frame mode for a reference block from a plurality of determined directional intra-frame modes. The image decoding apparatus 2000 can determine the most frequently determined intra-frame prediction mode as the intra-frame mode for the reference block of the current block. In embodiments of this disclosure, the image decoding apparatus 2000 can determine the intra-frame mode for the reference block based on HOG. The image decoding apparatus 2000 can determine the directional intra-frame mode with the highest frequency (or amplitude) as the intra-frame mode for the reference block.

[0320] In embodiments of this disclosure, operation S2120 and / or operation S2220 may determine the intra-frame mode for the reference block based on operations S2510 to S2530.

[0321] Figure 26 The process of determining the intra-frame mode for the reference block by using a reference block is illustrated according to an embodiment of the present disclosure.

[0322] In embodiments of this disclosure, the image decoding apparatus 2000 may determine the intra-frame mode for a reference block for the current block 2610 based on time information. The image decoding apparatus 2000 may also determine the intra-frame mode for the reference block using internal samples of the reference block 2620.

[0323] The image decoding device 2000 can determine the reference block C for the current block C 2610. pred2620. In embodiments of this disclosure, the current block 2610 may include blocks included in a P-frame (or P-strip). The reference block 2620 may be a block included in a reference image other than the current image, for inter-frame prediction of the current block 2610. The image decoding apparatus 2000 may determine the reference block 2620 included in the reference image (or reference strip) by using the motion vector of the current block 2610.

[0324] The image decoding apparatus 2000 can determine gradients using internal samples of the reference block 2620. The image decoding apparatus 2000 can determine horizontal and / or vertical variations using internal samples of the reference block 2620. For example, the image decoding apparatus 2000 can determine the horizontal and vertical variations of the reference block 2620 using a Sobel filter. The Sobel filter includes filters for obtaining horizontal or vertical variations within the input data. The image decoding apparatus 2000 can determine an N×N block (e.g., a 3×3 block) included in the reference block 2620. The image decoding apparatus 2000 can determine the N×N blocks in the reference block 2620 at specific intervals (e.g., l) in one direction and / or the vertical direction.

[0325] The image decoding device 2000 can determine the gradient based on horizontal and vertical variations. The image decoding device 2000 can determine the gradient as the ratio between the horizontal and vertical variations.

[0326] Image decoding apparatus 2000 can determine the intra-prediction mode corresponding to the gradient. Image decoding apparatus 2000 can generate HOG 2630 by using the gradients of a plurality of N×N blocks in reference block 2620. HOG 2630 may include information indicating the frequency (or amplitude) of the intra-prediction mode corresponding to the gradient of each of the plurality of blocks in reference block 2620. For example, HOG 2630 may include information including the frequency (or amplitude) corresponding to the intra-prediction mode, such as... Figure 26 As shown.

[0327] In embodiments of this disclosure, the magnitude (or frequency) of the intra-prediction mode of the HOG 2630 can be determined based on horizontal and vertical variations. For example, as horizontal and vertical variations increase, the magnitude of the intra-prediction mode can be determined to be higher. For example, the magnitude can be determined based on the sum of the absolute values ​​of the horizontal and vertical variations.

[0328] The image decoding apparatus 2000 can select an intra-frame mode for a reference block from a plurality of determined intra-frame prediction modes. In embodiments of this disclosure, the image decoding apparatus 2000 can determine the intra-frame mode for the reference block based on HOG 2630. The image decoding apparatus 2000 can determine the intra-frame prediction mode with the highest amplitude (or frequency) as the intra-frame mode for the reference block of the current block 2610.

[0329] The image decoding apparatus 2000 can determine the transform kernel by using a determined intra-frame mode for the reference block. Here, the transform kernel can refer to a kernel applied to the residual block to perform an inverse transform for at least one of a first transform, a second transform, or an inseparable first transform. The image decoding apparatus 2000 can determine the transform kernel for at least one of the first transform, the second transform, or the inseparable first transform by using a determined intra-frame mode for the reference block.

[0330] Figure 27 The process of determining the intra-frame mode for the reference block by using a reference block is illustrated according to an embodiment of the present disclosure.

[0331] In embodiments of this disclosure, the image decoding apparatus 2000 may determine the intra-frame mode for a reference block for the current block 2710 based on time information. The image decoding apparatus 2000 may determine the intra-frame mode for the reference block by using internal samples of reference blocks 2720 and 2740.

[0332] The image decoding device 2000 can determine the first reference block C for the current block C 2710. pred0 2720 ​​and the second reference block C pred1 2740. In embodiments of this disclosure, the current block 2710 may include blocks included in a B-frame (or B-strip). The first reference block 2720 and the second reference block 2740 may be blocks included in various reference images other than the current image for inter-frame prediction of the current block 2710. The image decoding apparatus 2000 may determine the reference block 2720 or 2740 included in the reference image (or reference strip) by using the motion vector of the current block 2710.

[0333] Image decoding apparatus 2000 can determine gradients using internal samples of reference blocks 2720 or 2740. Image decoding apparatus 2000 can determine horizontal and / or vertical variations using internal samples of reference blocks 2720 or 2740. For example, image decoding apparatus 2000 can determine the horizontal and vertical variations of reference blocks 2720 or 2740 using a Sobel filter. The Sobel filter includes filters for obtaining horizontal or vertical variations within the input data. Image decoding apparatus 2000 can determine N×N blocks (e.g., 3×3 blocks) included in reference blocks 2720 or 2740. Image decoding apparatus 2000 can determine N×N blocks in reference blocks 2720 or 2740 at specific intervals (e.g., l) in one direction and / or vertical direction.

[0334] The image decoding device 2000 can determine the gradient based on horizontal and vertical variations. The image decoding device 2000 can determine the gradient as the ratio between the horizontal and vertical variations.

[0335] Image decoding apparatus 2000 can determine an intra-prediction mode corresponding to a gradient. Image decoding apparatus 2000 can generate a HOG 2730 or 2750 by using the gradients of multiple N×N blocks in reference blocks 2720 or 2740. The size of the blocks in the first reference block 2720 may differ from the size of the blocks in the second reference block 2740. HOG 2730 or 2750 may include information indicating the frequency (or amplitude) of the intra-prediction mode corresponding to the gradient of each of the multiple blocks in reference blocks 2720 or 2740. For example, HOG 2730 or 2750 may include information including the frequency (or amplitude) corresponding to the intra-prediction mode, such as... Figure 27 As shown.

[0336] In embodiments of this disclosure, the amplitude (or frequency) of the intra-prediction mode of the HOG 2730 or 2750 can be determined based on horizontal and vertical variations. For example, as horizontal and vertical variations increase, the amplitude of the intra-prediction mode can be determined to be higher. For example, the amplitude can be determined based on the sum of the absolute values ​​of the horizontal and vertical variations.

[0337] The image decoding apparatus 2000 can select an intra-frame mode for a reference block from a plurality of determined intra-frame prediction modes. In embodiments of this disclosure, the image decoding apparatus 2000 can determine the intra-frame mode for the reference block based on HOG 2730 or 2750.

[0338] In embodiments of this disclosure, the image decoding apparatus 2000 may determine a first candidate intra-frame mode based on multiple gradients of the first reference block 2720. The image decoding apparatus 2000 may determine the first candidate intra-frame mode based on a first HOG 2730.

[0339] The image decoding apparatus 2000 can determine a second candidate intra-frame mode based on multiple gradients of the second reference block 2740. The image decoding apparatus 2000 can determine a first candidate intra-frame mode based on the second HOG 2750.

[0340] The image decoding apparatus 2000 can determine a first candidate intra-frame mode and a second candidate intra-frame mode having the highest amplitude (or frequency) from the first reference block 2720 and the second reference block 2740, respectively. The image decoding apparatus 2000 can select one of the first candidate intra-frame mode and the second candidate intra-frame mode as the intra-frame mode for the reference block. For example, when... Figure 26 When a single reference block exists, the image decoding apparatus 2000 can determine the intra-frame mode for the reference block without the need for a process of determining candidate intra-frame modes.

[0341] In embodiments of this disclosure, the image decoding apparatus 2000 may determine an intra-frame mode for a reference block based on an error. The image decoding apparatus 2000 may determine a first error by performing prediction on a plurality of internal samples of the first reference block 2720 based on a plurality of neighboring samples 2730 of the first reference block 2720 according to a first candidate intra-frame mode. The image decoding apparatus 2000 may determine a second error by performing prediction on a plurality of internal samples of the second reference block 2740 based on a plurality of neighboring samples 2750 of the second candidate intra-frame mode. The image decoding apparatus 2000 may determine an intra-frame mode for the reference block from the first candidate intra-frame mode and the second candidate intra-frame mode based on the first error and the second error. The image decoding apparatus 2000 may determine the candidate intra-frame mode with the smaller error from the first candidate intra-frame mode and the second candidate intra-frame mode as the intra-frame mode for the reference block.

[0342] In embodiments of this disclosure, the image decoding apparatus 2000 may determine an intra-frame mode for a reference block based on quantization parameters (QPs) of a strip including a reference block. In embodiments, the quantization parameters may refer to variables used in the processing of decoding transform coefficients. The image decoding apparatus 2000 may perform inverse quantization based on the quantization parameters. The image decoding apparatus 2000 may obtain a first quantization parameter for a first reference block 2720 and a second quantization parameter for a second reference block 2740. The image decoding apparatus 2000 may determine a candidate intra-frame mode for a reference block having a smaller quantization parameter as the intra-frame mode for the reference block. When the first quantization parameter is equal to or greater than the second quantization parameter, the image decoding apparatus 2000 may select the second candidate intra-frame mode as the intra-frame mode for the reference block. When the first quantization parameter is less than the second quantization parameter, the image decoding apparatus 2000 may select the first candidate intra-frame mode as the intra-frame mode for the reference block.

[0343] In embodiments of this disclosure, the image decoding apparatus 2000 may determine an intra-frame mode for a reference block based on the Frame Order Count (POC) value of the reference block. In embodiments, the POC value may refer to a variable related to the image display order. The image decoding apparatus 2000 may obtain a first POC value for a first reference block 2720 and a second POC value for a second reference block 2740. The image decoding apparatus 2000 may determine a candidate intra-frame mode for the reference block from the reference image whose POC value has a smaller difference from the current image. When the difference between the POC value of the current image 2710 and the first POC value is less than the difference between the POC value of the current image 2710 and the second POC value, the image decoding apparatus 2000 may select the first candidate intra-frame mode as the intra-frame mode for the reference block. When the difference between the POC value of the current image 2710 and the first POC value is equal to or greater than the difference between the POC value of the current image 2710 and the second POC value, the image decoding apparatus 2000 may select the second candidate intra-frame mode as the intra-frame mode for the reference block.

[0344] In embodiments of this disclosure, the image decoding apparatus 2000 can determine the intra-frame mode for a reference block based on bidirectional prediction weights. The image decoding apparatus 2000 can obtain the prediction samples of the current block based on a weighted sum among multiple reference blocks. The image decoding apparatus 2000 can obtain bidirectional prediction weights for a first reference block 2720 and a second reference block 2740. When the weight for the first reference block 2720 is equal to or greater than the weight for the second reference block 2740, the image decoding apparatus 2000 can select a first candidate intra-frame mode as the intra-frame mode for the reference block. When the weight for the first reference block 2720 is less than the weight for the second reference block, the image decoding apparatus 2000 can select a second candidate intra-frame mode as the intra-frame mode for the reference block.

[0345] In embodiments of this disclosure, the image decoding apparatus 2000 may determine the intra-frame mode for the reference block based on the stripe type of the reference block. When the type of the stripe including the reference block is intra-frame (I) stripe, the image decoding apparatus 2000 may determine the candidate intra-frame modes of the reference block as the intra-frame mode for the reference block. When the type of the stripe including the first reference block 2720 is I stripe, the image decoding apparatus 2000 may select the first candidate intra-frame mode as the intra-frame mode for the reference block. When the type of the stripe including the second reference block 2740 is I stripe, the image decoding apparatus 2000 may select the second candidate intra-frame mode as the intra-frame mode for the reference block.

[0346] The image decoding apparatus 2000 can determine the transform kernel by using a determined intra-frame mode for a reference block. Here, the transform kernel can refer to a kernel applied to a residual block to perform an inverse transform for at least one of a first transform, a second transform, or an inseparable first transform. The image decoding apparatus 2000 can determine the transform kernel for at least one of the first transform, the second transform, or the inseparable first transform by using a determined intra-frame mode for a reference block.

[0347] Figure 28 The process of determining the intra-frame mode for the reference block by using a reference block is illustrated according to an embodiment of the present disclosure.

[0348] In embodiments of this disclosure, the image decoding apparatus 2000 may determine the intra-frame mode for a reference block for the current block 2810 based on time information. The image decoding apparatus 2000 may determine the intra-frame mode for the reference block by using internal samples of reference blocks 2820 and 2830.

[0349] The image decoding device 2000 can determine the first reference block C for the current block C 2810. pred0 2820 and the second reference block C pred1 2830. In embodiments of this disclosure, the current block 2810 may include blocks included in a B-frame (or B-strip). The first reference block 2820 and the second reference block 2830 may be blocks included in various reference images other than the current image for inter-frame prediction of the current block 2810. The image decoding apparatus 2000 may determine the reference block 2820 or 2830 included in the reference image (or reference strip) by using the motion vector of the current block 2810.

[0350] Image decoding apparatus 2000 can determine gradients using internal samples of reference blocks 2820 or 2830. Image decoding apparatus 2000 can determine horizontal and / or vertical variations using internal samples of reference blocks 2820 or 2830. For example, image decoding apparatus 2000 can determine the horizontal and vertical variations of reference blocks 2820 or 2830 using a Sobel filter. The Sobel filter includes filters for obtaining horizontal or vertical variations within the input data. Image decoding apparatus 2000 can determine N×N blocks (e.g., 3×3 blocks) included in reference blocks 2820 or 2830. Image decoding apparatus 2000 can determine N×N blocks in reference blocks 2820 or 2830 at specific intervals (e.g., l) in one direction and / or vertical direction.

[0351] The image decoding device 2000 can determine the gradient based on horizontal and vertical variations. The image decoding device 2000 can determine the gradient as the ratio between the horizontal and vertical variations.

[0352] Image decoding apparatus 2000 can determine an intra-prediction mode corresponding to a gradient. Image decoding apparatus 2000 can generate a HOG 2840 by using the gradients of multiple N×N blocks in reference blocks 2820 or 2830. The size of the blocks in the first reference block 2820 may differ from the size of the blocks in the second reference block 2830. The size of the blocks in reference blocks 2820 or 2830 can be determined based on the size of a reference image. For example, when the size of reference 0 is twice the size of reference image 1, the gradient of the first reference block can be determined based on 2N×2N blocks, and the gradient of the second reference block can be determined based on N×N blocks. HOG 2840 may include information indicating the frequency (or amplitude) of the intra-prediction mode corresponding to the gradient of each of the multiple blocks in reference blocks 2820 or 2830. For example, HOG 2840 may include information including the frequency (or amplitude) corresponding to the intra-prediction mode, such as... Figure 28 As shown. HOG 2840 may include the amplitude (or frequency) of the intra-prediction mode of both the first reference block 2820 and the second reference block 2830. In an embodiment, HOG 2840 may have the form of a combination of HOG 2730 and HOG 2750.

[0353] In embodiments of this disclosure, the magnitude (or frequency) of the intra-prediction mode of the HOG 2840 can be determined based on horizontal and vertical variations. For example, as horizontal and vertical variations increase, the magnitude of the intra-prediction mode can be determined to be higher. For example, the magnitude can be determined based on the sum of the absolute values ​​of the horizontal and vertical variations.

[0354] The image decoding apparatus 2000 can select an intra-frame mode for a reference block from a plurality of determined intra-frame prediction modes. In embodiments of this disclosure, the image decoding apparatus 2000 can determine the intra-frame mode for the reference block based on HOG 2840. The image decoding apparatus 2000 can determine the intra-frame prediction mode with the highest amplitude (or frequency) as the intra-frame mode for the reference block of the current block 2810.

[0355] Figure 29 This is a flowchart of an image decoding method according to an embodiment of the present disclosure.

[0356] In embodiments of this disclosure, the image decoding method may be executed by an image decoding apparatus 2000. For example, the processor of the image decoding apparatus 2000 may execute at least one instruction included in memory to cause the image decoding apparatus 2000 to perform each operation of the image decoding method.

[0357] In operation S2910, the image decoding device 2000 can determine a reference block through template matching. In embodiments of this disclosure, the reference block may be included in the same image as the current block.

[0358] The image decoding device 2000 can identify the template of the current block. The template of the current block may include neighboring samples of the current block. The neighboring samples of the current block may include adjacent samples and / or non-adjacent samples of the current block. (Refer to...) Figure 35 A detailed description of the template form according to embodiments of this disclosure is provided.

[0359] The image decoding apparatus 2000 can determine a template similar to the template of the current block in the current image. The image decoding apparatus 2000 can also determine a block having a template similar to the current template as a reference block for the current block. Furthermore, the image decoding apparatus 2000 can determine a reference block from the blocks included in the search area.

[0360] In embodiments of this disclosure, the image decoding apparatus 2000 can determine a reference block using a cost function. The image decoding apparatus 2000 can determine the template error between the template of the current block and the candidate template based on at least one cost function selected from SAD, SATD, SSE, and MR-SAD. The image decoding apparatus 2000 can determine the template error based on each sample in the template of the current block and each sample in the candidate template. The image decoding apparatus 2000 can determine a block of candidate templates with smaller template errors as the reference block.

[0361] In operation S2920, the image decoding device 2000 can obtain the error based on multiple intra-frame prediction modes based on the neighboring samples and the internal samples of the reference block.

[0362] In embodiments of this disclosure, the image decoding apparatus 2000 can predict internal samples of a reference block using neighboring samples of the reference block. For example, the image decoding apparatus 2000 can determine predicted samples based on neighboring samples of the reference block, which serve as reference samples, according to multiple intra-frame prediction modes (e.g., intra-frame directional prediction modes 2 to 66 or all available intra-frame prediction modes). The image decoding apparatus 2000 can determine errors by comparing the predicted samples with the internal samples of the decoded reference block.

[0363] In embodiments of this disclosure, the image decoding apparatus 2000 may determine reference samples for determining errors based on intra-frame prediction modes separately from the template. For example, when the width and height of the template are the same as the current block, the width and height of the reference samples for determining errors based on intra-frame prediction modes may be twice that of the current block. Furthermore, the reference samples may include multiple reference lines. For example, similar to multi-line reference intra-frame prediction, the image decoding apparatus 2000 may perform prediction based on reference pixels of multiple lines.

[0364] In operation S2930, the image decoding device 2000 can determine the intra-frame mode for the reference block based on the error. The image decoding device 2000 can determine the intra-frame mode with the smallest error among the plurality of intra-frame modes as the intra-frame mode for the reference block.

[0365] In embodiments of this disclosure, operation S2120 and / or operation S2220 may determine the intra-frame mode for the reference block based on operations S2910 to S2930.

[0366] Figure 30 The process of determining the intra-frame mode for the reference block by using neighboring samples of the reference block according to an embodiment of the present disclosure is illustrated.

[0367] In embodiments of this disclosure, the image decoding apparatus 2000 can determine the intra-frame mode of a reference block for the current block 3010 based on spatial information. The image decoding apparatus 2000 can also determine the intra-frame mode of a reference block 3040 by using neighboring samples (or templates T) of the reference block 3040. R The intra-frame mode for the reference block is determined by the samples of reference block 3050 and the internal samples of reference block 3040. In embodiments of this disclosure, the neighboring samples (or templates T) of reference block 3040 are used to determine the intra-frame mode for the reference block. R 3050) can be referred to as spatially neighboring sample points.

[0368] Image decoding device 2000 can determine the template T for the current block C 3010. C 3020. In embodiments of this disclosure, reference block 3040 is determined using template matching and is different from reference block 3030 used for inter-frame prediction of current block 3010. Image decoding apparatus 2000 can determine reference block 3040 by template matching based on template 3020 of current block 3010. Image decoding apparatus 2000 can determine the template error between template 3020 of current block 3010 and templates included in the current image. Image decoding apparatus 2000 can determine the block with template 3050 having the minimum template error as reference block 3040.

[0369] Image decoding apparatus 2000 can perform intra-frame prediction based on one or more intra-frame prediction modes using neighboring samples of reference block 3040. In an embodiment, the neighboring samples of reference block 3040 used for intra-frame prediction may be the same as the template 3050 of reference block 3040. Image decoding apparatus 2000 can obtain predicted sample R by performing prediction on internal samples of reference block 3040 using neighboring samples 3050 of reference block 3040. predThe image decoding apparatus 2000 can obtain the prediction sample of the reference block 3040 by using the neighboring sample 3050 of the reference block 3040 as reference sample according to each of the plurality of intra prediction modes. For example, the image decoding apparatus 2000 can perform prediction for all intra prediction modes, but is not limited thereto, and can perform prediction for a predetermined set of intra prediction modes among the intra prediction modes.

[0370] The image decoding apparatus 2000 can determine the error by comparing the predicted sample points of the reference block 3040 with the neighboring sample points 3050. When the cost function is SAD, the image decoding apparatus 2000 can determine the error based on the sum of the absolute values ​​of the differences between the predicted sample points of the reference block 3040 and the neighboring sample points 3050.

[0371] The image decoding apparatus 2000 can determine the intra-frame mode for the reference block based on errors. The image decoding apparatus 2000 can determine the intra-frame mode for the reference block based on the error between intra-frame prediction samples and internal samples of the reference block. The image decoding apparatus 2000 can determine the intra-frame prediction mode with the minimum error as the intra-frame mode for the reference block. Even when the prediction mode of the current image 3010 is an inter-frame mode, the image decoding apparatus 2000 can determine the intra-frame mode for the reference block using the reference block of the current image.

[0372] The image decoding apparatus 2000 can determine the transform kernel by using a determined intra-frame mode for a reference block. Here, the transform kernel can refer to a kernel applied to a residual block to perform an inverse transform for at least one of a first transform, a second transform, or an inseparable first transform. The image decoding apparatus 2000 can determine the transform kernel for at least one of the first transform, the second transform, or the inseparable first transform by using a determined intra-frame mode for a reference block.

[0373] Figure 31 This is a flowchart of an image decoding method according to an embodiment of the present disclosure.

[0374] In embodiments of this disclosure, the image decoding method may be executed by an image decoding apparatus 2000. For example, the processor of the image decoding apparatus 2000 may execute at least one instruction included in memory to cause the image decoding apparatus 2000 to perform each operation of the image decoding method.

[0375] During operation S3110, the image decoding device 2000 can obtain neighboring samples of the current block. In embodiments of this disclosure, neighboring samples of the current block may include adjacent and non-adjacent samples of the current block. Adjacent samples of the current block may include samples to the left, upper left, and / or upper part of the current block. In embodiments of this disclosure, neighboring samples of the current block may be identical to the template of the current block. (Refer to...) Figure 35A detailed description of the template form according to embodiments of this disclosure is provided.

[0376] In operation S3120, the image decoding apparatus 2000 can obtain the error based on multiple intra-frame prediction modes based on the neighboring samples of the current block. The image decoding apparatus 2000 can determine at least some of the neighboring samples of the current block as reference samples and determine the other samples as reconstructed samples.

[0377] In embodiments of this disclosure, the image decoding apparatus 2000 can predict other samples of neighboring samples based on reference samples of neighboring samples of the current block. For example, the image decoding apparatus 2000 can determine predicted samples based on reference samples of neighboring samples of the current block, according to multiple intra-frame modes (e.g., intra-frame orientation prediction modes 2 to 66 or all available intra-frame prediction modes). The image decoding apparatus 2000 can determine the error by comparing the predicted samples with the reconstructed samples.

[0378] In embodiments of this disclosure, the image decoding apparatus 2000 may determine reference samples for determining errors based on an intra-frame prediction mode, separately from the template. For example, when the width and height of the template are the same as the current block, the width and height of the reference samples for determining errors based on the intra-frame prediction mode may be twice the width and height of the current block. Furthermore, the reference samples may include multiple reference lines. For example, similar to multi-line reference intra-frame prediction, the image decoding apparatus 2000 may perform prediction based on reference pixels of multiple lines.

[0379] In operation S3130, the image decoding apparatus 2000 can determine the intra-frame mode for the reference block based on the error. The image decoding apparatus 2000 can determine the intra-frame mode with the smallest error among a plurality of intra-frame modes as the intra-frame mode for the reference block.

[0380] In embodiments of this disclosure, operation S2120 and / or operation S2220 may determine the intra-frame mode for the reference block based on operations S3110 to S3130.

[0381] Figure 32 The present disclosure illustrates a process for determining an intra-frame mode for a reference block using neighboring samples of the current block.

[0382] In embodiments of this disclosure, the image decoding apparatus 2000 can determine the intra-frame mode of a reference block for the current block 3210 based on spatial information. The image decoding apparatus 2000 can also determine the intra-frame mode of a reference block for the current block 3210 by using neighboring samples (or templates T) of the current block 3210. R The intra-frame mode for the reference block is determined. In embodiments of this disclosure, the neighboring samples (or templates) of the current block 3210 may be referred to as spatial neighboring samples.

[0383] The image decoding device 2000 can determine the neighboring samples (or templates) of the current block C 3210. The neighboring samples of the current block 3210 may include adjacent samples and non-adjacent samples of the current block 3210. The image decoding device 2000 can determine some of the neighboring samples of the current block 3210 as reference samples R. ref 3220, and the remaining neighboring samples in the neighboring samples are identified as reconstructed samples 3230.

[0384] The image decoding apparatus 2000 can perform intra-frame prediction using reference sample 3220 according to one or more intra-frame prediction modes. The image decoding apparatus 2000 can obtain the prediction sample R by predicting the remaining neighboring samples other than the reference sample 3220 based on the reference sample 3220, which is the neighboring sample of the current block 3210. pred The image decoding apparatus 2000 can obtain prediction samples from the reference sample 3220 according to each of a plurality of intra prediction modes. For example, the image decoding apparatus 2000 can perform prediction for all intra prediction modes, but is not limited thereto, and can perform prediction for a predetermined set of intra prediction modes among the intra prediction modes.

[0385] Image decoding device 2000 can decode predicted samples R pred The error is determined by comparing the predicted sample with the reconstructed sample 3230. The image decoding device 2000 can determine the error by using a cost function. When the cost function is SAD, the image decoding device 2000 can determine the error based on the sum of the absolute values ​​of the differences between the predicted sample and the reconstructed sample 3230.

[0386] The image decoding apparatus 2000 can determine the intra-frame mode for the reference block based on the error. The image decoding apparatus 2000 can determine the intra-frame mode for the reference block based on the error between the predicted sample and the reconstructed sample 3230. The image decoding apparatus 2000 can determine the intra-frame prediction mode with the minimum error as the intra-frame mode for the reference block. Even when the prediction mode of the current image 3210 is an inter-frame mode, the image decoding apparatus 2000 can determine the intra-frame mode for the reference block using the reference block of the current image.

[0387] The image decoding apparatus 2000 can determine the transform kernel by using a determined intra-frame mode for a reference block. Here, the transform kernel can refer to a kernel applied to a residual block to perform an inverse transform for at least one of a first transform, a second transform, or an inseparable first transform. The image decoding apparatus 2000 can determine the transform kernel for at least one of the first transform, the second transform, or the inseparable first transform by using a determined intra-frame mode for a reference block.

[0388] Figure 33 This is a flowchart of an image decoding method according to an embodiment of the present disclosure.

[0389] In embodiments of this disclosure, the image decoding method may be executed by an image decoding apparatus 2000. For example, the processor of the image decoding apparatus 2000 may execute at least one instruction included in memory to cause the image decoding apparatus 2000 to perform each operation of the image decoding method.

[0390] In operation S3310, the image decoding device 2000 can determine a reference block through template matching. In embodiments of this disclosure, the reference block may be included in the same image as the current block.

[0391] The image decoding device 2000 can identify the template of the current block. The template of the current block may include neighboring samples of the current block. The neighboring samples of the current block may include adjacent samples and / or non-adjacent samples of the current block. (Refer to...) Figure 35 A detailed description of the template form according to embodiments of this disclosure is provided.

[0392] The image decoding apparatus 2000 can determine a template similar to the template of the current block in the current image. The image decoding apparatus 2000 can also determine a block having a template similar to the current template as a reference block for the current block. Furthermore, the image decoding apparatus 2000 can determine a reference block from the blocks included in the search area.

[0393] In embodiments of this disclosure, the image decoding apparatus 2000 can determine a reference block using a cost function. The image decoding apparatus 2000 can determine the template error between the template of the current block and the candidate template based on at least one cost function selected from SAD, SATD, SSE, and MR-SAD. The image decoding apparatus 2000 can determine the template error based on each sample in the template of the current block and each sample in the candidate template. The image decoding apparatus 2000 can determine a block of candidate templates with smaller template errors as the reference block.

[0394] In operation S3320, the image decoding device 2000 can determine multiple gradients using multiple internal samples of a reference block. The image decoding device 2000 can determine horizontal and vertical variations using multiple internal samples of the reference block. For example, the image decoding device 2000 can determine horizontal and vertical variations using a Sobel filter. The image decoding device 2000 can determine an N×N block (e.g., a 3×3 block) included in the reference block. The image decoding device 2000 can obtain the horizontal and vertical variations of the samples included in the determined N×N block. The image decoding device 2000 can determine the gradient based on the horizontal and vertical variations. The gradient can be determined based on the ratio between the horizontal and vertical variations. In embodiments of this disclosure, operation S3320 can correspond to... Figure 25 Operation S2510.

[0395] In operation S3330, the image decoding device 2000 can determine multiple directional intra-frame modes corresponding to multiple gradients. The image decoding device 2000 can determine the intra-frame prediction mode corresponding to the gradients.

[0396] Image decoding apparatus 2000 can generate a gradient histogram (HOG) using multiple gradients. Image decoding apparatus 2000 can determine the HOG based on multiple intra-prediction modes determined for multiple N×N blocks. The HOG can be referred to as gradient statistics or gradient information. In embodiments of this disclosure, the HOG may include information indicating the frequency (or magnitude) of the directional intra-prediction modes determined based on the gradients. In embodiments of this disclosure, weights can be applied to the frequencies of the directional intra-prediction modes in the HOG. Image decoding apparatus 2000 can determine the weights of the intra-prediction modes corresponding to the gradients. As the horizontal and vertical variations increase, image decoding apparatus 2000 can increase the weights of the intra-prediction modes determined based on the gradients. For example, the weights can be determined based on the sum of the absolute values ​​of the horizontal and vertical variations. In embodiments of this disclosure, operation S3330 can correspond to... Figure 25 Operation S2520.

[0397] In operation S3340, the image decoding apparatus 2000 can select an intra-frame mode for a reference block from a plurality of determined directional intra-frame modes. The image decoding apparatus 2000 can determine the most frequently determined intra-frame prediction mode as the intra-frame mode for the reference block of the current block. In embodiments of this disclosure, the image decoding apparatus 2000 can determine the intra-frame mode for the reference block based on HOG. The image decoding apparatus 2000 can determine the directional intra-frame mode with the highest frequency (or amplitude) as the intra-frame mode for the reference block. In embodiments of this disclosure, operation S3340 can correspond to... Figure 25 Operation S2530.

[0398] In embodiments of this disclosure, operation S2120 and / or operation S2220 may determine the intra-frame mode for the reference block based on operations S3310 to S3340.

[0399] Figure 34 The process of determining the intra-frame mode for the reference block by using a reference block is illustrated according to an embodiment of the present disclosure.

[0400] In embodiments of this disclosure, the image decoding apparatus 2000 can determine the intra-frame mode of a reference block of the current block 3410 based on spatial information. The image decoding apparatus 2000 can determine the intra-frame mode for the reference block by using samples of the reference block 3430.

[0401] Image decoding device 2000 can determine the template T for the current block C 3410. C3420. In embodiments of this disclosure, reference block 3430 is determined using template matching, and reference block 3430 is different from reference block 3030 already used for inter-frame prediction of current block 3410. Image decoding apparatus 2000 can determine reference block 3430 by template matching based on template 3420 of current block 3410. Image decoding apparatus 2000 can determine the template error between template 3420 of current block 3410 and templates included in the current image. Image decoding apparatus 2000 can determine the block with template 3440 having the minimum template error as reference block 3430.

[0402] The image decoding apparatus 2000 can determine gradients using internal samples of the reference block 3430. The image decoding apparatus 2000 can determine horizontal and / or vertical variations using internal samples of the reference block 3430. For example, the image decoding apparatus 2000 can determine the horizontal and vertical variations of the reference block 3430 using a Sobel filter. The Sobel filter includes filters for obtaining horizontal or vertical variations within the input data. The image decoding apparatus 2000 can determine an N×N block (e.g., a 3×3 block) included in the reference block 3430. The image decoding apparatus 2000 can determine the N×N blocks in the reference block 3430 at specific intervals (e.g., l) in one direction and / or the vertical direction.

[0403] The image decoding device 2000 can determine the gradient based on horizontal and vertical variations. The image decoding device 2000 can determine the gradient as the ratio between the horizontal and vertical variations.

[0404] The image decoding apparatus 2000 can determine the intra-prediction mode corresponding to the gradient. The image decoding apparatus 2000 can generate a HOG 3450 by using the gradients of a plurality of N×N blocks in reference block 3430. The HOG 3450 may include information indicating the frequency (or amplitude) of the intra-prediction mode corresponding to the gradient of each of the plurality of blocks in reference block 3430. For example, the HOG 3450 may include information including the frequency (or amplitude) corresponding to the intra-prediction mode, such as... Figure 34 As shown.

[0405] In embodiments of this disclosure, the magnitude (or frequency) of the intra-prediction mode of the HOG 3450 can be determined based on horizontal and vertical variations. For example, as horizontal and vertical variations increase, the magnitude of the intra-prediction mode can be determined to be higher. For example, the magnitude can be determined based on the sum of the absolute values ​​of the horizontal and vertical variations.

[0406] The image decoding apparatus 2000 can select an intra-frame mode for a reference block from a plurality of determined intra-frame prediction modes. In embodiments of this disclosure, the image decoding apparatus 2000 can determine the intra-frame mode for the reference block based on HOG 3450. The image decoding apparatus 2000 can determine the intra-frame prediction mode with the highest amplitude (or frequency) as the intra-frame mode for the reference block of the current block 3410.

[0407] The image decoding apparatus 2000 can determine the transform kernel by using a determined intra-frame mode for a reference block. Here, the transform kernel can refer to a kernel applied to a residual block to perform an inverse transform for at least one of a first transform, a second transform, or an inseparable first transform. The image decoding apparatus 2000 can determine the transform kernel for at least one of the first transform, the second transform, or the inseparable first transform by using a determined intra-frame mode for a reference block.

[0408] Figure 35 It is a diagram used to describe the form of a template according to embodiments of the present disclosure.

[0409] Reference Figure 35 The templates 3510, 3520, 3530, 3540, and 3550 of the current block 3500 can have various forms. In embodiments of this disclosure, the form of the template to be compared in the reconstructed area can be determined depending on the template of the current block 3500. In embodiments of this disclosure, templates with shapes similar to templates 3510 and 3530 can be referred to as having an L-shape.

[0410] In embodiments of this disclosure, the image decoding apparatus 2000 can determine that the template of the current block 3500 includes at least one of a first template located to the left of the current block 3500, a second template located above the current block 3500, and a third template located to the upper left of the current block 3500. For example, template 3510 may include a first template located to the left of the current block 3500, a second template located above the current block 3500, and a third template located to the upper left of the current block 3500, and template 3520 may include the first template and the second template but not the third template. Optionally, the template may include only the second template located above the current block 3500 or only the third template located to the left of the current block 3500. However, it is not limited to this, and the template may include a template located to the right of the current block 3500.

[0411] In embodiments of this disclosure, the image decoding apparatus 2000 can perform template matching prediction using a template 3530 that includes at least some of a right-side reference sample, an upper-side reference sample, and an upper-right reference sample. The image decoding apparatus 2000 can determine the template based on the block coding order (or coding direction). In embodiments, when block coding is performed from right to left, the image decoding apparatus 2000 can determine the reference block of the current block 3500 or perform template matching prediction by using the template 3530.

[0412] In embodiments of this disclosure, the image decoding apparatus 2000 can determine the reference block of the current block 3500 or perform template matching prediction by using a template 3540 including at least some of the right reference sample, top reference sample, left reference sample, top-left reference sample, and top-right reference sample, or perform template matching prediction by using a template 3540 including at least some of the right reference sample, top reference sample, and top-right reference sample. In embodiments of this disclosure, when the block encoding order (or encoding direction) changes, the image decoding apparatus 2000 can determine the reference block of the current block 3500 or perform template matching prediction by using a template 3540 including samples for reference.

[0413] In embodiments of this disclosure, the height of a first template located to the left of the current block 3500 or the width of a second template located above the current block 3500 can be determined based on the height or width of the current block 3500. For example, template 3550 may include a first template having twice the height of the current block 3500 and a second template having twice the width of the current block 3500. For example, template 3550 may have a form produced by modifying the height of the first template and the width of the second template, but is not limited thereto, and templates 3510, 3530, and 3540 may also have modified forms.

[0414] The template for the current block 3500 may include one or more reference lines. The first template may include m reference lines, where m is an integer of 1 or greater, and the second template may include n reference lines, where n is an integer of 1 or greater.

[0415] In embodiments of this disclosure, the number of reference lines can be predetermined. For example, the first template and the second template can each include four reference lines (i.e., m=4 and n=4). In embodiments of this disclosure, the number of reference lines can be determined based on the dimensions of the current block 3500. For example, the first template and the second template can each include as many reference lines as half the width and half the height of the current block 3500 (i.e., m=W / 2 and n=H / 2).

[0416] In embodiments of this disclosure, the template can be transmitted as a signal. The image decoding device 2000 can obtain information about the template format from the bitstream. In embodiments of this disclosure, the template format can be determined based on the size of the current block 3500.

[0417] Figure 35 This is a diagram used to describe a template according to embodiments of the present disclosure; however, the template is not limited thereto, and various forms of templates can be determined. Furthermore, although the image decoding apparatus 2000 is described in this disclosure as performing template matching based on a template 3510 including all left-side reference samples, top reference samples, and upper-left reference samples, it is not limited thereto, and templates such as... Figure 35 Various forms of templates described in the document are used to perform template matching prediction.

[0418] Figure 36 This is a block diagram illustrating the configuration of an image encoding apparatus according to an embodiment of the present disclosure.

[0419] Reference Figure 36 The image encoding device 3600 may include a predictive encoder 3610 and a generator 3620.

[0420] In embodiments of this disclosure, the predictive encoder 3610 and generator 3620 may be implemented by at least one processor. In embodiments of this disclosure, the image encoding apparatus 3600 may include a memory for storing input / output data of the predictive encoder 3610 and generator 3620. The predictive encoder 3610 and generator 3620 may operate according to instructions stored in the memory. In embodiments of this disclosure, the image encoding apparatus 3600 may include a memory controller for controlling input / output of data to / from the memory.

[0421] In embodiments of this disclosure, the predictive encoder 3610 may correspond to Figure 19 The predictive encoder 1915 is shown. In embodiments of this disclosure, the generator 3620 may correspond to... Figure 19 The entropy encoder 1925 is shown.

[0422] The predictive encoder 3610 can determine the prediction mode of the current block. The current block may include at least one of the following: the largest coding unit, coding unit, transform unit, or prediction unit divided from the current image. In embodiments of this disclosure, the prediction mode of the current block may include at least one of the following: intra-frame mode, inter-frame mode, combined mode, geometric partitioning mode, block copy mode, or template matching prediction mode.

[0423] In embodiments of this disclosure, intra-frame modes may include planar modes, intra-DC modes, angular modes (intra-angular modes 2 to 66), and wide-angular modes (intra-wide-angular modes) -14 to -1 and 67 to 80. In embodiments of this disclosure, planar mode may refer to a mode used to determine predicted samples based on a weighted average of the distances to the left, top, bottom left, and top right reference samples of the current block. In embodiments of this disclosure, DC mode may refer to a mode used to determine the average value of reference samples as predicted samples. In embodiments of this disclosure, in angular mode, the position of reference samples can be identified by considering the direction indicated by the intra-angular mode to generate predicted samples for samples in the current block. Wide-angular mode can be used to identify reference samples for samples in non-square current blocks. In embodiments of this disclosure, predictive encoder 3610 may determine one of the wide-angular modes as an intra-prediction mode for current blocks with a non-square shape. In embodiments of this disclosure, the prediction encoder 3610 may be configured differently in terms of the number and type of intra-prediction modes available in intra-frame mode.

[0424] In embodiments of this disclosure, the predictive encoder 3610 may determine an intra-prediction mode based on the most probable mode (MPM). The predictive encoder 3610 may determine whether an MPM is used. The generator 3620 may generate a bitstream including information related to whether an MPM is used. The predictive encoder 3610 may determine a candidate mode list. The predictive encoder 3610 may determine the candidate mode list based on the intra-prediction modes of the blocks above and to the left of the current block. The predictive encoder 3610 may determine one of the candidate modes in the list as the intra-prediction mode for the current block. The predictive encoder 3610 may generate a bitstream including information indicating the intra-prediction mode for the current block in the candidate mode list.

[0425] In embodiments of this disclosure, the predictive encoder 3610 can determine an intra-prediction mode based on a template. The predictive encoder 3610 can determine a template for the current block. The template for the current block may include left-side samples, top-left samples, and / or top samples of the current block. The predictive encoder 3610 can determine neighboring samples of the template for the current block. Neighboring samples of the template may include left-side, top-left, and / or top samples of the template. The predictive encoder 3610 can perform prediction on the template using neighboring samples as reference samples. The predictive encoder 3610 can determine an intra-mode for a reference block by comparing the predicted template with the reconstructed template of the current block. The predictive encoder 3610 can determine the intra-mode for the reference block as the intra-mode with the minimum error between the predicted template and the reconstructed template of the current block. In embodiments of this disclosure, the process by which the predictive encoder 3610 determines an intra-prediction mode by performing prediction on a template can be referred to as Template-Based Intra-Mode Derivation (TIMD).

[0426] In embodiments of this disclosure, the predictive encoder 3610 can infer the intra-prediction mode of the current block by using neighboring samples of the current block. The predictive encoder 3610 can determine the gradient by using neighboring samples of the current block. The predictive encoder 3610 can determine multiple 3×3 blocks adjacent to the current block. The predictive encoder 3610 can obtain the horizontal and vertical variations of the samples included in each determined 3×3 block. The predictive encoder 3610 can determine the gradient based on the horizontal and vertical variations. The predictive encoder 3610 can determine the horizontal and vertical variations by using a Sobel filter. The predictive encoder 3610 can determine the intra-prediction mode corresponding to the gradient. The predictive encoder 3610 can determine the intra-prediction mode of the current block based on multiple intra-prediction modes determined for multiple 3×3 blocks. In embodiments of this disclosure, the predictive encoder 3610 can determine the most frequently determined intra-prediction mode as the intra-prediction mode for the current block. In embodiments of this disclosure, the predictive encoder 3610 can determine the amplitude based on the horizontal and vertical variations. The predictive encoder 3610 can determine the intra-prediction mode of the current block based on amplitude. The predictive encoder 3610 can determine the weights of the intra-prediction mode corresponding to the gradient as amplitude. For example, the predictive encoder 3610 can increase the weights of the gradient-determined intra-prediction mode as horizontal and vertical variations increase. The predictive encoder 3610 can determine the intra-prediction mode of the current block based on the result reflecting the amplitude-determined weights. In embodiments of this disclosure, the process by which the predictive encoder 3610 determines the intra-prediction mode based on gradients can be referred to as decoder-side intra-mode derivation (DIMD).

[0427] In embodiments of this disclosure, the block copying mode may include an intra-block copying mode. In embodiments, the block copying mode may include an intra-block copying mode. In embodiments, the intra-block copying mode may be a sub-mode of an intra-mode, but is not limited to, and may indicate a mode separate from the intra-mode. In embodiments, the template matching prediction mode may include a template matching intra-prediction mode. Combination modes may include a combined inter-frame intra-prediction (CIIP) mode, wherein predictions from an intra-mode and predictions from an inter-mode are combined to perform prediction. Geometric partitioning modes may include a mode for partitioning a block to include directionality within the block. Geometric partitioning modes may perform prediction for each partitioned region of the block using inter-frame prediction or intra-frame prediction.

[0428] In embodiments of this disclosure, when the prediction mode of the current block is CIIP mode, the predictive encoder 3610 can combine inter-frame prediction and intra-frame prediction to perform prediction on the current block. For example, the predictive encoder 3610 can perform intra-frame prediction according to a planar mode. For example, the predictive encoder 3610 can determine the motion vector of a reference block for the current block. The predictive encoder 3610 can perform inter-frame prediction by using the motion vector. The predictive encoder 3610 can predict the current block by using a weighted sum of prediction blocks from inter-frame prediction and prediction blocks from intra-frame prediction. The weights can be determined based on whether neighboring blocks of the current block have already been intra-frame predicted (or inter-frame predicted).

[0429] In embodiments of this disclosure, when the prediction mode of the current block is a geometric partitioning mode, the predictive encoder 3610 can partition the current block and perform prediction. The predictive encoder 3610 can determine the partition angle and partition distance relative to the boundary where the partitioning is performed in the current block. The predictive encoder 3610 can partition the current block based on the partition angle and partition distance. The predictive encoder 3610 can predict the current block by performing inter-frame prediction or intra-frame prediction for each partition region in the current block. The predictive encoder 3610 can (i) perform intra-frame prediction for two partition regions, (ii) perform inter-frame prediction for one region and intra-frame prediction for another region, or (iii) perform inter-frame prediction for two partition regions.

[0430] In embodiments of this disclosure, when the prediction mode for the current block is template matching prediction mode, the predictive encoder 3610 can reconstruct the current block using a reference block. The generator 3620 can generate information related to whether or not the template matching prediction mode is used. The predictive encoder 3610 can determine whether to use the template matching prediction mode based on the obtained information. The reference block can be determined based on at least one of regions included in the current image or regions included in a previously decoded image. In embodiments of this disclosure, the predictive encoder 3610 can determine the reference block based on a cost function. The cost function can include at least one of Sum of Absolute Differences (SAD), Sum of Squared Differences (SSD), Sum of Absolute Transform Differences (SATD), Sum of Squared Errors (SSE), or Mean Removal SAD (MR-SAD). For example, the predictive encoder 3610 can determine the error between templates based on at least one cost function of SAD, SATD, SSE, and MR-SAD. When the cost function is SAD, the predictive decoder 3610 can determine the error based on the sum of the absolute values ​​of the differences between the individual samples of the template of the current block and the individual samples of the template of the candidate block. When the cost function is SSD, the predictive encoder 3610 can determine the error based on the sum of the squares of the differences between each sample of the template of the current block and each sample of the template of the candidate block. The cost function may include a function indicating the number of identical samples. The predictive encoder 3610 can determine the block with the smaller error from the candidate blocks as a reference block. The predictive encoder 3610 can determine the prediction block by performing template matching intra-frame prediction on the current image. In this disclosure, the process for determining the reference block of the current block based on the template can be referred to as template matching (TM). In this disclosure, the prediction performed on the current block based on template matching can be referred to as template matching prediction (TMP) or intra-frame template matching prediction.

[0431] In embodiments of this disclosure, when the prediction mode of the current block is intra-frame mode, the prediction encoder 3610 can determine the intra-frame prediction mode of the current block.

[0432] In embodiments of this disclosure, when the prediction mode of the current block is block copy mode, the prediction encoder 3610 can determine information about the block vector of the reference block.

[0433] In embodiments of this disclosure, the predictive encoder 3610 may perform intra-frame prediction or inter-frame prediction on the current block according to the prediction mode of the current block, and encode the current block by using a prediction block generated as a result of performing intra-frame prediction or inter-frame prediction.

[0434] In embodiments of this disclosure, when the prediction mode of the current block is block copy mode, the predictive encoder 3610 can determine the prediction block from a reference block. For example, the predictive encoder 3610 can determine that the prediction block is the same as the reference block or determine the prediction block by filtering the reference block. The reference block can be determined based on at least one of a region included in the current image or a region included in a previously decoded image.

[0435] In embodiments of this disclosure, when the prediction mode of the current block is a template matching prediction mode, the prediction encoder 3610 can reconstruct the current block using a reference block. The prediction encoder 3610 can determine the prediction mode by using the reference block.

[0436] The predictive encoder 3610 can perform deblocking filtering. The deblocking filter can improve image quality by smoothing the edges between blocks.

[0437] The predictive encoder 3610 can perform filtering on samples in the current block that have already undergone deblocking filtering by using a Sample Adaptive Offset (SAO) filter and / or a Bilateral Filter (BIF). The SAO filter and BIF can improve image quality by reducing the error between the reconstructed image and the original image. The SAO filter and BIF can perform filtering based on samples.

[0438] The predictive encoder 3610 can perform filtering using an adaptive loop filter (ALF). ALF improves image quality by reducing the error between the reconstructed image and the original image. ALF can perform filtering on a block-by-block basis.

[0439] In embodiments of this disclosure, the encoding of the current block may refer to the process by which the image decoding apparatus 2000 generates information that allows the reconstruction of the current block. The information generated through encoding can be included in the bitstream.

[0440] In embodiments of this disclosure, the predictive encoder 3610 can generate residual data corresponding to the difference between the predicted block and the current block. When the predicted block is determined to be the current block, residual data may not be generated.

[0441] Generator 3620 can generate a bitstream that includes the image encoding results. The bitstream may include the results of encoding the current block.

[0442] In embodiments of this disclosure, when the prediction mode for the current block is block copy mode, generator 3620 may generate a bitstream including information about the block vector of a reference block.

[0443] In embodiments of this disclosure, generator 3620 can send a bitstream to image decoding device 2000 via a network.

[0444] In embodiments of this disclosure, generator 3620 may store bitstreams in a data storage medium, including at least one of magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as optical disc (CD) read-only memory (ROM) (CD-ROM) and digital multifunction disc (DVD); or magneto-optical media such as optical floppy disks.

[0445] Generator 3620 can generate a bitstream that includes syntax elements generated through image encoding. Values ​​corresponding to the syntax elements can be included in the bitstream according to the hierarchical structure of the image.

[0446] Generator 3620 can obtain the binary bits included in the bit stream by entropy encoding the syntax elements.

[0447] In embodiments of this disclosure, the bitstream may include information about the prediction pattern of the current block in the current image.

[0448] In embodiments of this disclosure, when the prediction mode of the current block is intra-frame mode, the bitstream may include information indicating the intra-frame prediction mode of the current block.

[0449] In a prediction mode (e.g., intra-frame mode) that uses reference samples included in the current image, assuming continuity between neighboring samples of the current block and samples within the current block, a predicted block for the current block can be generated based on neighboring samples of the current block according to the intra-frame prediction mode. In embodiments of this disclosure, the predictive encoder 3610 can perform intra-frame prediction not only using neighboring samples of the current block included in the current image but also using spatial reference samples included in the current image. When using samples reconstructed earlier than the current block, the size of the residual data can be reduced by predicting samples of the current block not only based on samples directly adjacent to the current block but also based on samples far from the current block. In embodiments of this disclosure, the image encoding apparatus 3600 can perform intra-frame prediction based on reference blocks including unreconstructed samples, thereby increasing the range of regions that can be identified as reference blocks. In embodiments of this disclosure, the image encoding apparatus 3600 can enhance compression efficiency by improving the efficiency of intra-frame prediction.

[0450] In a prediction mode (e.g., an inter-frame mode) that includes reference samples in a reference image instead of the current image, assuming continuity between the current image and the reference image, a predicted block for the current block can be generated based on a reference block (or reference samples) in the reference image according to the prediction mode. In embodiments of this disclosure, the image coding apparatus 3600 can improve compression efficiency by increasing the efficiency of intra-frame prediction.

[0451] Image encoding apparatus 3600 can improve prediction accuracy by considering both reference blocks (or reference samples) included in the current image and reference blocks (or reference samples) included in images other than the current image. In embodiments of this disclosure, image encoding apparatus 3600 can enhance prediction accuracy by considering both the current image and images other than the current image.

[0452] Figure 37 This is a flowchart of an image encoding method according to an embodiment of the present disclosure.

[0453] In embodiments of this disclosure, the image encoding method may be executed by the image encoding apparatus 3600. For example, the processor of the image encoding apparatus 3600 may execute at least one instruction included in memory to cause the image encoding apparatus 3600 to perform each operation of the image encoding method.

[0454] In operation S3710, the image coding apparatus 3600 can perform prediction on the current block based on inter-frame prediction. The image coding apparatus 3600 can obtain a reference block included in the reference image. The image coding apparatus 3600 can obtain the reference block based on the motion vector of the current block. The image coding apparatus 3600 can obtain the prediction samples of the current block by performing inter-frame prediction based on the reference block.

[0455] Image encoding apparatus 3600 can obtain one or more reference blocks corresponding to one or more motion vectors of the current block. In embodiments of this disclosure, the current block may be a block included in a bidirectional prediction (B) frame (or B-strip). A block included in a B frame (or B-strip) can undergo inter-frame prediction based on up to two motion vectors and a reference index. When the current block is included in a B frame or B-strip, image encoding apparatus 3600 can obtain two reference blocks corresponding to two motion vectors of the current block. In embodiments of this disclosure, the current block may be a block included in a prediction (P) frame (or P-strip). A block included in a P frame (or P-strip) can undergo inter-frame prediction based on up to one motion vector and a reference index. When the current block is included in a P frame or P-strip, image encoding apparatus 3600 can obtain one reference block corresponding to one motion vector of the current block.

[0456] In operation S3720, the image encoding device 3600 may determine an intra-frame mode for the reference block based on at least one of neighboring samples of the reference block, internal samples of the reference block, or neighboring samples of the current block. The image encoding device 3600 may determine the intra-frame mode for the reference block based on temporal information or spatial information of the current block. Temporal information may include information about images other than the current image including the current block. Spatial information may include information about the current image including the current block.

[0457] In embodiments of this disclosure, the reference block may include a block that has already been used for inter-frame prediction of the current block. In embodiments, neighboring samples of the reference block (or the current block) may include adjacent and / or non-adjacent samples of the reference block (or the current block). The neighboring samples of the reference block may include samples to the left, top-left, and top of the reference block.

[0458] In embodiments of this disclosure, the image encoding apparatus 3600 may determine an intra-frame mode for a reference block based on neighboring samples and internal samples of the reference block. The image encoding apparatus 3600 may also determine the intra-frame mode for the reference block based on the result of predicting internal samples of the reference block using neighboring samples. In embodiments of this disclosure, the process by which the image encoding apparatus 3600 determines the intra-frame mode for the reference block based on neighboring samples and internal samples corresponds to the above-described reference... Figures 22 to 24 as well as Figure 29 and Figure 30 The process of determining the intra-frame mode for a reference block in the image decoding apparatus 2000 described herein.

[0459] In embodiments of this disclosure, the image encoding apparatus 3600 may determine an intra-frame mode for a reference block based on internal samples of the reference block. The image encoding apparatus 3600 may determine the intra-frame mode for the reference block based on multiple gradients determined for the internal samples of the reference block. In embodiments of this disclosure, the process by which the image encoding apparatus 3600 determines the intra-frame mode for the reference block based on the internal samples of the reference block corresponds to the above-described reference... Figures 25 to 28 as well as Figure 33 and Figure 34 The image decoding apparatus 2000 determines the intra-frame mode for the reference block.

[0460] In embodiments of this disclosure, the image encoding apparatus 3600 can determine the intra-frame mode for a reference block by using neighboring samples of the current sample. The neighboring samples of the current block can be samples included in the template region of the current block. In embodiments of this disclosure, the process by which the image encoding apparatus 3600 determines the intra-frame mode for the reference block based on the neighboring samples of the current block corresponds to the above-described reference... Figure 33 and Figure 34 The image decoding apparatus 2000 determines the intra-frame mode for the reference block.

[0461] In embodiments of this disclosure, the image encoding apparatus 3600 may determine one of two intra-frame modes obtained from two reference blocks as the intra-frame mode for the reference blocks. The image encoding apparatus 3600 may determine two intra-frame modes from corresponding two reference blocks and designate one of the determined intra-frame modes as the intra-frame mode for the reference blocks. Optionally, the image encoding apparatus 3600 may refer to all two reference blocks to determine a virtual intra-frame mode.

[0462] In operation S3730, the image encoding device 3600 can determine the transform kernel by using the intra-frame mode for the reference block. The image encoding device 3600 can determine a transform kernel set including multiple transform kernels corresponding to the intra-frame mode for the reference block. The image encoding device 3600 can determine the type of the transform kernel set based on the intra-frame mode for the reference block. For example, the range of the direction index of the intra-frame mode for the reference block may correspond to one of the multiple transform kernel sets. A transform kernel set may include multiple transform kernels. For example, a first transform kernel set may include a first transform kernel and a second transform kernel, and a second transform kernel set may include a third transform kernel and a fourth transform kernel. In embodiments, depending on the type of transform, there are one or more (e.g., N, where N is 1 or greater) transform kernel sets, and each transform kernel set may include one or more (e.g., M, where M is 1 or greater) transform kernels.

[0463] In embodiments of this disclosure, the number of transform cores included in each transform core set can be determined based on the sum of the transform coefficients. For example, when the sum of the transform coefficients is less than a predetermined first value, the number of transform cores can be N1 (e.g., N1=1); when the sum of the transform coefficients is greater than a predetermined first value and less than a predetermined second value, the number of transform cores can be N2 (e.g., N1=4); when the sum of the transform coefficients is greater than a predetermined second value, the number of transform cores can be N3 (e.g., N1=6).

[0464] In embodiments of this disclosure, the image encoding apparatus 3600 can determine a transform kernel set based on the size of the current block and the intra-frame mode of the reference block. The transform kernel set can be predetermined based on the size of the current block. For example, M transform kernel sets can be predetermined for each size of the current block.

[0465] The image encoding apparatus 3600 can select one of a plurality of transform kernels as the transform kernel for the current block. The image encoding apparatus 3600 can determine one of a plurality of transform kernels included in a set of transform kernels. In embodiments of this disclosure, the image encoding apparatus 3600 can generate a bitstream including index information indicating the transform kernel of the current block among the plurality of transform kernels.

[0466] In embodiments of this disclosure, the transform core may involve at least one of a core for a first transform, a core for a second transform, or an inseparable first transform core that combines the first and second transforms.

[0467] In embodiments of this disclosure, the image encoding apparatus 3600 may determine a transform kernel for a first transform by using an intra-frame mode for a reference block. The transform kernel for the first transform may include a Discrete Cosine Transform (DCT) kernel and / or a Discrete Sine Transform (DST) kernel. The first transform may be applied in the vertical and / or horizontal directions. The image encoding apparatus 3600 may generate a bitstream including information indicating whether it is a vertical or horizontal transform. The first transform may be referred to as Multiple Transform Selection (MTS). The transform kernel for the first transform may be referred to as a core transform kernel.

[0468] In embodiments of this disclosure, the image encoding apparatus 3600 can determine the transform kernel for the second transform by using an intra-frame mode for a reference block. The second transform can be applied only to a portion of the residual signal that has already undergone the first transform. For example, the second transform can be performed only on the left or upper left region of the block. The second transform is performed without separating the vertical and horizontal directions. The second transform may be referred to as the Low Frequency Inseparable Transform (LFNST).

[0469] In embodiments of this disclosure, the image encoding apparatus 3600 can determine the transform kernel for the inseparable first transform by using an intra-frame mode for a reference block. The image encoding apparatus 3600 may not perform an additional second transform on blocks that have already undergone the inseparable first transform. The inseparable first transform may take the form of a combination of the first transform and the second transform. For example, the transform kernel for the inseparable first transform may be a combination of a transform kernel for the first transform and a transform kernel for the second transform. The inseparable first transform is performed without separating the vertical and horizontal directions.

[0470] In operation S3740, the image encoding device 3600 can perform a transformation on the current block using a transform kernel. The image encoding device 3600 can perform a transformation by applying a selected transform kernel to the current block. The image encoding device 3600 can obtain transform coefficients by performing a transformation based on the transform kernel.

[0471] The image encoding device 3600 can generate transform coefficients by applying a determined transform kernel for the first transform to the residual signal (or residual data).

[0472] The image encoding apparatus 3600 generates transform coefficients by applying a transform kernel for the second transform to the residual signal that has undergone the first transform. The image encoding apparatus 3600 can determine whether to perform the second transform for each block.

[0473] The image encoding device 3600 can generate transform coefficients by applying a transform kernel for a separable first transform to the residual signal.

[0474] When the prediction mode of the current block is inter-frame mode, the image coding apparatus 3600 can determine the transform kernel set for the current block by determining the intra-frame mode used for the reference block. The image coding apparatus 3600 can derive the intra-frame mode representing the internal direction for the reference block using the reconstructed reference block, and even when the current block is in inter-frame mode, it can improve coding efficiency (or transform efficiency) by applying the derived intra-frame mode for the reference block to the current block. In other words, by determining the intra-frame mode for the reference block, the image coding apparatus 3600 can select a suitable transform kernel from various types of transform kernels for the current block, even in inter-frame mode, just as it would in intra-frame mode, thereby improving video compression efficiency. In embodiments of this disclosure, even when the prediction mode of the current block is inter-frame mode, a second transform and an inseparable first transform can be applied, and various transform kernels for the first transform can be applied. By applying the same direction-dependent transform kernel as the second transform, residuals at block boundaries can be effectively compressed. Such advantages are merely examples, and the advantages and purposes of this disclosure are not limited to those described herein.

[0475] Figure 38 This is a flowchart of an image encoding method according to an embodiment of the present disclosure.

[0476] In embodiments of this disclosure, the image encoding method may be executed by the image encoding apparatus 3600. For example, the processor of the image encoding apparatus 3600 may execute at least one instruction included in memory to cause the image encoding apparatus 3600 to perform each operation of the image encoding method.

[0477] In operation S3810, the image encoding apparatus 3600 can obtain a reference block in the reference image. In embodiments of this disclosure, the reference block can be determined based on the motion vector of the current block. The reference block may include a block used for inter-frame prediction of the current block.

[0478] In operation S3820, the image encoding device 3600 can determine the intra-frame mode for the reference block based on at least one of a plurality of neighboring samples and a plurality of internal samples of the reference block.

[0479] In embodiments of this disclosure, adjacent samples of the reference block may include samples within a specific distance from the reference block. Nearby samples of the reference block may include at least one of adjacent and non-adjacent samples of the reference block. Nearby samples of the reference block may include samples located above the reference block, samples located to the left of the reference block, and / or samples located to the upper left of the reference block.

[0480] In embodiments of this disclosure, the image encoding apparatus 3600 can predict internal samples of a reference block using multiple neighboring samples. For example, the image encoding apparatus 3600 can determine predicted samples based on neighboring samples of the reference block (which serve as reference samples) according to multiple intra-frame modes (e.g., intra-frame direction prediction modes 2 to 66 or all available intra-frame prediction modes). The image encoding apparatus 3600 can determine an error by comparing the predicted samples with internal samples of the reference block. The image encoding apparatus 3600 can determine an intra-frame mode for the reference block based on the error. For example, the image encoding apparatus 3600 can determine the intra-frame mode with the smallest error among multiple intra-frame modes as the intra-frame mode for the reference block.

[0481] In operation S3830, the image encoding apparatus 3600 may determine the transform kernel of the current block from one or more transform kernels based on the intra-frame mode used for the reference block. In embodiments of this disclosure, operation S3830 may correspond to operation S3730.

[0482] In operation S3840, the image encoding device 3600 can perform a transformation on the current block using a transform kernel. In embodiments of this disclosure, operation S2240 may correspond to operation S3740.

[0483] The methods or embodiments presented in this disclosure can be used individually or in any combination in any order. Furthermore, the methods (or embodiments), encoders, and decoders can each be implemented with their own processing circuitry (e.g., one or more processors or one or more integrated circuits). In embodiments, one or more processors execute a program stored in a non-transitory computer-readable medium.

[0484] The image coding apparatus according to embodiments of the present disclosure can derive an intra-frame mode representing the internal orientation of a reference block using a reconstructed reference block, and can improve coding efficiency (or transform efficiency) even when the current block is in inter-frame mode by applying the derived intra-frame mode for the reference block to the current block. In other words, by determining the intra-frame mode for the reference block, even in inter-frame mode, the image coding apparatus can select a suitable transform kernel from various types of transform kernels for the current block as in intra-frame mode, thereby improving video compression efficiency. Even when the prediction mode of the current block is inter-frame mode, a second transform and an inseparable first transform can be applied, and various transform kernels for the first transform can be applied. Residual data at block boundaries can be effectively compressed by applying the same direction-dependent transform kernel as the second transform. However, the technical effects of the image decoding method according to embodiments of the present disclosure are not limited to the above description, but may include technical features derived from the present disclosure.

[0485] According to embodiments of this disclosure, an image decoding method is provided. The image decoding method may include: obtaining a reference block in a reference image. The image decoding method may include: determining an intra-frame mode for the reference block based on at least one of a plurality of neighboring samples and a plurality of internal samples of the reference block. The image decoding method may include: determining a transform kernel for a current block from one or more transform kernels using the intra-frame mode for the reference block. The image decoding method may include: performing an inverse transform on the current block using the transform kernel.

[0486] In embodiments of this disclosure, one or more transform cores may include at least one of: a first transform core associated with multiple transform selection (MTS), a second transform core associated with low-frequency non-separable transform (LFNST), or a third transform core associated with non-separable primary transform (NSPT).

[0487] In embodiments of this disclosure, determining the intra-mode for a reference block may include performing intra-prediction based on one or more intra-prediction modes using a plurality of neighboring samples. Determining the intra-mode for a reference block may also include determining the intra-mode for the reference block based on the error between intra-prediction samples and internal samples of the reference block.

[0488] In embodiments of this disclosure, determining the intra-frame mode for a reference block may include: determining a plurality of gradients using a plurality of internal samples of the reference block. Determining the intra-frame mode for the reference block may include: determining a plurality of directional intra-frame modes corresponding to the plurality of gradients. Determining the intra-frame mode for the reference block may include: selecting an intra-frame mode for the reference block from the determined plurality of directional intra-frame modes.

[0489] In embodiments of this disclosure, determining the intra-frame mode for a reference block may include: determining a first candidate intra-frame mode based on at least one of a plurality of neighboring samples and a plurality of internal samples of the reference block. Determining the intra-frame mode for a reference block may include: determining a second candidate intra-frame mode based on at least one of a plurality of neighboring samples of a second reference block and a plurality of internal samples of a second reference block. Determining the intra-frame mode for a reference block may include: selecting one of the first candidate intra-frame mode and the second candidate intra-frame mode as the intra-frame mode for the reference block.

[0490] In embodiments of this disclosure, selecting one of a first candidate intra-frame mode and a second candidate intra-frame mode as the intra-frame mode for a reference block may include: determining a first error by predicting a plurality of internal samples of the reference block based on a plurality of neighboring samples of the reference block according to the first candidate intra-frame mode. Selecting one of the first candidate intra-frame mode and the second candidate intra-frame mode as the intra-frame mode for a reference block may include: determining a second error by predicting a plurality of internal samples of the second reference block based on a plurality of neighboring samples of the second candidate intra-frame mode. Selecting one of the first candidate intra-frame mode and the second candidate intra-frame mode as the intra-frame mode for a reference block may include: determining the intra-frame mode for the reference block from the first candidate intra-frame mode and the second candidate intra-frame mode based on the first error and the second error.

[0491] In embodiments of this disclosure, selecting one of a first candidate intra-frame mode and a second candidate intra-frame mode as the intra-frame mode for a reference block may include: obtaining a first quantization parameter of the reference block and a second quantization parameter of the reference block. Selecting one of the first candidate intra-frame mode and the second candidate intra-frame mode as the intra-frame mode for the reference block may include: selecting the second candidate intra-frame mode as the intra-frame mode for the reference block when the first quantization parameter is equal to or greater than the second quantization parameter. Selecting one of the first candidate intra-frame mode and the second candidate intra-frame mode as the intra-frame mode for the reference block may include: selecting the first candidate intra-frame mode as the intra-frame mode for the reference block when the first quantization parameter is less than the second quantization parameter.

[0492] In embodiments of this disclosure, the step of selecting one of a first candidate intra-frame mode and a second candidate intra-frame mode as the intra-frame mode for a reference block may include: obtaining a first POC value of a reference image and a second POC value of a second reference image. Selecting one of the first candidate intra-frame mode and the second candidate intra-frame mode as the intra-frame mode for the reference block may include: selecting the first candidate intra-frame mode as the intra-frame mode for the reference block when the difference between the POC value of the current image and the first POC value is less than the difference between the POC value of the current image and the second POC value. Selecting one of the first candidate intra-frame mode and the second candidate intra-frame mode as the intra-frame mode for the reference block may include: selecting the second candidate intra-frame mode as the intra-frame mode for the reference block when the difference between the POC value of the current image and the first POC value is equal to or greater than the difference between the POC value of the current image and the second POC value.

[0493] In embodiments of this disclosure, selecting one of a first candidate intra-frame mode and a second candidate intra-frame mode as the intra-frame mode for a reference block may include: obtaining bidirectional prediction weights for the reference block and the second reference block. Selecting one of the first candidate intra-frame mode and the second candidate intra-frame mode as the intra-frame mode for a reference block may include: selecting the first candidate intra-frame mode as the intra-frame mode for the reference block when the weight for the reference block is equal to or greater than the weight for the second reference block. Selecting one of the first candidate intra-frame mode and the second candidate intra-frame mode as the intra-frame mode for a reference block may include: selecting the second candidate intra-frame mode as the intra-frame mode for the reference block when the weight for the reference block is less than the weight for the second reference block.

[0494] In embodiments of this disclosure, selecting one of a first candidate intra-frame mode and a second candidate intra-frame mode as the intra-frame mode for the reference block may include: when the type of the strip including the reference block is I-strip, selecting the first candidate intra-frame mode as the intra-frame mode for the reference block. Selecting one of the first candidate intra-frame mode and the second candidate intra-frame mode as the intra-frame mode for the reference block may include: when the type of the strip including the second reference block is I-strip, selecting the second candidate intra-frame mode as the intra-frame mode for the reference block.

[0495] In embodiments of this disclosure, determining the intra-frame mode for a reference block may include: determining a plurality of first gradients using a plurality of internal samples of the reference block. Determining the intra-frame mode for a reference block may include: determining a plurality of second gradients using a plurality of internal samples of a second reference block of a second reference image other than the reference image. Determining the intra-frame mode for a reference block may include: determining a plurality of directional intra-frame modes corresponding to the plurality of first gradients and the plurality of second gradients. Determining the intra-frame mode for a reference block may include: determining the intra-frame mode for the reference block from the determined plurality of directional intra-frame modes.

[0496] According to embodiments of this disclosure, an image decoding apparatus is provided. The image decoding apparatus may include a memory storing one or more instructions and at least one processor. The at least one processor may execute one or more instructions to obtain a reference block in a reference image. The at least one processor may execute one or more instructions to determine an intra-frame mode for the reference block based on at least one of a plurality of neighboring samples and a plurality of internal samples of the reference block. The at least one processor may execute one or more instructions to determine a transform kernel for a current block from one or more transform kernels using the intra-frame mode for the reference block. The at least one processor may execute the one or more instructions to perform an inverse transform on the current block using the transform kernel.

[0497] In embodiments of this disclosure, one or more transform cores may include at least one of a first transform core associated with multiple transform selection, a second transform core associated with a low-frequency non-separable transform, or a third transform core associated with a non-separable first transform.

[0498] In embodiments of this disclosure, at least one processor may execute the one or more instructions to perform intra-prediction based on one or more intra-prediction modes using a plurality of neighboring samples. At least one processor may execute one or more instructions to determine an intra-prediction mode for a reference block based on the error between intra-prediction samples and internal samples of the reference block.

[0499] In embodiments of this disclosure, at least one processor may execute one or more instructions to determine multiple gradients using multiple internal samples of a reference block. At least one processor may execute one or more instructions to determine multiple directional intra-modes corresponding to the multiple gradients. At least one processor may execute one or more instructions to select an intra-mode for the reference block from the determined multiple directional intra-modes.

[0500] In embodiments of this disclosure, at least one processor may execute one or more instructions to determine a first candidate intra-mode based on at least one of a plurality of neighboring samples and a plurality of internal samples of the reference block. At least one processor may execute one or more instructions to determine a second candidate intra-mode based on at least one of a plurality of neighboring samples of a second reference block and a plurality of internal samples of the second reference block. At least one processor may execute one or more instructions to select one of the first candidate intra-mode and the second candidate intra-mode as the intra-mode for the reference block.

[0501] In embodiments of this disclosure, at least one processor may execute one or more instructions to determine a first error by predicting multiple internal samples of a reference block based on multiple neighbor samples of the reference block according to a first candidate intra-frame mode. At least one processor may execute one or more instructions to determine a second error by predicting multiple internal samples of a second reference block based on multiple neighbor samples of a second candidate intra-frame mode. The at least one processor may execute one or more instructions to determine an intra-frame mode for the reference block from the first candidate intra-frame mode and the second candidate intra-frame mode based on the first error and the second error.

[0502] In embodiments of this disclosure, at least one processor may execute one or more instructions to obtain a first quantization parameter for a reference block and a second quantization parameter for a second reference block. The at least one processor may execute the one or more instructions to select a second candidate intra-mode as the intra-mode for the reference block when the first quantization parameter is equal to or greater than the second quantization parameter. The at least one processor may also execute the one or more instructions to select a first candidate intra-mode as the intra-mode for the reference block when the first quantization parameter is less than the second quantization parameter.

[0503] In embodiments of this disclosure, at least one processor may execute one or more instructions to obtain a first POC value of a reference image and a second POC value of a second reference image. At least one processor may execute one or more instructions to select a first candidate intra-frame mode as the intra-frame mode for the reference block when the difference between the POC value of the current image and the first POC value is less than the difference between the POC value of the current image and the second POC value. At least one processor may execute one or more instructions to select a second candidate intra-frame mode as the intra-frame mode for the reference block when the difference between the POC value of the current image and the first POC value is equal to or greater than the difference between the POC value of the current image and the second POC value.

[0504] In embodiments of this disclosure, at least one processor may execute one or more instructions to obtain bidirectional prediction weights for a reference block and a second reference block. At least one processor may execute one or more instructions to: select a first candidate intra-mode as the intra-mode for the reference block when the weights for the reference block are equal to or greater than the weights for the second reference block. At least one processor may execute one or more instructions to: select a second candidate intra-mode as the intra-mode for the reference block when the weights for the reference block are less than the weights for the second reference block.

[0505] In embodiments of this disclosure, at least one processor may execute one or more instructions to select a first candidate intra-mode as the intra-mode for the reference block when the type of the strip including the reference block is I-strip. At least one processor may execute one or more instructions to select a second candidate intra-mode as the intra-mode for the reference block when the type of the strip including the second reference block is I-strip.

[0506] In embodiments of this disclosure, at least one processor may execute one or more instructions to determine a plurality of first gradients using a plurality of internal samples of a reference block. At least one processor may execute one or more instructions to determine a plurality of second gradients using a plurality of internal samples of a second reference block of a second reference image other than a reference image. At least one processor may execute one or more instructions to determine a plurality of directional intra-modes corresponding to the plurality of first gradients and the plurality of second gradients. At least one processor may execute one or more instructions to determine an intra-mode for the reference block from the determined plurality of directional intra-modes.

[0507] According to embodiments of this disclosure, an image encoding method is provided. The image encoding method may include: obtaining a reference block in a reference image. The image encoding method may include: determining an intra-frame mode for the reference block based on at least one of a plurality of neighboring samples and a plurality of internal samples of the reference block. The image encoding method may include: determining a transform kernel for a current block from one or more transform kernels by using the intra-frame mode for the reference block. The image encoding method may include: performing a transform on the current block using the transform kernel.

[0508] In embodiments of this disclosure, one or more transform cores may include at least one of a first transform core associat...

Claims

1. An image decoding method, comprising: Obtain the reference block in the reference image; The intra-frame mode for the reference block is determined based on at least one of multiple neighboring samples and multiple internal samples of the reference block. The transform kernel of the current block is determined from one or more transform kernels by using the intra-frame mode used for the reference block; as well as Perform an inverse transformation on the current block using a transformation kernel.

2. The image decoding method according to claim 1, wherein, The one or more transform cores include at least one of a first transform core associated with Multi-Transform Selection (MTS), a second transform core associated with Low-Frequency Inseparable Transform (LFNST), or a third transform core associated with Inseparable Primary Transform (NSPT).

3. The image decoding method according to any one of claims 1 and 2, wherein, Determining the intra-frame mode for the reference block includes: Intra-frame prediction is performed using the plurality of neighboring samples according to one or more intra-frame prediction modes; and The intra-frame mode for the reference block is determined based on the error between the intra-predicted samples and the internal samples of the reference block.

4. The image decoding method according to any one of claims 1 to 3, wherein, Determining the intra-frame mode for the reference block includes: Multiple gradients are determined by using the multiple internal samples of the reference block; Determine multiple intra-frame modes corresponding to the multiple gradients; and Select the intra-frame mode for the reference block from the determined plurality of directional intra-frame modes.

5. The image decoding method according to any one of claims 1 to 4, wherein, Determining the intra-frame mode for the reference block includes: A first candidate intra-frame mode is determined based on at least one of the plurality of neighboring samples of the reference block and the plurality of internal samples of the reference block. A second candidate intra-frame mode is determined based on at least one of multiple neighboring samples of the current block and multiple internal samples of the second reference block; and Choose one of the first candidate intra-frame mode and the second candidate intra-frame mode as the intra-frame mode for the reference block.

6. The image decoding method according to claim 5, wherein, Selecting one of the first candidate intra-frame mode and the second candidate intra-frame mode as the intra-frame mode for the reference block includes: The first error is determined by performing prediction on the plurality of internal samples of the reference block using the plurality of neighboring samples of the reference block according to the first candidate intra-frame mode. The second error is determined by performing prediction on the plurality of internal samples of the second reference block using the plurality of neighboring samples of the second reference block according to the second candidate intra-frame mode; and The intra-mode for the reference block is determined from the first candidate intra-mode and the second candidate intra-mode based on the first error and the second error.

7. The image decoding method according to any one of claims 5 and 6, wherein, Selecting one of the first candidate intra-frame mode and the second candidate intra-frame mode as the intra-frame mode for the reference block includes: Obtain the first quantization parameters of the reference block and the second quantization parameters of the second reference block; When the first quantization parameter is equal to or greater than the second quantization parameter, the second candidate intra-frame mode is selected as the intra-frame mode for the reference block; and When the first quantization parameter is less than the second quantization parameter, the first candidate intra-frame mode is selected as the intra-frame mode for the reference block.

8. The image decoding method according to any one of claims 5 to 7, wherein, Selecting one of the first candidate intra-frame mode and the second candidate intra-frame mode as the intra-frame mode for the reference block includes: Obtain the first POC value of the reference image and the second POC value of the second reference image; When the difference between the current image's POC value and the first POC value is less than the difference between the current image's POC value and the second POC value, the first candidate intra-frame mode is selected as the intra-frame mode for the reference block; and When the difference between the POC value of the current image and the first POC value is equal to or greater than the difference between the POC value of the current image and the second POC value, the second candidate intra-frame mode is selected as the intra-frame mode for the reference block.

9. The image decoding method according to any one of claims 5 to 8, wherein, Selecting one of the first candidate intra-frame mode and the second candidate intra-frame mode as the intra-frame mode for the reference block includes: Obtain bidirectional prediction weights for the reference block and the second reference block; When the weight used for the reference block is equal to or greater than the weight used for the second reference block, the first candidate intra-frame mode is selected as the intra-frame mode for the reference block; and When the weight used for the reference block is less than the weight used for the second reference block, the second candidate intra-mode is selected as the intra-mode used for the reference block.

10. The image decoding method according to any one of claims 5 to 9, wherein, Selecting one of the first candidate intra-frame mode and the second candidate intra-frame mode as the intra-frame mode for the reference block includes: When the type of the strip including the reference block is I stripe, the first candidate intra mode is selected as the intra mode for the reference block; and When the type of the strip including the second reference block is I stripe, the second candidate intra mode is selected as the intra mode for the reference block.

11. The image decoding method according to any one of claims 1 to 10, wherein, Determining the intra-frame mode for the reference block includes: Multiple first gradients are determined by using the multiple internal samples of the reference block; Multiple second gradients are determined by using multiple internal samples of a second reference block of a second reference image other than the reference image; Determine multiple directional intra-frame modes corresponding to the multiple first gradients and the multiple second gradients; and The intra-frame mode for the reference block is determined from the determined plurality of directional intra-frame modes.

12. An image decoding apparatus, comprising: Memory, which stores one or more instructions; and At least one processor, including processing circuitry, The at least one processor is configured to execute the one or more instructions to cause the image decoding device to perform the following operations: Obtain the reference block in the reference image. The intra-frame mode for the reference block is determined based on at least one of multiple neighboring samples and multiple internal samples of the reference block. The transform kernel of the current block is determined from one or more transform kernels by using the intra-frame mode used for the reference block; Perform an inverse transformation on the current block using a transformation kernel.

13. An image encoding method, comprising: Obtain the reference block in the reference image; The intra-frame mode for the reference block is determined based on at least one of multiple neighboring samples and multiple internal samples of the reference block. The transform kernel of the current block is determined from one or more transform kernels by using the intra-frame mode used for the reference block; as well as Perform a transformation on the current block using a transformation kernel.

14. The image encoding method according to claim 13, wherein, The one or more transform cores include at least one of a first transform core associated with Multi-Transform Selection (MTS), a second transform core associated with Low-Frequency Inseparable Transform (LFNST), or a third transform core associated with Inseparable Primary Transform (NSPT).

15. A computer-readable storage medium for storing a bitstream encoded by an image encoding method, the image encoding method comprising: Obtain the reference block in the reference image; The intra-frame mode for the reference block is determined based on at least one of multiple neighboring samples and multiple internal samples of the reference block. The transform kernel of the current block is determined from one or more transform kernels by using the intra-frame mode used for the reference block; as well as Perform a transformation on the current block using a transformation kernel.

Images