Apparatus for encoding and decoding images and method for encoding and decoding images thereof

Abstract

An apparatus for encoding and decoding an image and a method for encoding and decoding an image thereof are provided. An image decoding method may include: obtaining information indicating a plurality of first reference image lists for an image sequence including a current image from a sequence parameter set of a bitstream; obtaining, from a group header of the bitstream, an indicator for a current block group including a current block in a current image; obtaining a second reference image list by modifying the first reference image list indicated by the indicator among the plurality of first reference image lists to the second reference image list; and performing prediction decoding on a lower layer block of the current block based on the second reference picture included in the second reference picture list.

Description

[0001] This application is a divisional application of the invention patent application filed on February 28, 2020, with application number "202080017337.7" and titled "Apparatus for Encoding and Decoding Images and Method Therefor for Encoding and Decoding Images".

[0002] This application is the national phase application of international application number PCT / KR2020 / 002924, filed on February 28, 2020, which claims priority to U.S. Provisional Application No. 62 / 811,764, filed on February 28, 2019, with the United States Patent and Trademark Office, the entire disclosure of which is incorporated herein by reference. Technical Field

[0003] This disclosure relates to image encoding and decoding. More specifically, this disclosure relates to a method and apparatus for encoding an image using a hierarchical structure of the image, and a method and apparatus for decoding an image. Background Technology

[0004] In image encoding and decoding, an image can be divided into blocks, and each block can be predictively encoded and predictedly decoded via inter-frame prediction or intra-frame prediction.

[0005] Inter-frame prediction is a method for compressing images by removing temporal redundancy between them, and a representative example of inter-frame prediction is motion estimation coding. In motion estimation coding, blocks of the current image are predicted using at least one reference image. A reference block most similar to the current block is searched within a predetermined search range using a predetermined evaluation function. The current block is predicted based on the reference block, and the predicted block generated as a result is subtracted from the current block to generate a residual block, which is then encoded. Here, to perform prediction more accurately, interpolation can be performed on the reference image to generate sub-pixel units (sub-pel units), and inter-frame prediction can be performed based on these sub-pel unit pixels.

[0006] In codecs such as H.264 Advanced Video Coding (AVC) and High-Efficiency Video Coding (HEVC), in order to predict the motion vector of the current block, the motion vectors of previously encoded blocks adjacent to the current block or blocks included in previously encoded frames are used as the predicted motion vectors for the current block. A differential motion vector, representing the difference between the motion vector of the current block and the predicted motion vector, is sent to the decoder as a signal using a predetermined method. Summary of the Invention

[0007] The technical objective of the image encoding and decoding apparatus and the image encoding and decoding method according to the embodiments is to encode and decode images at a low bit rate by using the hierarchical structure of the images.

[0008] According to one aspect of this disclosure, an image decoding method is provided, comprising: obtaining information from a sequence parameter set of a bitstream indicating a plurality of first reference image lists for an image sequence including a current image; obtaining an indicator from a group header of the bitstream for a current block group including a current block in the current image; obtaining a second reference image list by modifying the first reference image list indicated by the indicator in the plurality of first reference image lists to a second reference image list; and performing predictive decoding on lower-level blocks of the current block based on the second reference images included in the second reference image list.

[0009] The image decoding method may further include: predictively decoding the lower-level blocks included in the next block group in the current image based on another first reference image list (excluding the first reference image list indicated by the indicator) and a second reference image list in the plurality of first reference image lists.

[0010] The first reference image list indicated by the indicator may include only reference images of the first type. The step of obtaining the second reference image list may include modifying the first reference image list indicated by the indicator into a second reference image list by adding reference images of the second type indicated by a picture sequence count (POC) related value obtained from the group head to the first reference image list indicated by the indicator.

[0011] The step of obtaining a second reference image list may include modifying the first reference image list indicated by the indicator into a second reference image list by changing the order of one or more first reference images included in the first reference image list indicated by the indicator.

[0012] The first list of reference images indicated by the indicator may include reference images of a first type and reference images of a second type. The step of obtaining a second list of reference images may include: obtaining the second list of reference images by excluding reference images of the second type from the first list of reference images indicated by the indicator.

[0013] The first list of reference images indicated by the indicator may include reference images of a first type and reference images of a second type. The step of obtaining the second list of reference images may include: excluding reference images of the second type from the first list of reference images indicated by the indicator, and adding reference images of the second type, indicated by a Picture Order Count (POC) related value obtained from the group header, to the first list of reference images indicated by the indicator.

[0014] The step of obtaining a second reference image list may include: obtaining a second reference image list that includes reference images of a first type included in any one of the reference image lists indicated by the indicator and reference images of a second type included in another reference image list indicated by the indicator.

[0015] A higher index can be assigned to a reference image of the other type of reference image compared to a reference image of the first type or the second type.

[0016] The image decoding method may further include: obtaining order information of a first type of reference image and a second type of reference image from the group header, wherein an index of the order information is assigned to the first type of reference image and the second type of reference image.

[0017] The image decoding method may further include: obtaining from the group header the difference between the picture order count (POC) related value of one or more first reference images included in the first reference image list indicated by the indicator and the POC related value of one or more second reference images to be included in the second reference image list, wherein the step of obtaining the second reference image list may include: obtaining the second reference image list by replacing the one or more first reference images in the first reference images based on the difference.

[0018] The image decoding method may further include: determining multiple blocks in the current image; obtaining address information for the block group from the bit stream; and configuring the block group, each including one or more blocks, in the current image according to the address information, wherein the current block can be any one of the multiple blocks, and the current block group is any block group among the block groups.

[0019] The address information may include the identification information of the lower right block among the plurality of blocks included in each block group. The step of configuring the block group may include: configuring a first block group, wherein the first block group includes a top-left block located on the top left side among the plurality of blocks and a lower right block indicated by the identification information of the lower right block of the first block group; identifying a top-left block of a second block group based on the identification information of the plurality of blocks included in the first block group; and configuring a second block group, wherein the second block group includes a lower right block indicated by the identification information of the lower right block of the second group and the identified top-left block.

[0020] The image decoding method may further include: obtaining identification information from the group header or picture parameter set of the bitstream, indicating a set of post-processing parameters that are applied to the brightness mapping of prediction samples for lower-level blocks obtained as a result of prediction decoding; and performing brightness mapping on the prediction samples according to the set of post-processing parameters indicated by the identification information.

[0021] According to another aspect of this disclosure, an image decoding apparatus is provided, comprising: at least one memory storing one or more instructions; and at least one processor configured to execute the one or more instructions to perform the following operations: obtaining information from a sequence parameter set of a bitstream indicating a plurality of first reference image lists for an image sequence including a current image; obtaining from a group header of the bitstream an indicator for a current block group including a current block in the current image; obtaining a second reference image list by modifying the first reference image list indicated by the indicator in the plurality of first reference image lists to a second reference image list; and performing predictive decoding of lower-level blocks of the current block based on reference images included in the second reference image list.

[0022] The at least one processor is further configured to modify the first reference image list indicated by the indicator into a second reference image list by adding a second type of reference image indicated by a picture sequence count (POC) related value obtained from the group head to the first reference image list indicated by the indicator.

[0023] The at least one processor is further configured to modify the first reference image list indicated by the indicator into a second reference image list by changing the order of one or more first reference images included in the first reference image list indicated by the indicator.

[0024] The at least one processor is further configured to modify the first reference image list indicated by the indicator into a second reference image list by removing at least one reference image from a plurality of reference images of different types from the first reference image list indicated by the indicator.

[0025] According to another aspect of this disclosure, an image coding method is provided, comprising: constructing a plurality of first reference image lists for an image sequence including a current image; selecting from the plurality of first reference image lists a first reference image list for a current block group including a current block in the current image; obtaining a second reference image list by modifying the selected first reference image list to a second reference image list; and performing predictive coding on lower-level blocks of the current block based on reference images included in the second reference image list.

[0026] The image encoding and decoding method according to the embodiments may include encoding and decoding the image at a low bit rate by using the hierarchical structure of the image, and the image encoding and decoding device according to the embodiments may perform the image encoding and decoding method.

[0027] However, the effects that can be achieved by the image encoding and decoding apparatus and the image encoding and decoding method according to the embodiments are not limited to those described above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description. Attached Figure Description

[0028] The above and other aspects, features and advantages of specific embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings.

[0029] Figure 1 This is a block diagram of an image decoding device according to an embodiment.

[0030] Figure 2 This is a block diagram of an image encoding device according to an embodiment.

[0031] Figure 3 The illustration shows a process performed by an image decoding device according to an embodiment to determine at least one coding unit by dividing the current coding unit.

[0032] Figure 4 The illustration shows a process performed by an image decoding device according to an embodiment to determine at least one coding unit by dividing a non-square coding unit.

[0033] Figure 5 The illustration shows a process performed by an image decoding device according to an embodiment, which divides the encoding unit based on at least one of block shape information and segmentation shape pattern information.

[0034] Figure 6 A method for determining a predetermined coding unit from an odd number of coding units, performed by an image decoding device according to an embodiment, is shown.

[0035] Figure 7 This illustrates the order in which the image decoding device processes the plurality of coding units when it determines the plurality of coding units by dividing the current coding unit, according to an embodiment.

[0036] Figure 8 The illustration shows the process by which an image decoding device determines that the current encoding unit is divided into an odd number of encoding units when the encoding units cannot be processed in a predetermined order, according to an embodiment.

[0037] Figure 9 The illustration shows a process performed by an image decoding device according to an embodiment to determine at least one coding unit by dividing a first coding unit.

[0038] Figure 10 The embodiment shows that when a second encoding unit with a non-square shape, determined by the image decoding device dividing the first encoding unit, satisfies predetermined conditions, the shape into which the second encoding unit can be divided is limited.

[0039] Figure 11 The illustration shows the process performed by the image decoding device to divide the square coding unit when the division shape pattern information cannot indicate that the square coding unit is divided into four square coding units, according to an embodiment.

[0040] Figure 12 This illustrates that, according to an embodiment, the processing order among multiple coding units can be changed based on the process of dividing the coding units.

[0041] Figure 13 The illustration shows a process for determining the depth of a coding unit when the shape and size of the coding unit change, as the coding unit is recursively divided to determine a plurality of coding units, according to an embodiment.

[0042] Figure 14 The diagram illustrates a depth that can be determined based on the shape and size of the coding unit, and a partial index (PID) used to distinguish the coding units, according to an embodiment.

[0043] Figure 15 The illustration shows how multiple encoding units are determined based on multiple predetermined data units included in the screen, according to an embodiment.

[0044] Figure 16 The illustration shows an encoding unit that can be determined for each frame when the combination of shapes into which the encoding unit can be divided is different for each frame, according to an embodiment.

[0045] Figure 17 Various shapes of coding units, which can be determined based on partition shape pattern information that can be represented as binary code, are shown according to embodiments.

[0046] Figure 18 This illustrates another shape of the coding unit that can be determined based on partition shape pattern information that can be represented as binary code, according to an embodiment.

[0047] Figure 19 A block diagram of an image encoding and decoding system that performs loop filtering is shown.

[0048] Figure 20 This is a diagram illustrating the components of an image decoding device according to an embodiment.

[0049] Figure 21 This is an example diagram illustrating the structure of the bitstream generated based on the hierarchical structure of the image.

[0050] Figure 22 This is a diagram showing the stripes, parallel blocks, and coding tree units (CTUs) identified in the current image.

[0051] Figure 23 This is a diagram illustrating a method for configuring stripes in the current image.

[0052] Figure 24 This is a diagram illustrating another method for configuring stripes in the current image.

[0053] Figure 25 This is an example diagram showing a list of multiple first reference images obtained through a sequence parameter set.

[0054] Figure 26 This is a diagram illustrating a method for obtaining a second list of reference images.

[0055] Figure 27 This is a diagram illustrating a method for obtaining a second list of reference images.

[0056] Figure 28 This is an illustration used to describe another method for obtaining a second list of reference images.

[0057] Figure 29 This is an illustration used to describe another method for obtaining a second list of reference images.

[0058] Figure 30 This is an illustration used to describe another method for obtaining a second list of reference images.

[0059] Figure 31 This is a diagram showing a bitstream that includes multiple sets of post-processing parameters for luminance mapping or adaptive loop filtering.

[0060] Figure 32 This is a diagram used to describe an image decoding method according to an embodiment.

[0061] Figure 33 This is a diagram illustrating the components of an image encoding device according to an embodiment.

[0062] Figure 34 This is an illustration used to describe an image encoding method according to an embodiment. Detailed Implementation

[0063] The embodiments are described in more detail below with reference to the accompanying drawings.

[0064] In the following description, similar reference numerals are used for similar elements, even in different figures. Matters defined in the description, such as detailed structures and elements, are provided to aid in a comprehensive understanding of the exemplary embodiments. However, it will be apparent that the exemplary embodiments can be practiced without those specifically defined matters. Furthermore, well-known functions or structures are not described in detail, as this would obscure the description with unnecessary detail.

[0065] In the description of the embodiments, detailed explanations of the related technologies are omitted when it is believed that such detailed explanations might unnecessarily obscure the essence of this disclosure. Furthermore, the numbers used in the description (e.g., first, second, etc.) are merely identifier codes used to distinguish one element from another.

[0066] When expressions such as "at least one of..." follow a list of elements, they modify the entire list of elements, not individual elements. For example, the expression "at least one of a, b, and c" should be understood as including only a, only b, only c, both a and b, both a and c, both b and c, or all of a, b, and c.

[0067] Furthermore, it will be understood in this specification that when elements are “connected” or “coupled” to each other, the elements may be directly connected or coupled to each other, but may optionally be connected or coupled to each other through intermediate elements between the elements, unless otherwise stated.

[0068] In this specification, elements referred to as "units" or "modules" may be combined into one element, or one element may be divided into two or more elements according to subdivided functions. Furthermore, each element described below, in addition to its own primary function, may additionally perform some or all of the functions performed by another element, and some of the primary functions of each element may be entirely performed by another component.

[0069] Furthermore, in this specification, "image" or "picture" may refer to a still image or a moving image of a video, that is, the video itself.

[0070] Furthermore, in this specification, "sample" or "signal" refers to data assigned to a sampling location in an image, i.e., the data to be processed. For example, pixel values ​​of an image in the spatial domain and transform coefficients in the transform domain can be samples. A unit comprising at least one such sample can be defined as a block.

[0071] In the following text, refer to Figures 1 to 19 The image encoding method and apparatus, as well as the image decoding method and apparatus, are described according to embodiments based on tree-structured encoding and transformation units.

[0072] Figure 1 This is a block diagram of an image decoding device 100 according to an embodiment.

[0073] Image decoding device 100 may include a bitstream acquirer 110 and a decoder 120. The bitstream acquirer 110 and decoder 120 may include at least one processor. Furthermore, the bitstream acquirer 110 and decoder 120 may include memory storing instructions to be executed by the at least one processor.

[0074] Bitstream acquirer 110 can receive bitstreams. The bitstream includes information about image encoding of the image encoding device 200, which will be described later. Furthermore, bitstreams can be transmitted from the image encoding device 200. The image encoding device 200 and the image decoding device 100 can be connected via wired or wireless means, and the bitstream acquirer 110 can receive bitstreams via wired or wireless means. The bitstream acquirer 110 can receive bitstreams from storage media such as optical media or hard disks. Decoder 120 can reconstruct an image based on information obtained from the received bitstream. Decoder 120 can obtain syntax elements for reconstructing the image from the bitstream. Decoder 120 can reconstruct the image based on said syntax elements.

[0075] To describe in detail the operation of the image decoding device 100, the bitstream receiver 110 can receive bitstreams.

[0076] The image decoding device 100 can perform the operation of obtaining a binary bit string corresponding to the partition shape pattern of the coding unit from the bit stream. Furthermore, the image decoding device 100 can perform the operation of determining the partitioning rules of the coding unit. Additionally, the image decoding device 100 can perform the operation of dividing the coding unit into multiple coding units based on at least one of the binary bit string corresponding to the partition shape pattern and the partitioning rules. To determine the partitioning rules, the image decoding device 100 can determine a first range of allowable sizes for the coding unit based on the ratio between the width and height of the coding unit. To determine the partitioning rules, the image decoding device 100 can determine a second range of allowable sizes for the coding unit based on the partition shape pattern of the coding unit.

[0077] The division of coding units is described in detail below according to embodiments of the present disclosure.

[0078] First, a frame can be divided into one or more stripes or one or more parallel blocks. A stripe or a parallel block can be a sequence of one or more maximum coding units (i.e., coding tree units (CTUs)). According to embodiments, a stripe may include one or more parallel blocks, and a stripe may include one or more CTUs. Stripes comprising one or more parallel blocks can be determined within a frame.

[0079] As a concept compared to CTU, there exists a maximum coding block (i.e., a coding tree block (CTB)). A CTB represents an N×N block containing N×N samples (N is an integer). Each color component can be divided into one or more CTBs.

[0080] When the image has three sample arrays (sample arrays for the Y, Cr, and Cb components), the CTU includes the CTB of the luma sample, two CTBs for the chroma sample corresponding to the luma sample, and a syntax structure for encoding the luma and chroma samples. When the image is monochrome, the CTU includes the CTB of the monochrome sample and a syntax structure for encoding the monochrome sample. When the image is encoded in a color plane separated according to the color components, the CTU includes a syntax structure for encoding the image and the image samples.

[0081] A CTB can be divided into an M×N coded block containing M×N samples (M and N are integers).

[0082] When the image has a sample array for the Y, Cr, and Cb components, the encoding unit includes an encoding block for luminance samples, two encoding blocks for chrominance samples corresponding to the luminance samples, and a syntax structure for encoding the luminance and chrominance samples. When the image is a monochrome image, the encoding unit includes an encoding block for monochrome samples and a syntax structure for encoding the monochrome samples. When the image is encoded in a color plane separated according to the color components, the encoding unit includes a syntax structure for encoding the image and the image samples.

[0083] As stated above, CTB and CTU are conceptually distinct from each other, and coded blocks and coded units are conceptually distinct from each other. That is, a coded unit (CTU) refers to a data structure that includes a coded block (CTB) containing corresponding samples and the syntax elements corresponding to the coded block (CTB). However, because those skilled in the art understand that a coded unit (CTU) or a coded block (CTB) refers to a block of a specific size containing a specific number of samples, CTB and CTU, or coded blocks and coded units, will be referred to in the following specification without distinction unless otherwise described.

[0084] The image can be divided into CTUs. The size of each CTU can be determined based on information obtained from the bitstream. Each CTU can be a square shape of the same size. However, the embodiments are not limited to this.

[0085] For example, information about the maximum size of a luminance-coded block can be obtained from the bitstream. For example, the maximum size of a luminance-coded block, indicated by the information about the maximum size of the luminance-coded block, can be one of 4×4, 8×8, 16×16, 32×32, 64×64, 128×128, and 256×256.

[0086] For example, information about the luma block size difference and the maximum size of a luma coding block that can be divided into two can be obtained from the bitstream. The information about the luma block size difference can refer to the size difference between the luma CTU and the luma CTB that can be divided into two. Therefore, when the information about the maximum size of the luma coding block that can be divided into two and the information about the luma block size difference obtained from the bitstream are combined, the size of the luma CTU can be determined. The size of the chroma CTU can be determined by using the size of the luma CTU. For example, when the Y:Cb:Cr ratio is 4:2:0 according to the color format, the size of the chroma block can be half the size of the luma block, and the size of the chroma CTU can be half the size of the luma CTU.

[0087] According to the embodiment, since the information regarding the maximum size of a divisible luminance coding block is obtained from the bitstream, the maximum size of the divisible luminance coding block can be variably determined. Conversely, the maximum size of a tridivisible luminance coding block can be fixed. For example, the maximum size of a tridivisible luminance coding block in an I-frame can be 32×32, and the maximum size of a tridivisible luminance coding block in a P-frame or B-frame can be 64×64.

[0088] Furthermore, the CTU can be hierarchically divided into coding units based on the partition shape pattern information obtained from the bitstream. At least one of the following can be obtained from the bitstream as partition shape pattern information: information indicating whether a quad partition is performed, information indicating whether a multi-partition is performed, partition direction information, and partition type information.

[0089] For example, information indicating whether a quad partition is performed can indicate whether the current coding unit is quad partitioned (QUAD_SPLIT) or not.

[0090] When the current coding unit is not quad-partitioned, the information indicating whether to perform multi-partitioning can indicate whether the current coding unit will not be partitioned again (NO_SPLIT) or will be partitioned twice / three times.

[0091] When the current coding unit is divided into two or three parts, the division direction information indicates that the current coding unit is divided in one of the horizontal or vertical directions.

[0092] When the current coding unit is divided in the horizontal or vertical direction, the division type information indicates whether the current coding unit is divided into two or three parts.

[0093] The partitioning mode of the current coding unit can be determined based on the partitioning direction and partitioning type information. When the current coding unit is partitioned into two parts horizontally, the partitioning mode can be determined as a horizontal two-part partitioning mode (SPLIT_BT_HOR); when the current coding unit is partitioned into three parts horizontally, the partitioning mode can be determined as a horizontal three-part partitioning mode (SPLIT_TT_HOR); when the current coding unit is partitioned into two parts vertically, the partitioning mode can be determined as a vertical two-part partitioning mode (SPLIT_BT_VER); and when the current coding unit is partitioned into three parts vertically, the partitioning mode can be determined as a vertical three-part partitioning mode (SPLIT_TT_VER).

[0094] Image decoding device 100 can obtain partitioning shape pattern information from a binary bit string in a bit stream. The bit stream received by image decoding device 100 may be in the form of fixed-length binary code, unary code, truncated unary code, predetermined binary code, etc. The binary bit string is information about binary bits. The binary bit string may include at least one bit. Image decoding device 100 can obtain partitioning shape pattern information corresponding to the binary bit string based on partitioning rules. Image decoding device 100 can determine, based on a binary bit string, whether to perform four partitions on the coding unit, whether not to partition the coding unit, the partitioning direction, and the partitioning type.

[0095] A coding unit can be smaller than or equal to a CTU. For example, because a CTU is the coding unit with the largest size, it is one of the coding units. When the partitioning shape pattern information about a CTU indicates that partitioning is not performed, the coding units determined in the CTU have the same size as the CTU. When the partitioning shape pattern information about a CTU indicates that partitioning is performed, the CTU can be partitioned into coding units. Furthermore, when the partitioning shape pattern information about coding units indicates that partitioning is performed, the coding unit can be divided into smaller coding units. However, the partitioning of an image is not limited to this, and CTUs and coding units may not be distinguished. (See reference...) Figures 3 to 16 Describe in detail the division of coding units.

[0096] Furthermore, one or more prediction blocks for prediction can be determined from the coding unit. The prediction blocks may be equal to or smaller than the coding unit. Furthermore, one or more transform blocks for transform can be determined from the coding unit. The transform blocks may be equal to or smaller than the coding unit.

[0097] The shapes and sizes of the transform blocks and the prediction blocks can be independent of each other.

[0098] In another embodiment, prediction can be performed by using the coding unit as a prediction unit. Furthermore, transformation can be performed by using the coding unit as a transform block.

[0099] Reference Figures 3 to 16 The division of coding units is described in detail. The current block and neighboring blocks in this disclosure can refer to one of a CTU, coding unit, prediction block, and transform block. Furthermore, the current block of the current coding unit is the block currently being decoded or encoded, or the block currently being divided. Neighboring blocks can be blocks reconstructed prior to the current block. Neighboring blocks can be spatially or temporally adjacent to the current block. Neighboring blocks can be located at one of the following locations: below left, left side, above left, above, above right, right side, and below right of the current block.

[0100] Figure 3 The illustration shows a process performed by an image decoding device 100 according to an embodiment to determine at least one coding unit by dividing the current coding unit.

[0101] The block shape may include 4N×4N, 4N×2N, 2N×4N, 4N×N, N×4N, 32N×N, N×32N, 16N×N, N×16N, 8N×N, or N×8N. Here, N can be a positive integer. Block shape information is information indicating at least one of the shape, orientation, aspect ratio, or size of the coding unit.

[0102] The shape of the encoding unit can be square or non-square. When the width and height of the encoding unit are the same (i.e., when the block shape of the encoding unit is 4N×4N), the image decoding device 100 can determine the block shape information of the encoding unit as square. The image decoding device 100 can also determine the shape of the encoding unit as non-square.

[0103] When the width and height of the coding unit are different from each other (i.e., when the block shape of the coding unit is 4N×2N, 2N×4N, 4N×N, N×4N, 32N×N, N×32N, 16N×N, N×16N, 8N×N, or N×8N), the image decoding device 100 can determine the block shape information of the coding unit as a non-square shape. When the shape of the coding unit is non-square, the image decoding device 100 can determine the aspect ratio in the block shape information of the coding unit as at least one of 1:2, 2:1, 1:4, 4:1, 1:8, 8:1, 1:16, 16:1, 1:32, and 32:1. Furthermore, the image decoding device 100 can determine whether the coding unit is horizontal or vertical based on its width and height. Additionally, the image decoding device 100 can determine the size of the coding unit based on at least one of its width, height, or area.

[0104] According to an embodiment, the image decoding device 100 can determine the shape of the coding unit by using block shape information, and can determine the coding unit partitioning method by using partition shape pattern information. That is, the coding unit partitioning method indicated by the partition shape pattern information can be determined based on the block shape indicated by the block shape information used by the image decoding device 100.

[0105] Image decoding device 100 can obtain partition shape pattern information from the bitstream. However, the embodiments are not limited to this, and image decoding device 100 and image encoding device 200 can determine pre-agreed partition shape pattern information based on block shape information. Image decoding device 100 can determine pre-agreed partition shape pattern information for a CTU or a minimum coding unit. For example, image decoding device 100 can determine the partition shape pattern information for a CTU as a four-part partition. Furthermore, image decoding device 100 can determine the partition shape pattern information for the minimum coding unit as "no partitioning". Specifically, image decoding device 100 can determine the size of the CTU to be 256×256. Image decoding device 100 can determine the pre-agreed partition shape pattern information as a four-part partition. A four-part partition is a partition shape pattern in which the width and height of the coding unit are both divided into two equal parts. Image decoding device 100 can obtain a 128×128-sized coding unit from a 256×256-sized CTU based on the partition shape pattern information. Furthermore, image decoding device 100 can determine the size of the minimum coding unit to be 4×4. Image decoding device 100 can obtain partitioning shape pattern information indicating "do not perform partitioning" for the smallest coding unit.

[0106] According to an embodiment, the image decoding device 100 can use block shape information indicating that the current coding unit has a square shape. For example, the image decoding device 100 can determine whether to not divide the square coding unit, whether to divide the square coding unit vertically, whether to divide the square coding unit horizontally, or whether to divide the square coding unit into four coding units based on the division shape pattern information. (See also...) Figure 3 When the block shape information of the current coding unit 300 indicates a square shape, the decoder 120 may not divide the coding unit 310a with the same size as the current coding unit 300 based on the division shape pattern information indicating that division is not performed, or may determine the coding units 310b, 310c, 310d, 310e or 310f divided based on the division shape pattern information indicating a specific division method.

[0107] Reference Figure 3According to an embodiment, the image decoding device 100 can determine two coding units 310b obtained by dividing the current coding unit 300 in the vertical direction based on the division shape pattern information indicating that division is performed in the vertical direction. The image decoding device 100 can determine two coding units 310c obtained by dividing the current coding unit 300 in the horizontal direction based on the division shape pattern information indicating that division is performed in the horizontal direction. The image decoding device 100 can determine four coding units 310d obtained by dividing the current coding unit 300 in the vertical and horizontal directions based on the division shape pattern information indicating that division is performed in both the vertical and horizontal directions. According to an embodiment, the image decoding device 100 can determine three coding units 310e obtained by dividing the current coding unit 300 in the vertical direction based on the division shape pattern information indicating that division is performed in the vertical direction. The image decoding device 100 can determine three coding units 310f obtained by dividing the current coding unit 300 in the horizontal direction based on the division shape pattern information indicating that division is performed in the horizontal direction. However, the method of dividing square coding units is not limited to the above methods, and the division shape pattern information can indicate various methods. The following describes in detail, with regard to various embodiments, a specific partitioning method for dividing square coding units.

[0108] Figure 4 The illustration shows a process performed by an image decoding device 100 according to an embodiment to determine at least one coding unit by dividing a non-square coding unit.

[0109] According to an embodiment, the image decoding device 100 can use block shape information indicating that the current coding unit has a non-square shape. The image decoding device 100 can determine, based on the partitioning shape pattern information, whether not to partition the non-square current coding unit or whether to partition the non-square current coding unit using a specific partitioning method. (Refer to...) Figure 4 When the block shape information of the current encoding unit 400 or 450 indicates a non-square shape, the image decoding device 100 can determine an encoding unit 410 or 460 with the same size as the current encoding unit 400 or 450 based on the partitioning shape pattern information indicating that partitioning is not performed, or it can determine encoding units 420a and 420b, 430a to 430c, 470a and 470b, or 480a to 480c partitioned based on the partitioning shape pattern information indicating a specific partitioning method. Specific partitioning methods for partitioning non-square encoding units will be described in detail below with respect to various embodiments.

[0110] According to an embodiment, the image decoding device 100 can determine the method of dividing coding units by using division shape pattern information, and in this case, the division shape pattern information can indicate the number of one or more coding units generated by dividing the coding units. (See also...) Figure 4 When the partitioning shape pattern information indicates that the current coding unit 400 or 450 is divided into two coding units, the image decoding device 100 can determine the two coding units 420a and 420b or 470a and 470b included in the current coding unit 400 or 450 by partitioning the current coding unit 400 or 450 based on the partitioning shape pattern information.

[0111] According to an embodiment, when the image decoding device 100 divides a non-square current coding unit 400 or 450 based on division shape pattern information, the image decoding device 100 may consider the position of the long side of the non-square current coding unit 400 or 450 to divide the current coding unit. For example, the image decoding device 100 may determine multiple coding units by dividing the long side of the current coding unit 400 or 450 based on its shape.

[0112] According to an embodiment, when the partitioning shape pattern information indicates that the coding unit is divided (tripartitely divided) into an odd number of blocks, the image decoding device 100 can determine the odd number of coding units included in the current coding unit 400 or 450. For example, when the partitioning shape pattern information indicates that the current coding unit 400 or 450 is divided into three coding units, the image decoding device 100 can divide the current coding unit 400 or 450 into three coding units 430a, 430b and 430c or 480a, 480b and 480c.

[0113] According to an embodiment, the width-to-height ratio of the current coding unit 400 or 450 can be 4:1 or 1:4. When the width-to-height ratio is 4:1, the block shape information can be horizontal because the width is longer than the height. When the width-to-height ratio is 1:4, the block shape information can be vertical because the width is shorter than the height. The image decoding device 100 can determine whether to divide the current coding unit into an odd number of blocks based on the division shape pattern information. Furthermore, the image decoding device 100 can determine the division direction of the current coding unit 400 or 450 based on the block shape information of the current coding unit 400 or 450. For example, when the current coding unit 400 is vertical, the image decoding device 100 can determine coding units 430a to 430c by dividing the current coding unit 400 horizontally. Similarly, when the current coding unit 450 is horizontal, the image decoding device 100 can determine coding units 480a to 480c by dividing the current coding unit 450 vertically.

[0114] According to an embodiment, the image decoding device 100 can determine an odd number of coding units included in the current coding unit 400 or 450, and not all determined coding units may have the same size. For example, a particular coding unit 430b or 480b among the determined odd number of coding units 430a, 430b, and 430c, or 480a, 480b, and 480c, may have a different size than the other coding units 430a and 430c or 480a and 480c. That is, the coding units determined by dividing the current coding unit 400 or 450 may have multiple sizes, and in some cases, all the odd number of coding units 430a, 430b, and 430c, or 480a, 480b, and 480c, may have different sizes.

[0115] According to an embodiment, when the shape pattern information indicates that the coding unit should be divided into an odd number of blocks, the image decoding device 100 can determine the odd number of coding units included in the current coding unit 400 or 450, and furthermore, can impose specific restrictions on at least one of the odd number of coding units generated by dividing the current coding unit 400 or 450. (Refer to...) Figure 4 The image decoding device 100 may configure the decoding process for encoding unit 430b or 480b to differ from the decoding processes for other encoding units 430a and 430c or 480a or 480c, wherein encoding unit 430b or 480b is located at the center of the three encoding units 430a, 430b and 430c or 480a, 480b and 480c generated by dividing the current encoding unit 400 or 450. For example, unlike other encoding units 430a and 430c or 480a and 480c, the image decoding device 100 may restrict the encoding unit 430b or 480b at the center position from being divided again or only divided a specific number of times.

[0116] Figure 5 This illustrates a process performed by an image decoding device 100 according to an embodiment, which involves dividing encoding units based on at least one of block shape information and segmentation shape pattern information.

[0117] According to an embodiment, the image decoding device 100 can determine whether to divide the square first coding unit 500 into coding units or not to divide the square first coding unit 500 based on at least one of block shape information and partition shape pattern information. According to an embodiment, when the partition shape pattern information indicates that the first coding unit 500 is divided in the horizontal direction, the image decoding device 100 can determine the second coding unit 510 by dividing the first coding unit 500 in the horizontal direction. The terms first coding unit, second coding unit, and third coding unit used in the embodiment are terms used to understand the relationship before and after the division of coding units. For example, the second coding unit can be determined by dividing the first coding unit, and the third coding unit can be determined by dividing the second coding unit. It will be understood that the structure of the first coding unit, second coding unit, and third coding unit follows the above description.

[0118] According to an embodiment, the image decoding device 100 may determine, based on the division shape pattern information, whether to divide the determined second coding unit 510 into coding units or not to divide the determined second coding unit 510. (See also...) Figure 5 The image decoding device 100 can divide a non-square second coding unit 510, determined by dividing the first coding unit 500, into one or more third coding units 520a, or 520b, 520c, and 520d, based on the division shape pattern information, or it can choose not to divide the non-square second coding unit 510. The image decoding device 100 can obtain the division shape pattern information and can obtain multiple second coding units (e.g., 510) of various shapes by dividing the first coding unit 500 based on the obtained division shape pattern information. It can also divide the second coding unit 510 based on the division shape pattern information using the division method of the first coding unit 500. According to an embodiment, when the first coding unit 500 is divided into second coding units 510 based on the division shape pattern information of the first coding unit 500, the second coding unit 510 can also be divided into third coding units 520a, or 520b, 520c, and 520d based on the division shape pattern information of the second coding unit 510. That is, the coding units can be recursively divided based on the division shape pattern information of each coding unit. Therefore, square coding units can be determined by dividing non-square coding units, and non-square coding units can be determined by recursively dividing square coding units.

[0119] Reference Figure 5Specific coding units (e.g., coding units at the center or square coding units) among an odd number of third coding units 520b, 520c, and 520d determined by dividing a non-square second coding unit 510 can be recursively partitioned. According to an embodiment, a square third coding unit 520c among the odd number of third coding units 520b, 520c, and 520d can be horizontally divided into multiple fourth coding units. Non-square fourth coding units 530b or 530d among the multiple fourth coding units 530a, 530b, 530c, and 530d can again be divided into multiple coding units. For example, a non-square fourth coding unit 530b or 530d can again be divided into an odd number of coding units. Methods for recursively partitioning coding units will be described below with respect to various embodiments.

[0120] According to an embodiment, the image decoding device 100 can divide each of the third coding units 520a, or 520b, 520c, and 520d, into coding units based on the division shape pattern information. Furthermore, the image decoding device 100 can determine, based on the division shape pattern information, not to divide the second coding unit 510. According to an embodiment, the image decoding device 100 can divide the non-square second coding unit 510 into an odd number of third coding units 520b, 520c, and 520d. The image decoding device 100 can impose specific restrictions on certain third coding units among the odd number of third coding units 520b, 520c, and 520d. For example, the image decoding device 100 can limit the number of times the third coding unit 520c at the center position among the odd number of third coding units 520b, 520c, and 520d will not be divided or will be divided.

[0121] Reference Figure 5 The image decoding device 100 may restrict the third coding unit 520c at the center position of the odd number of third coding units 520b, 520c, and 520d included in the non-square second coding unit 510 to no longer be divided, to be divided using a specific division method (e.g., divided into only four coding units or divided using the division method of the second coding unit 510), or to be divided only a specific number of times (e.g., divided only n times (where n>0)). However, the restriction on the third coding unit 520c at the center position is not limited to the above examples and may include various restrictions for decoding the third coding unit 520c at the center position differently from the other third coding units 520b and 520d.

[0122] According to an embodiment, the image decoding device 100 can obtain partitioning shape pattern information for partitioning the current coding unit from a specific position in the current coding unit.

[0123] Figure 6 This illustrates a method performed by an image decoding device 100 according to an embodiment to determine a specific coding unit from an odd number of coding units.

[0124] Reference Figure 6 The partitioning shape pattern information of the current encoding unit 600 or 650 can be obtained from a sample at a specific location among multiple sample points included in the current encoding unit 600 or 650 (e.g., sample point 640 or 690 at the center location). However, the specific location in the current encoding unit 600 from which the partitioning shape pattern information can be obtained is not limited to... Figure 6 The center position in the image can include various positions included in the current encoding unit 600 (e.g., above, below, left, right, upper left, lower left, upper right, and lower right positions). The image decoding device 100 can obtain the division shape pattern information from the specific position and can determine whether to divide the current encoding unit into encoding units of various shapes and sizes or not to divide the current encoding unit.

[0125] According to an embodiment, when the current coding unit is divided into a specific number of coding units, the image decoding device 100 may select one coding unit from the coding units. As will be described below with respect to various embodiments, various methods may be used to select one coding unit from a plurality of coding units.

[0126] According to an embodiment, the image decoding device 100 can divide the current coding unit into multiple coding units and determine the coding unit at a specific location.

[0127] According to an embodiment, the image decoding device 100 can use information indicating the positions of an odd number of coding units to determine the coding unit at the center position among the odd number of coding units. (See also...) Figure 6The image decoding device 100 can determine an odd number of coding units 620a, 620b, and 620c or an odd number of coding units 660a, 660b, and 660c by dividing the current coding unit 600 or the current coding unit 650. The image decoding device 100 can determine an intermediate coding unit 620b or an intermediate coding unit 660b by using information about the positions of the odd number of coding units 620a, 620b, and 620c or the odd number of coding units 660a, 660b, and 660c. For example, the image decoding device 100 can determine the coding unit 620b at the center position by determining the positions of coding units 620a, 620b, and 620c based on information indicating the positions of specific samples included in coding units 620a, 620b, and 620c. In detail, the image decoding device 100 can determine the position of the encoding units 620a, 620b and 620c based on the information indicating the positions of the upper left samples 630a, 630b and 630c of the encoding units 620a, 620b and 620c, and determine the encoding unit 620b at the center position.

[0128] According to an embodiment, the information indicating the positions of the top-left samples 630a, 630b, and 630c included in encoding units 620a, 620b, and 620c respectively may include information about the positions or coordinates of the encoding units 620a, 620b, and 620c in the image. According to an embodiment, the information indicating the positions of the top-left samples 630a, 630b, and 630c included in the encoding units 620a, 620b, and 620c respectively may include information indicating the width or height of the encoding units 620a, 620b, and 620c included in the current encoding unit 600, and the width or height may correspond to information indicating the difference between the coordinates of the encoding units 620a, 620b, and 620c in the image. That is, the image decoding device 100 can determine the encoding unit 620b at the center position by directly using information about the positions or coordinates of the encoding units 620a, 620b, and 620c in the image, or by using information about the width or height corresponding to the difference between the coordinates of the encoding units.

[0129] According to an embodiment, the information indicating the position of the upper left sample 630a of the upper encoding unit 620a may include coordinates (xa, ya), the information indicating the position of the upper left sample 630b of the middle encoding unit 620b may include coordinates (xb, yb), and the information indicating the position of the upper left sample 630c of the lower encoding unit 620c may include coordinates (xc, yc). The image decoding device 100 can determine the middle encoding unit 620b by using the coordinates of the upper left samples 630a, 630b, and 630c included in the encoding units 620a, 620b, and 620c, respectively. For example, when the coordinates of the upper left samples 630a, 630b, and 630c are sorted in ascending or descending order, the encoding unit 620b including the coordinates (xb, yb) of the sample 630b at the center position can be determined as the encoding unit at the center position among the encoding units 620a, 620b, and 620c determined by dividing the current encoding unit 600. However, the coordinates indicating the positions of the top-left samples 630a, 630b, and 630c may include coordinates indicating absolute positions within the frame, or coordinates (dxb, dyb) indicating the relative position of the top-left sample 630b of the intermediate encoding unit 620b relative to the top-left sample 630a of the upper encoding unit 620a, and coordinates (dxc, dyc) indicating the relative position of the top-left sample 630c of the lower encoding unit 620c relative to the top-left sample 630a of the upper encoding unit 620a. The method of determining the encoding unit at a specific location by using the coordinates of samples included in the encoding unit as information indicating the position of the samples is not limited to the methods described above, and may include various arithmetic methods capable of using the coordinates of the samples.

[0130] According to an embodiment, the image decoding device 100 can divide the current encoding unit 600 into a plurality of encoding units 620a, 620b, and 620c, and can select one of the encoding units 620a, 620b, and 620c based on a specific criterion. For example, the image decoding device 100 can select encoding unit 620b, whose size is different from that of other encoding units, from the encoding units 620a, 620b, and 620c.

[0131] According to an embodiment, the image decoding device 100 can determine the width or height of each of the encoding units 620a, 620b, and 620c by using coordinates (xa, ya) indicating the position of the upper left sample point 630a of the upper encoding unit 620a, coordinates (xb, yb) indicating the position of the upper left sample point 630b of the middle encoding unit 620b, and coordinates (xc, yc) indicating the position of the upper left sample point 630c of the lower encoding unit 620c. The image decoding device 100 can determine the respective dimensions of the encoding units 620a, 620b, and 620c by using coordinates (xa, ya), (xb, yb), and (xc, yc) indicating the positions of the encoding units 620a, 620b, and 620c. According to an embodiment, the image decoding device 100 can determine the width of the upper encoding unit 620a as the width of the current encoding unit 600. The image decoding device 100 can determine the height of the upper encoding unit 620a as yb-ya. According to an embodiment, the image decoding device 100 can determine the width of the intermediate encoding unit 620b as the width of the current encoding unit 600. The image decoding device 100 can determine the height of the intermediate encoding unit 620b as yc-yb. According to an embodiment, the image decoding device 100 can determine the width or height of the lower encoding unit 620c by using the width or height of the current encoding unit 600 or the width or height of the upper encoding unit 620a and the intermediate encoding unit 620b. The image decoding device 100 can determine encoding units with dimensions different from the dimensions of other encoding units based on the determined widths and heights of the encoding units 620a to 620c. (Refer to...) Figure 6 The image decoding device 100 can determine an intermediate encoding unit 620b, which has a size different from that of the upper encoding unit 620a and the lower encoding unit 620c, as an encoding unit at a specific location. However, the method described above by the image decoding device 100 for determining an encoding unit with a size different from that of other encoding units corresponds only to the example of determining an encoding unit at a specific location by using the size of the encoding unit determined based on the coordinates of a sample point. Therefore, various methods can be used to determine an encoding unit at a specific location by comparing the size of the encoding unit determined based on the coordinates of a specific sample point.

[0132] The image decoding device 100 can determine the width or height of each of the encoding units 660a, 660b, and 660c by using coordinates (xd, yd) indicating the position of the upper left sample 670a of the left encoding unit 660a, coordinates (xe, ye) indicating the position of the upper left sample 670b of the middle encoding unit 660b, and coordinates (xf, yf) indicating the position of the upper left sample 670c of the right encoding unit 660c. The image decoding device 100 can also determine the respective dimensions of the encoding units 660a, 660b, and 660c by using coordinates (xd, yd), (xe, ye), and (xf, yf) indicating the positions of the encoding units 660a, 660b, and 660c.

[0133] According to an embodiment, the image decoding device 100 can determine the width of the left coding unit 660a as xe-xd. The image decoding device 100 can determine the height of the left coding unit 660a as the height of the current coding unit 650. According to an embodiment, the image decoding device 100 can determine the width of the middle coding unit 660b as xf-xe. The image decoding device 100 can determine the height of the middle coding unit 660b as the height of the current coding unit 650. According to an embodiment, the image decoding device 100 can determine the width or height of the right coding unit 660c by using the width or height of the current coding unit 650 or the width or height of the left coding unit 660a and the middle coding unit 660b. The image decoding device 100 can determine coding units with dimensions different from other coding units based on the determined widths and heights of the coding units 660a to 660c. (Refer to...) Figure 6 The image decoding device 100 can determine an intermediate coding unit 660b, whose size differs from that of the left coding unit 660a and the right coding unit 660c, as a coding unit at a specific location. However, the method described above for determining a coding unit whose size differs from that of other coding units, performed by the image decoding device 100, corresponds only to the example of determining a coding unit at a specific location by using the size of a coding unit determined based on the coordinates of a sample point. Therefore, various methods can be used to determine a coding unit at a specific location by comparing the size of a coding unit determined based on the coordinates of a specific sample point.

[0134] However, the position of the sample points considered in determining the position of the coding unit is not limited to the aforementioned upper left position, and information about any position of the sample points included in the coding unit can be used.

[0135] According to an embodiment, the image decoding device 100 may consider the shape of the current coding unit and select a coding unit at a specific location from an odd number of coding units determined by dividing the current coding unit. For example, when the current coding unit has a non-square shape with a width greater than its height, the image decoding device 100 may determine a coding unit at a specific location along the horizontal direction. That is, the image decoding device 100 may determine one coding unit among coding units located at different positions in the horizontal direction and impose restrictions on that coding unit. When the current coding unit has a non-square shape with a height greater than its width, the image decoding device 100 may determine a coding unit at a specific location along the vertical direction. That is, the image decoding device 100 may determine one coding unit among coding units located at different positions in the vertical direction and may impose restrictions on that coding unit.

[0136] According to an embodiment, the image decoding device 100 can use information indicating the positions of each of an even number of coding units to determine a coding unit at a specific position among the even number of coding units. The image decoding device 100 can determine the even number of coding units by dividing (bi-dividing) the current coding unit, and can determine the coding unit at a specific position by using information about the positions of the even number of coding units. The associated operations are similar to those already described above regarding... Figure 6 The operation corresponding to determining a specific position (e.g., the center position) among an odd number of coding units is described in detail, so its detailed description is not provided here.

[0137] According to an embodiment, when a non-square current coding unit is divided into multiple coding units, specific information about the coding unit at a specific location can be used in the division operation to determine the coding unit at that specific location among the multiple coding units. For example, the image decoding device 100 can use at least one of block shape information and division shape pattern information stored in the samples included in the intermediate coding unit to determine the coding unit at the center position among the multiple coding units determined by dividing the current coding unit in the division operation.

[0138] Reference Figure 6The image decoding device 100 can divide the current coding unit 600 into multiple coding units 620a, 620b, and 620c based on the division shape pattern information, and can determine the coding unit 620b at the center position among the multiple coding units 620a, 620b, and 620c. Furthermore, the image decoding device 100 can determine the coding unit 620b at the center position based on the location where the division shape pattern information is obtained. That is, the division shape pattern information of the current coding unit 600 can be obtained from the sample point 640 at the center position of the current coding unit 600, and when the current coding unit 600 is divided into multiple coding units 620a, 620b, and 620c based on the division shape pattern information, the coding unit 620b including the sample point 640 can be determined as the coding unit at the center position. However, the information used to determine the coding unit at the center position is not limited to the division shape pattern information, and various types of information can be used to determine the coding unit at the center position.

[0139] According to an embodiment, specific information for identifying a coding unit at a specific location can be obtained from specific samples included in the coding unit to be determined. (Refer to...) Figure 6 Image decoding device 100 can use partitioning shape pattern information obtained from samples at specific locations in the current coding unit 600 (e.g., samples at the center of the current coding unit 600) to determine a coding unit at a specific location (e.g., a coding unit at the center of the partitioned coding units) among a plurality of coding units 620a, 620b, and 620c determined by partitioning the current coding unit 600. That is, image decoding device 100 can determine samples at specific locations by considering the block shape of the current coding unit 600, determine coding unit 620b from the plurality of coding units 620a, 620b, and 620c determined by partitioning the current coding unit 600 that includes samples from which specific information (e.g., partitioning shape pattern information) can be obtained, and can impose specific constraints on coding unit 620b. (Refer to...) Figure 6 According to an embodiment, during the decoding operation, the image decoding device 100 may determine a sample 640 at the center position of the current encoding unit 600 as a sample from which specific information can be obtained, and may impose specific restrictions on the encoding unit 620b including the sample 640. However, the position of the sample from which specific information can be obtained is not limited to the above-described position, and may include any position of the sample included in the encoding unit 620b to be determined as subject to restrictions.

[0140] According to an embodiment, the location of a sample point from which specific information can be obtained can be determined based on the shape of the current coding unit 600. According to the embodiment, block shape information can indicate whether the current coding unit has a square shape or a non-square shape, and the location of the sample point from which specific information can be obtained can be determined based on the shape. For example, the image decoding device 100 can determine a sample point located on a boundary that bisects at least one of the width and height of the current coding unit as a sample point from which specific information can be obtained by using at least one of information about the width of the current coding unit and information about the height of the current coding unit. As another example, when the block shape information of the current coding unit indicates a non-square shape, the image decoding device 100 can determine one of the sample points adjacent to the boundary that bisects the long side of the current coding unit as a sample point from which predetermined information can be obtained.

[0141] According to an embodiment, when the current coding unit is divided into multiple coding units, the image decoding device 100 can use partition shape pattern information to determine the coding unit at a specific location among the multiple coding units. According to an embodiment, the image decoding device 100 can obtain partition shape pattern information from samples at a specific location in the coding unit, and partition the multiple coding units generated by partitioning the current coding unit using the partition shape pattern information, wherein the partition shape pattern information is obtained from samples at a specific location in each of the multiple coding units. That is, the coding units can be recursively partitioned based on the partition shape pattern information, wherein the partition shape pattern information is obtained from samples at a specific location in each coding unit. The above already relates to... Figure 5 The operation of recursively dividing the coding unit is described, so its detailed description will not be provided here.

[0142] According to an embodiment, the image decoding device 100 can determine one or more coding units by dividing the current coding unit, and can determine the order in which the one or more coding units are decoded based on a specific block (e.g., the current coding unit).

[0143] Figure 7 This illustrates the order in which the image decoding device 100 processes the plurality of coding units when it determines the plurality of coding units by dividing the current coding unit, according to an embodiment.

[0144] According to an embodiment, based on the division shape pattern information, the image decoding device 100 can determine the second coding units 710a and 710b by dividing the first coding unit 700 in the vertical direction, determine the second coding units 730a and 730b by dividing the first coding unit 700 in the horizontal direction, or determine the second coding units 750a to 750d by dividing the first coding unit 700 in both the vertical and horizontal directions.

[0145] Reference Figure 7 The image decoding device 100 can determine to process second coding units 710a and 710b, determined by dividing the first coding unit 700 in the vertical direction, in a horizontal order 710c. The image decoding device 100 can determine to process second coding units 730a and 730b, determined by dividing the first coding unit 700 in the horizontal direction, in a vertical order 730c. The image decoding device 100 can determine to process second coding units 750a to 750d, determined by dividing the first coding unit 700 in both the vertical and horizontal directions, in a specific order (e.g., in raster scan order or zigzag scan order 750e), wherein coding units in one row are processed in the specific order, and then coding units in the next row are processed.

[0146] According to an embodiment, the image decoding device 100 can recursively divide encoding units. (See also...) Figure 7 The image decoding device 100 can determine a plurality of coding units 710a and 710b, 730a and 730b, or 750a to 750d by dividing a first coding unit 700, and can recursively divide each of the determined plurality of coding units 710a and 710b, 730a and 730b, or 750a to 750d. The method of dividing the plurality of coding units 710a and 710b, 730a and 730b, or 750a to 750d can correspond to the method of dividing the first coding unit 700. Thus, each of the plurality of coding units 710a and 710b, 730a and 730b, or 750a to 750d can be independently divided into a plurality of coding units. (Refer to...) Figure 7 The image decoding device 100 can determine the second coding units 710a and 710b by dividing the first coding unit 700 in the vertical direction, and can determine whether to divide or not divide each of the second coding units 710a and 710b independently.

[0147] According to an embodiment, the image decoding device 100 can determine the third coding units 720a and 720b by dividing the left second coding unit 710a in the horizontal direction, and may not divide the right second coding unit 710b.

[0148] According to an embodiment, the processing order of coding units can be determined based on the operation of dividing coding units. In other words, the processing order of the divided coding units can be determined based on the processing order of the coding units immediately preceding the division. The image decoding device 100 can determine the processing order of the third coding units 720a and 720b determined by dividing the left second coding unit 710a independently of the right second coding unit 710b. Because the third coding units 720a and 720b are determined by dividing the left second coding unit 710a in the horizontal direction, the third coding units 720a and 720b can be processed in the vertical order 720c. Because the left second coding unit 710a and the right second coding unit 710b are processed in the horizontal order 710c, the right second coding unit 710b can be processed after the third coding units 720a and 720b included in the left second coding unit 710a are processed in the vertical order 720c. The operation of determining the processing order of coding units based on the coding units before partitioning is not limited to the above example, and various methods can be used to process the partitioned and determined coding units of various shapes independently in a specific order.

[0149] Figure 8 The illustration shows the process by which an image decoding device determines that the current encoding unit is divided into an odd number of encoding units when the encoding units cannot be processed in a predetermined order, according to an embodiment.

[0150] According to an embodiment, the image decoding device 100 can determine, based on the obtained partition shape pattern information, that the current coding unit is divided into an odd number of coding units. (See also...) Figure 8 The square first coding unit 800 can be divided into non-square second coding units 810a and 810b, and the second coding units 810a and 810b can be independently divided into third coding units 820a and 820b, and 820c to 820e. The second coding units 810a and 810b can be processed in horizontal order 810c. According to an embodiment, the image decoding device 100 can determine a plurality of third coding units 820a and 820b by dividing the left second coding unit 810a in the horizontal direction, and can divide the right second coding unit 810b into an odd number of third coding units 820c to 820e.

[0151] According to an embodiment, the image decoding device 100 can determine whether any coding unit is divided into an odd number of coding units by determining whether the third coding units 820a and 820b and 820c to 820e can be processed in a specific order. (See also...) Figure 8The image decoding device 100 can determine the third coding units 820a and 820b, and 820c to 820e, by recursively dividing the first coding unit 800. The image decoding device 100 can determine whether any one of the following coding units is divided into an odd number of coding units based on at least one of block shape information and division shape pattern information: the first coding unit 800, the second coding units 810a and 810b, and the third coding units 820a and 820b, and 820c to 820e. For example, the rightmost second coding unit 810b of the second coding units 810a and 810b can be divided into an odd number of third coding units 820c, 820d, and 820e. The processing order of the multiple coding units included in the first coding unit 800 can be a specific order (e.g., zigzag scanning order 830). The image decoding device 100 can determine whether the third coding units 820c, 820d, and 820e, determined by dividing the right second coding unit 810b into an odd number of coding units, meet the conditions for processing in a specific order.

[0152] According to an embodiment, the image decoding device 100 can determine whether the third encoding units 820a and 820b, and 820c to 820e included in the first encoding unit 800 satisfy a condition for processing in a specific order, and this condition relates to whether at least one of the width and height of the second encoding units 810a and 810b is halved along the boundary of the third encoding units 820a and 820b, and 820c to 820e. For example, the third encoding units 820a and 820b determined when the height of the non-square-shaped left second encoding unit 810a is halved can satisfy the condition. Because the boundary of the third encoding units 820c to 820e determined when the right second encoding unit 810b is divided into three encoding units fails to halve the width or height of the right second encoding unit 810b, it can be determined that the third encoding units 820c to 820e do not satisfy the condition. When the conditions described above are not met, the image decoding device 100 can determine that the scanning order is discontinuous, and based on the determination result, can determine that the right-side second coding unit 810b is divided into an odd number of coding units. According to an embodiment, when the coding unit is divided into an odd number of coding units, the image decoding device 100 can impose specific restrictions on the coding units at specific positions within the divided coding units. These restrictions or specific positions have already been described above with respect to various embodiments, and therefore will not be described in detail here.

[0153] Figure 9 The illustration shows a process performed by an image decoding device 100 according to an embodiment, which involves dividing a first encoding unit 900 to determine at least one encoding unit.

[0154] According to an embodiment, the image decoding device 100 can divide the first coding unit 900 based on the division shape pattern information obtained by the bitstream acquirer 110. The square first coding unit 900 can be divided into four square coding units, or it can be divided into multiple non-square coding units. For example, referring to… Figure 9 When the partitioning shape pattern information indicates that the first coding unit 900 should be partitioned into non-square coding units, the image decoding device 100 may partition the first coding unit 900 into a plurality of non-square coding units. Specifically, when the partitioning shape pattern information indicates that an odd number of coding units should be determined by partitioning the first coding unit 900 in the horizontal or vertical direction, the image decoding device 100 may partition the square first coding unit 900 into an odd number of coding units (e.g., second coding units 910a, 910b, and 910c determined by partitioning the square first coding unit 900 in the vertical direction, or second coding units 920a, 920b, and 920c determined by partitioning the square first coding unit 900 in the horizontal direction).

[0155] According to an embodiment, the image decoding device 100 can determine whether second encoding units 910a, 910b, 910c, 920a, 920b, and 920c included in the first encoding unit 900 satisfy conditions for processing in a specific order, and these conditions relate to whether at least one of the width and height of the first encoding unit 900 is divided in half along the boundaries of the second encoding units 910a, 910b, 910c, 920a, 920b, and 920c. (Refer to...) Figure 9 Because the boundaries of the second coding units 910a, 910b, and 910c, defined by the first coding unit 900 dividing the square in the vertical direction, do not halve the width of the first coding unit 900, it can be determined that the first coding unit 900 does not meet the conditions for processing in a specific order. Furthermore, because the boundaries of the second coding units 920a, 920b, and 920c, defined by the first coding unit 900 dividing the square in the horizontal direction, do not halve the height of the first coding unit 900, it can be determined that the first coding unit 900 does not meet the conditions for processing in a specific order. When the conditions are not met as described above, the image decoding device 100 can determine that the scanning order is discontinuous, and can determine, based on the determination result, that the first coding unit 900 is divided into an odd number of coding units. According to an embodiment, when the coding unit is divided into an odd number of coding units, the image decoding device 100 can impose specific restrictions on the coding units at specific positions within the divided coding units. The restrictions or specific positions have already been described above with respect to various embodiments, and therefore will not be described in detail here.

[0156] According to an embodiment, the image decoding device 100 can determine coding units of various shapes by dividing a first coding unit.

[0157] Reference Figure 9 The image decoding device 100 can divide the square first coding unit 900 or the non-square first coding unit 930 or 950 into coding units of various shapes.

[0158] Figure 10 The shape into which the second encoding unit can be divided is limited when the second encoding unit, which has a non-square shape and is determined by the image decoding device 100 dividing the first encoding unit 1000, meets certain conditions according to the embodiment.

[0159] According to an embodiment, the image decoding device 100 can determine, based on the partitioning shape pattern information obtained by the bitstream acquirer 110, to divide the square first coding unit 1000 into non-square second coding units 1010a and 1010b, or 1020a and 1020b. The second coding units 1010a and 1010b, or 1020a and 1020b, can be divided independently. Thus, the image decoding device 100 can determine, based on the partitioning shape pattern information of each of the second coding units 1010a and 1010b, or 1020a and 1020b, to divide each of the second coding units 1010a and 1010b, or 1020a and 1020b, into multiple coding units, or not to divide each of the second coding units 1010a and 1010b, or 1020a and 1020b. According to an embodiment, the image decoding device 100 can determine the third coding units 1012a and 1012b by dividing the non-square left second coding unit 1010a, determined by dividing the first coding unit 1000 in the vertical direction, in the horizontal direction. However, when the left second coding unit 1010a is divided in the horizontal direction, the image decoding device 100 can restrict the right second coding unit 1010b to not be divided in the horizontal direction in which the left second coding unit 1010a is divided. When the third coding units 1014a and 1014b are determined by dividing the right second coding unit 1010b in the same direction, since the left second coding unit 1010a and the right second second coding unit 1010b are divided independently in the horizontal direction, the third coding units 1012a and 1012b, or 1014a and 1014b, can be determined. However, this situation works in the same way as when the image decoding device 100 divides the first encoding unit 1000 into four squares, namely the second encoding units 1030a, 1030b, 1030c and 1030d, based on the division shape pattern information, and may be inefficient in terms of image decoding.

[0160] According to an embodiment, the image decoding device 100 can determine the third coding units 1022a and 1022b, or 1024a and 1024b, by dividing the non-square second coding units 1020a or 1020b, which are determined by dividing the first coding unit 1000 in the horizontal direction, in the vertical direction. However, when the second coding unit (e.g., the upper second coding unit 1020a) is divided in the vertical direction, for the reasons described above, the image decoding device 100 can restrict another second coding unit (e.g., the lower second coding unit 1020b) to not be divided in the vertical direction in which the upper second coding unit 1020a is divided.

[0161] Figure 11 The illustration shows the process performed by the image decoding device 100 to divide the square coding unit when the division shape pattern information cannot indicate that the square coding unit is divided into four square coding units, according to an embodiment.

[0162] According to an embodiment, the image decoding device 100 can determine second coding units 1110a and 1110b, or 1120a and 1120b, etc., by dividing the first coding unit 1100 based on division shape pattern information. The division shape pattern information may include information about various methods of dividing the coding units, but may not include information for dividing the coding unit into four square coding units. Based on such division shape pattern information, the image decoding device 100 may not divide the square first coding unit 1100 into four square coding units 1130a, 1130b, 1130c, and 1130d. The image decoding device 100 can determine non-square second coding units 1110a and 1110b, or 1120a and 1120b, etc., based on the division shape pattern information.

[0163] According to an embodiment, the image decoding device 100 can independently divide non-square second coding units 1110a and 1110b, or 1120a and 1120b, etc. Each of the second coding units 1110a and 1110b, or 1120a and 1120b, etc., can be recursively divided in a specific order, and this division method can correspond to the method of dividing the first coding unit 1100 based on the division shape pattern information.

[0164] For example, the image decoding device 100 can determine the third coding units 1112a and 1112b of the square by dividing the left second coding unit 1110a in the horizontal direction, and can determine the third coding units 1114a and 1114b of the square by dividing the right second coding unit 1110b in the horizontal direction. Furthermore, the image decoding device 100 can determine the third coding units 1116a, 1116b, 1116c, and 1116d of the square by dividing both the left second coding unit 1110a and the right second coding unit 1110b in the horizontal direction. In this case, coding units with the same shape as the second coding units 1130a, 1130b, 1130c, and 1130d of the four squares divided from the first coding unit 1100 can be determined.

[0165] As another example, the image decoding device 100 can determine the third coding units 1122a and 1122b of the square by dividing the upper second coding unit 1120a in the vertical direction, and can determine the third coding units 1124a and 1124b of the square by dividing the lower second coding unit 1120b in the vertical direction. Furthermore, the image decoding device 100 can determine the third coding units 1126a, 1126b, 1126c, and 1126d of the square by dividing both the upper second coding unit 1120a and the lower second coding unit 1120b in the vertical direction. In this case, coding units with the same shape as the second coding units 1130a, 1130b, 1130c, and 1130d of the four squares divided from the first coding unit 1100 can be determined.

[0166] Figure 12 This illustrates that the processing order among multiple coding units according to an embodiment can be changed based on the process of dividing the coding units.

[0167] According to an embodiment, the image decoding device 100 can divide the first coding unit 1200 based on the division shape pattern information. When the block shape indicates a square shape and the division shape pattern information indicates that the first coding unit 1200 is divided in at least one direction, either horizontal or vertical, the image decoding device 100 can determine second coding units 1210a and 1210b, or 1220a and 1220b, etc., by dividing the first coding unit 1200. (See also...) Figure 12The non-square second coding units 1210a and 1210b, or 1220a and 1220b, determined by dividing the first coding unit 1200 only in the horizontal or vertical direction, can be independently divided based on the division shape pattern information of each coding unit. For example, the image decoding device 100 can determine third coding units 1216a, 1216b, 1216c, and 1216d by dividing the second coding units 1210a and 1210b generated by dividing the first coding unit 1200 in the vertical direction in the horizontal direction, and can determine third coding units 1226a, 1226b, 1226c, and 1226d by dividing the second coding units 1220a and 1220b generated by dividing the first coding unit 1200 in the horizontal direction in the vertical direction. The above has already discussed... Figure 11 The operation of dividing the second coding units 1210a and 1210b, or 1220a and 1220b, is described, so its detailed description will not be provided here.

[0168] According to an embodiment, the image decoding device 100 can process the encoding units in a specific order. The above has already mentioned... Figure 7 The operation of processing coded units in a specific order is described, therefore its detailed description will not be provided here. (See reference...) Figure 12 The image decoding device 100 can determine the third encoding units 1216a, 1216b, 1216c, and 1216d, as well as 1226a, 1226b, 1226c, and 1226d, of the four squares by dividing the first encoding unit 1200 into squares. According to an embodiment, the image decoding device 100 can determine the processing order of the third encoding units 1216a, 1216b, 1216c, and 1216d, as well as 1226a, 1226b, 1226c, and 1226d, based on the division method of the first encoding unit 1200.

[0169] According to an embodiment, the image decoding device 100 can determine third coding units 1216a, 1216b, 1216c, and 1216d by dividing the second coding units 1210a and 1210b generated by dividing the first coding unit 1200 in the vertical direction in the horizontal direction, and can process the third coding units 1216a, 1216b, 1216c, and 1216d in the following processing order 1217: first, the third coding units 1216a and 1216c included in the left second coding unit 1210a are processed in the vertical direction, and then the third coding units 1216b and 1216d included in the right second coding unit 1210b are processed in the vertical direction.

[0170] According to an embodiment, the image decoding device 100 can determine third coding units 1226a, 1226b, 1226c, and 1226d by dividing the second coding units 1220a and 1220b generated by dividing the first coding unit 1200 in the horizontal direction in the vertical direction, and can process the third coding units 1226a, 1226b, 1226c, and 1226d in the following processing order 1227: firstly, the third coding units 1226a and 1226b included in the upper second coding unit 1220a are processed in the horizontal direction, and then the third coding units 1226c and 1226d included in the lower second coding unit 1220b are processed in the horizontal direction.

[0171] Reference Figure 12 The third coding units 1216a, 1216b, 1216c and 1216d, as well as 1226a, 1226b, 1226c and 1226d of the square can be determined by dividing the second coding units 1210a and 1210b, and 1220a and 1220b, respectively. Although the second coding units 1210a and 1210b determined by dividing the first coding unit 1200 in the vertical direction are different from the second coding units 1220a and 1220b determined by dividing the first coding unit 1200 in the horizontal direction, the third coding units 1216a, 1216b, 1216c, and 1216d, as well as the third coding units 1226a, 1226b, 1226c, and 1226d derived from the second coding units 1210a and 1210b and the second coding units 1220a and 1220b, ultimately show coding units of the same shape derived from the first coding unit 1200. Thus, by recursively dividing the coding units in different ways based on the division shape pattern information, even if the coding units are ultimately determined to have the same shape, the image decoding device 100 can process multiple coding units in different orders.

[0172] Figure 13 The illustration shows the process of determining the depth of a coding unit when the shape and size of the coding unit change, according to an embodiment, as the coding unit is recursively divided to determine multiple coding units.

[0173] According to an embodiment, the image decoding device 100 can determine the depth of a coding unit based on a specific criterion. For example, the specific criterion may be the length of the long side of the coding unit. When the length of the long side of the coding unit before partitioning is 2n (n>0) times the length of the long side of the current coding unit after partitioning, the image decoding device 100 can determine that the depth of the current coding unit is increased by n compared to the depth of the coding unit before partitioning. In the following description, a coding unit with an increased depth is referred to as a coding unit with a lower depth.

[0174] Reference Figure 13 According to an embodiment, the image decoding device 100 can determine a second encoding unit 1302 and a third encoding unit 1304 of lower depth by dividing a first encoding unit 1300 of a square based on block shape information indicating the shape of the square (e.g., the block shape information may be represented as "0: SQUARE"). Assuming the size of the first encoding unit 1300 of the square is 2N×2N, the second encoding unit 1302, determined by dividing the width and height of the first encoding unit 1300 by half, can have a size of N×N. Furthermore, the third encoding unit 1304, determined by dividing the width and height of the second encoding unit 1302 by half, can have a size of N / 2×N / 2. In this case, the width and height of the third encoding unit 1304 are 1 / 4 of the width and height of the first encoding unit 1300. When the depth of the first coding unit 1300 is D, the depth of the second coding unit 1302, whose width and height are half of the width and height of the first coding unit 1300, can be D+1, and the depth of the third coding unit 1304, whose width and height are one-quarter of the width and height of the first coding unit 1300, can be D+2.

[0175] According to an embodiment, the image decoding device 100 can determine a lower-depth second encoding unit 1312 or 1322 and a third encoding unit 1314 or 1324 by dividing a non-square first encoding unit 1310 or 1320 based on block shape information indicating a non-square shape (for example, the block shape information may be represented as "1: NS_VER" indicating a non-square shape with a height greater than its width, or "2: NS_HOR" indicating a non-square shape with a width greater than its height).

[0176] The image decoding device 100 can determine the second encoding unit 1302, 1312, or 1322 by dividing the width and height of the first encoding unit 1310, which has a size of N×2N. That is, the image decoding device 100 can determine the second encoding unit 1302 or the second encoding unit 1322 with a size of N×N by dividing the first encoding unit 1310 in the horizontal direction, or it can determine the second encoding unit 1312 with a size of N / 2×N by dividing the first encoding unit 1310 in both the horizontal and vertical directions.

[0177] According to an embodiment, the image decoding device 100 can determine the second encoding unit 1302, 1312, or 1322 by dividing the width and height of the first encoding unit 1320, which has a size of 2N×N. That is, the image decoding device 100 can determine the second encoding unit 1302 or the second encoding unit 1312 with a size of N×N by dividing the first encoding unit 1320 in the vertical direction, or it can determine the second encoding unit 1322 with a size of N×N / 2 by dividing the first encoding unit 1320 in both the horizontal and vertical directions.

[0178] According to an embodiment, the image decoding device 100 can determine a third encoding unit 1304, 1314, or 1324 by dividing the width and height of a second encoding unit 1302 with a size of N×N. That is, the image decoding device 100 can determine a third encoding unit 1304 with a size of N / 2×N / 2, a third encoding unit 1314 with a size of N / 4×N / 2, or a third encoding unit 1324 with a size of N / 2×N / 4 by dividing the second encoding unit 1302 in the vertical and horizontal directions.

[0179] According to an embodiment, the image decoding device 100 can determine the third encoding unit 1304, 1314, or 1324 by dividing the width and height of the second encoding unit 1312, which has a size of N / 2 × N. That is, the image decoding device 100 can determine the third encoding unit 1304 or the third encoding unit 1324 with a size of N / 2 × N / 4 by dividing the second encoding unit 1312 in the horizontal direction, or it can determine the third encoding unit 1314 with a size of N / 4 × N / 2 by dividing the second encoding unit 1312 in both the vertical and horizontal directions.

[0180] According to an embodiment, the image decoding device 100 can determine the third encoding unit 1304, 1314, or 1324 by dividing the width and height of the second encoding unit 1322, which has a size of N×N / 2. That is, the image decoding device 100 can determine the third encoding unit 1304 or the third encoding unit 1314 with a size of N / 2×N / 2 by dividing the second encoding unit 1322 in the vertical direction, or it can determine the third encoding unit 1324 with a size of N / 2×N / 4 by dividing the second encoding unit 1322 in both the vertical and horizontal directions.

[0181] According to an embodiment, the image decoding device 100 can divide square coding units 1300, 1302, or 1304 in the horizontal or vertical direction. For example, the image decoding device 100 can determine a first coding unit 1310 of size N×2N by dividing a first coding unit 1300 of size 2N×2N in the vertical direction, or it can determine a first coding unit 1320 of size 2N×N by dividing a first coding unit 1300 of size 2N×2N in the horizontal direction. According to an embodiment, when the depth is determined based on the length of the longest side of the coding unit, the depth of the coding unit determined by dividing a first coding unit 1300 of size 2N×2N in the horizontal or vertical direction can be the same as the depth of the first coding unit 1300.

[0182] According to an embodiment, the width and height of the third encoding unit 1314 or 1324 can be 1 / 4 of the width and height of the first encoding unit 1310 or 1320. When the depth of the first encoding unit 1310 or 1320 is D, the depth of the second encoding unit 1312 or 1322, whose width and height are 1 / 2 of the width and height of the first encoding unit 1310 or 1320, can be D+1, and the depth of the third encoding unit 1314 or 1324, whose width and height are 1 / 4 of the width and height of the first encoding unit 1310 or 1320, can be D+2.

[0183] Figure 14 The diagram illustrates a depth that can be determined based on the shape and size of the coding unit, and a partial index (PID) used to distinguish the coding units, according to an embodiment.

[0184] According to an embodiment, the image decoding device 100 can determine second coding units of various shapes by dividing a first coding unit 1400 into squares. (See also...) Figure 14 The image decoding device 100 can determine second coding units 1402a and 1402b, second coding units 1404a and 1404b, and second coding units 1406a, 1406b, 1406c, and 1406d by dividing the first coding unit 1400 in at least one direction, either vertical or horizontal, based on the division shape pattern information. In other words, the image decoding device 100 can determine the second coding units 1402a and 1402b, 1404a and 1404b, and 1406a, 1406b, 1406c, and 1406d based on the division shape pattern information of the first coding unit 1400.

[0185] According to an embodiment, the depths of the second coding units 1402a and 1402b, second coding units 1404a and 1404b, and second coding units 1406a, 1406b, 1406c, and 1406d, determined based on the division shape pattern information of the square first coding unit 1400, can be determined based on the length of their longer sides. For example, since the length of the side of the square first coding unit 1400 is equal to the length of the longer side of the non-square second coding units 1402a and 1402b and 1404a and 1404b, the first coding unit 1400 and the non-square second coding units 1402a and 1402b and 1404a and 1404b can have the same depth, such as D. However, when the image decoding device 100 divides the first encoding unit 1400 into four square second encoding units 1406a, 1406b, 1406c and 1406d based on the division shape pattern information, the depth of the second encoding units 1406a, 1406b, 1406c and 1406d can be D+1, which is 1 lower than the depth D of the first encoding unit 1400.

[0186] According to an embodiment, the image decoding device 100 can determine a plurality of second encoding units 1412a and 1412b, as well as 1414a, 1414b, and 1414c, by dividing a first encoding unit 1410 in the horizontal direction with a height greater than its width based on division shape pattern information. According to an embodiment, the image decoding device 100 can determine a plurality of second encoding units 1422a and 1422b, as well as 1424a, 1424b, and 1424c, by dividing a first encoding unit 1420 in the vertical direction with a width greater than its height based on division shape pattern information.

[0187] According to an embodiment, the depths of the second coding units 1412a and 1412b, and 1414a, 1414b and 1414c, or 1422a and 1422b, and 1424a, 1424b and 1424c, determined based on the division shape pattern information of the non-square first coding unit 1410 or 1420, can be determined based on the length of their longer sides. For example, since the length of the side of the square second coding units 1412a and 1412b is half the length of the longer side of the non-square first coding unit 1410 whose height is greater than its width, the depth of the square second coding units 1412a and 1412b is D+1, which is 1 lower than the depth D of the non-square first coding unit 1410.

[0188] Furthermore, the image decoding device 100 can divide the non-square first coding unit 1410 into an odd number of second coding units 1414a, 1414b, and 1414c based on the division shape pattern information. The odd number of second coding units 1414a, 1414b, and 1414c may include the non-square second coding units 1414a and 1414c and the square second coding unit 1414b. In this case, since the length of the long side of the non-square second coding units 1414a and 1414c and the length of the side of the square second coding unit 1414b are half the length of the long side of the first coding unit 1410, the depth of the second coding units 1414a, 1414b, and 1414c can be D+1, which is 1 lower than the depth D of the non-square first coding unit 1410. The image decoding device 100 can determine the depth of the coding unit divided from the first coding unit 1420, which has a width longer than its height and has a non-square shape, by using the method described above for determining the depth of the coding unit divided from the first coding unit 1410.

[0189] According to an embodiment, when an odd number of the divided coding units do not have equal sizes, the image decoding device 100 can determine the PID used to identify the divided coding units based on the size ratio between the coding units. (See also...) Figure 14 In an odd-numbered set of coding units 1414a, 1414b, and 1414c, the width of the central coding unit 1414b can be equal to the width of the other coding units 1414a and 1414c, and its height is twice the height of the other coding units 1414a and 1414c. That is, in this case, the central coding unit 1414b may include two other coding units 1414a or 1414c. Therefore, when the PID of the central coding unit 1414b is 1 based on the scan order, the PID of the coding unit 1414c located adjacent to coding unit 1414b can be increased by 2 and thus can be 3. That is, there may be discontinuous PID values. According to an embodiment, the image decoding device 100 can determine whether the odd-numbered coding units do not have equal sizes based on whether there is a discontinuity in the PID used to identify the divided coding units.

[0190] According to an embodiment, the image decoding device 100 may determine whether to use a specific partitioning method based on PID values ​​used to identify multiple coding units determined by partitioning the current coding unit. (See also...) Figure 14The image decoding device 100 can determine an even number of coding units 1412a and 1412b or an odd number of coding units 1414a, 1414b, and 1414c by dividing a first coding unit 1410 having a rectangular shape with a height longer than its width. The image decoding device 100 can use a PID indicating the corresponding coding unit to identify it. According to an embodiment, the PID can be obtained from a sample point at a specific location of each coding unit (e.g., the upper left sample point).

[0191] According to an embodiment, the image decoding device 100 can determine the coding unit at a specific location within the divided coding units by using a PID for distinguishing coding units. According to an embodiment, when the division shape pattern information of a first coding unit 1410 having a rectangular shape with a height greater than its width indicates that the coding unit should be divided into three coding units, the image decoding device 100 can divide the first coding unit 1410 into three coding units 1414a, 1414b, and 1414c. The image decoding device 100 can assign a PID to each of the three coding units 1414a, 1414b, and 1414c. The image decoding device 100 can compare the PIDs of an odd number of divided coding units to determine the coding unit at the center position within the coding units. The image decoding device 100 can determine the coding unit 1414b, whose PID corresponds to the median value of the PIDs of the coding units, as the coding unit at the center position within the coding units determined by dividing the first coding unit 1410. According to an embodiment, when the divided coding units do not have equal sizes, the image decoding device 100 can determine a PID for distinguishing the divided coding units based on the size ratio between the coding units. (Refer to...) Figure 14The width of the coding unit 1414b generated by dividing the first coding unit 1410 can be equal to the width of the other coding units 1414a and 1414c, and its height can be twice the height of the other coding units 1414a and 1414c. In this case, when the PID of the coding unit 1414b at the center position is 1, the PID of the coding unit 1414c located adjacent to the coding unit 1414b can be increased by 2 and therefore can be 3. When the PID does not increase uniformly as described above, the image decoding device 100 can determine that the coding unit is divided into a plurality of coding units, wherein the plurality of coding units includes coding units with dimensions different from those of the other coding units. According to an embodiment, when the division shape pattern information indicates that the coding unit is divided into an odd number of coding units, the image decoding device 100 can divide the current coding unit in such a way that the coding unit at a specific position in the odd number of coding units (e.g., the coding unit at the center position) has a dimension different from that of the other coding units. In this case, the image decoding device 100 can determine the coding unit at the center position with a different dimension by using the PID of the coding unit. However, the PID and size or position of the encoding unit at a specific location are not limited to the examples above, and various PIDs, positions and sizes of encoding units can be used.

[0192] According to an embodiment, the image decoding device 100 may use a specific data unit, in which encoding units are recursively divided.

[0193] Figure 15 This illustrates how multiple encoding units are determined based on multiple specific data units included in a screen, according to an embodiment.

[0194] According to an embodiment, a specific data unit can be defined as a data unit that recursively divides coding units by using division shape pattern information. That is, a specific data unit can correspond to a coding unit used to determine the highest depth of multiple coding units divided from the current frame. In the following description, for ease of explanation, the specific data unit is referred to as a reference data unit.

[0195] According to an embodiment, the reference data unit may have a specific size and a specific shape. According to an embodiment, the reference data unit may include M×N sample points. Here, M and N may be equal to each other and may be integers represented as powers of 2. That is, the reference data unit may have a square shape or a non-square shape and may be divided into an integer number of encoding units.

[0196] According to an embodiment, the image decoding device 100 can divide the current frame into multiple reference data units. According to an embodiment, the image decoding device 100 can divide the multiple reference data units from the current frame using the division shape pattern information of each reference data unit. The operation of dividing the reference data units can correspond to a division operation using a quadtree structure.

[0197] According to an embodiment, the image decoding device 100 can predetermine the minimum allowed size of the reference data units included in the current frame. Therefore, the image decoding device 100 can determine various reference data units having sizes equal to or greater than the minimum size, and can determine one or more coding units by referring to the determined reference data units and using partitioning shape pattern information.

[0198] Reference Figure 15 The image decoding device 100 may use a square reference coding unit 1500 or a non-square reference coding unit 1502. According to an embodiment, the shape and size of the reference coding unit may be determined based on various data units (e.g., sequences, frames, stripes, strip segments, parallel blocks, parallel block groups, CTUs, etc.) that can include at least one reference coding unit.

[0199] According to an embodiment, for each of the various data units described above, the bitstream acquirer 110 of the image decoding device 100 can obtain from the bitstream at least one of information about the shape of the reference coding unit and information about the size of the reference coding unit. The above has already discussed... Figure 3 The operation of dividing the current coding unit 300 describes the operation of dividing the square reference coding unit 1500 into one or more coding units, and the above has already been discussed... Figure 4 The operation of dividing the current coding unit 400 or 450 describes the operation of dividing the non-square reference coding unit 1502 into one or more coding units. Therefore, its detailed description will not be provided here.

[0200] According to an embodiment, the image decoding device 100 can determine the size and shape of a reference coding unit using a PID (PID) used to identify the size and shape of the reference coding unit, based on a set of data units predetermined according to specific conditions. That is, the bitstream acquirer 110 can obtain from the bitstream only the PID for each slice, slice segment, parallel block, parallel block group, or CTU for identifying the size and shape of the reference coding unit, wherein the slice, slice segment, parallel block, parallel block group, or CTU is a data unit (e.g., a data unit with a size equal to or smaller than the slice) among various data units (e.g., a sequence, a frame, a slice, a slice segment, a parallel block, a parallel block group, a CTU, etc.) that meets specific conditions (e.g., a data unit with a size equal to or smaller than the slice). The image decoding device 100 can determine the size and shape of the reference coding unit for each data unit that meets the specific conditions by using the PID. When obtaining and using reference coding unit shape information and reference coding unit size information from the bitstream based on each data unit with a relatively small size, the efficiency of using the bitstream may be low, and therefore, it is preferable to obtain and use only the PID, rather than directly obtaining the reference coding unit shape information and reference coding unit size information. In this case, at least one of the dimensions and shapes of the reference coding unit corresponding to the PID used to identify the dimensions and shape of the reference coding unit can be predetermined. That is, the image decoding device 100 can determine at least one of the dimensions and shapes of the reference coding unit included in the data unit used as the unit for obtaining the PID by selecting at least one of the dimensions and shapes of the reference coding unit predetermined based on the PID.

[0201] According to embodiments, the image decoding device 100 may use one or more reference coding units included in the CTU. That is, the CTU divided from the image may include one or more reference coding units, and the coding unit can be determined by recursively dividing each reference coding unit. According to embodiments, at least one of the width and height of the CTU may be an integer multiple of at least one of the width and height of the reference coding unit. According to embodiments, the size of the reference coding unit can be obtained by dividing the CTU n times based on a quadtree structure. That is, according to various embodiments, the image decoding device 100 may determine the reference coding unit by dividing the CTU n times based on a quadtree structure, and may divide the reference coding unit based on at least one of block shape information and division shape pattern information.

[0202] According to an embodiment, the image decoding device 100 can obtain block shape information indicating the shape of the current coding unit or partition shape pattern information indicating the partitioning method of the current coding unit from the bitstream, and can use the obtained information. The partition shape pattern information can be included in the bitstream associated with various data units. For example, the image decoding device 100 can use the partition shape pattern information included in sequence parameter sets, picture parameter sets, video parameter sets, strip headers, strip segment headers, parallel block headers, or parallel block group headers. Furthermore, the image decoding device 100 can obtain syntax elements corresponding to the block shape information or partition shape pattern information from the bitstream according to each CTU, each reference coding unit, or each processing block, and can use the obtained syntax elements.

[0203] The method for determining partitioning rules according to embodiments of the present disclosure will be described in detail below.

[0204] Image decoding device 100 can determine the partitioning rules of an image. The partitioning rules can be predetermined between image decoding device 100 and image encoding device 200. Image decoding device 100 can determine the partitioning rules based on information obtained from the bitstream. Image decoding device 100 can determine the partitioning rules based on information obtained from at least one of sequence parameter set, picture parameter set, video parameter set, strip header, strip segment header, parallel block header, or parallel block group header. Image decoding device 100 can determine different partitioning rules based on frames, stripes, parallel blocks, time layers, CTUs, or coding units.

[0205] Image decoding device 100 may determine partitioning rules based on the block shape of the coding units. The block shape may include the size, shape, aspect ratio, and orientation of the coding units. Image encoding device 200 and image decoding device 100 may pre-determine partitioning rules based on the block shape of the coding units. However, embodiments are not limited to this. Image decoding device 100 may determine image partitioning rules based on information obtained from the bitstream received from image encoding device 200.

[0206] The shape of the encoding unit can be square or non-square. When the width and height of the encoding unit are the same, the image decoding device 100 can determine that the shape of the encoding unit is square. Furthermore, when the width and height of the encoding unit are not the same, the image decoding device 100 can determine that the shape of the encoding unit is non-square.

[0207] The size of the coding unit can include various sizes, such as 4×4, 8×4, 4×8, 8×8, 16×4, 16×8, ... and 256×256. The size of the coding unit can be classified based on the length of its long side, the length of its short side, or its area. The image decoding device 100 can apply the same classification rules to coding units classified into the same group. For example, the image decoding device 100 can classify coding units with the same long side length as having the same size. Furthermore, the image decoding device 100 can apply the same classification rules to coding units with the same long side length.

[0208] The aspect ratio of the coding unit can include 1:2, 2:1, 1:4, 4:1, 1:8, 8:1, 1:16, 16:1, 32:1, 1:32, etc. Furthermore, the orientation of the coding unit can include horizontal and vertical directions. A horizontal orientation indicates that the width of the coding unit is longer than its height. A vertical orientation indicates that the width of the coding unit is shorter than its height.

[0209] The image decoding device 100 can adaptively determine the partitioning rules based on the size of the coding unit. The image decoding device 100 can determine the permissible partitioning shape patterns based on different coding unit sizes. For example, the image decoding device 100 can determine whether partitioning is permissible based on the size of the coding unit. The image decoding device 100 can determine the partitioning direction based on the size of the coding unit. The image decoding device 100 can determine the permissible partitioning type based on the size of the coding unit.

[0210] The partitioning rule determined based on the size of the coding unit can be a pre-determined partitioning rule between the image encoding device 200 and the image decoding device 100. Alternatively, the image decoding device 100 can determine the partitioning rule based on information obtained from the bitstream.

[0211] The image decoding device 100 can adaptively determine the partitioning rules based on the position of the coding units in the image.

[0212] Furthermore, the image decoding device 100 can determine partitioning rules such that encoding units generated via different partitioning paths do not have the same block shape. However, the embodiment is not limited to this, and encoding units generated via different partitioning paths may have the same block shape. Encoding units generated via different partitioning paths may have different decoding processing orders. As already referred to above... Figure 12 The decoding process order has been described, so its details are not provided here.

[0213] Figure 16The illustration shows an encoding unit that can be determined for each frame when the combination of shapes into which the encoding unit can be divided is different for each frame, according to an embodiment.

[0214] Reference Figure 16 The image decoding device 100 can determine different combinations of division shapes into which the encoding units can be divided for each frame. For example, the image decoding device 100 can decode an image using one or more frames included in the image, such as frame 1600 which can be divided into four encoding units, frame 1610 which can be divided into two or four encoding units, and frame 1620 which can be divided into two, three, or four encoding units. To divide frame 1600 into multiple encoding units, the image decoding device 100 can use only the division shape information indicating division into four square encoding units. To divide frame 1610, the image decoding device 100 can use only the division shape information indicating division into two or four encoding units. To divide frame 1620, the image decoding device 100 can use only the division shape information indicating division into two, three, or four encoding units. The above combinations of division shapes are merely embodiments for describing the operation of the image decoding device 100. Therefore, the above combinations of division shapes should not be construed as limited to the above embodiments, but should be construed as allowing various types of combinations of division shapes to be used for predetermined data units.

[0215] According to an embodiment, the bitstream acquirer 110 of the image decoding device 100 can acquire a bitstream including an index, wherein the index indicates a combination of partition shape information for each predetermined data unit (e.g., sequence, frame, strip, strip segment, parallel block, or parallel block group). For example, the bitstream acquirer 110 can acquire the index indicating the combination of partition shape information from a sequence parameter set, a frame parameter set, a strip header, a parallel block header, or a parallel block group header. The bitstream acquirer 110 of the image decoding device 100 can determine the combination of partition shapes into which the encoding unit can be divided for each predetermined data unit by using the acquired index; therefore, different combinations of partition shapes can be used for each predetermined data unit.

[0216] Figure 17 Various shapes of coding units, which can be determined based on partition shape pattern information that can be represented as binary code, are shown according to embodiments.

[0217] According to an embodiment, the image decoding device 100 can divide the encoding unit into various shapes using block shape information and partition shape pattern information obtained by the bitstream acquirer 110. The shapes into which the encoding unit can be divided can correspond to various shapes including those described in the above embodiments.

[0218] Reference Figure 17The image decoding device 100 can divide square coding units in at least one direction, either horizontal or vertical, based on the division shape pattern information, and can also divide non-square coding units in either the horizontal or vertical direction.

[0219] According to an embodiment, when the image decoding device 100 can divide a square coding unit into four square coding units in the horizontal and vertical directions, the division shape indicated by the division shape pattern information for the square coding unit can correspond to four types. According to an embodiment, the division shape pattern information can be represented as a two-bit binary code, and each division shape can be assigned a binary code. For example, when the coding unit is not divided, the division shape pattern information can be represented as (00)b; when the coding unit is divided in the horizontal and vertical directions, the division shape pattern information can be represented as (01)b; when the coding unit is divided in the horizontal direction, the division shape pattern information can be represented as (10)b; and when the coding unit is divided in the vertical direction, the division shape pattern information can be represented as (11)b.

[0220] According to an embodiment, when the image decoding device 100 divides non-square coding units in the horizontal or vertical direction, the division shape type, which can be indicated by the division shape pattern information, can be determined based on the number of coding units into which the non-square coding units are divided. (Refer to...) Figure 17 According to an embodiment, the image decoding device 100 can divide a non-square coding unit into up to three coding units. The image decoding device 100 can divide a coding unit into two coding units, and in this case, the division shape pattern information can be represented as (10)b. The image decoding device 100 can divide a coding unit into three coding units, and in this case, the division shape pattern information can be represented as (11)b. The image decoding device 100 can determine not to divide the coding unit, and in this case, the division shape pattern information can be represented as (0)b. That is, in order to use the binary code indicating the division shape pattern information, the image decoding device 100 can use variable-length coding (VLC) instead of fixed-length coding (FLC).

[0221] Reference Figure 17 According to an embodiment, the binary code indicating that the coding unit is not divided into partition shape pattern information can be represented as (0)b. When the binary code indicating that the coding unit is not divided into partition shape pattern information is configured as (00)b, even if there is no partition shape pattern information configured as (01)b, it may be necessary to use all 2 bits of the partition shape pattern information. However, as Figure 17As shown, when using the three partition shape types for non-square coding units, the image decoding device 100 can determine that the coding unit should not be partitioned even by using a 1-bit binary code (0)b as partition shape pattern information. Therefore, the bitstream can be used efficiently. However, the partition shape of the non-square coding unit indicated by the partition shape pattern information should not be interpreted as limited to Figure 17 The three types of shape division shown should be interpreted as including various shapes that include the above embodiments.

[0222] Figure 18 This illustrates another shape of the coding unit that can be determined based on partition shape pattern information that can be represented as binary code, according to an embodiment.

[0223] Reference Figure 18 The image decoding device 100 can divide square coding units in the horizontal or vertical direction based on the division shape pattern information, and can also divide non-square coding units in the horizontal or vertical direction. That is, the division shape pattern information can indicate the division of square coding units in one direction. In this case, the binary code of the division shape pattern information indicating that no square coding units are divided can be represented as (0)b. When the binary code of the division shape pattern information indicating that no coding units are divided is configured as (00)b, even if there is no division shape pattern information configured as (01)b, it may be necessary to use all 2 bits of the binary code of the division shape pattern information. However, as Figure 18 As shown, when using the three partition shape types for square coding units, the image decoding device 100 can determine that the coding unit should not be partitioned even by using a 1-bit binary code (0)b as partition shape pattern information. Therefore, the bitstream can be used efficiently. However, the partition shape of the square coding unit indicated by the partition shape pattern information should not be interpreted as limited to Figure 18 The three types of shape division shown should be interpreted as including various shapes that include the above embodiments.

[0224] According to embodiments, block shape information or partition shape pattern information can be represented using binary codes, and the block shape information or partition shape pattern information can be directly generated as a bitstream. Alternatively, the block shape information or partition shape pattern information that can be represented as binary codes may not be directly generated as a bitstream, but may be used as binary codes input in Context Adaptive Binary Arithmetic Coding (CABAC).

[0225] According to an embodiment, the image decoding device 100 is described as obtaining a syntax for block shape information or partition shape pattern information via CABAC. A bitstream including binary codes for the syntax can be obtained by a bitstream acquirer 110. The image decoding device 100 can detect syntax elements indicating block shape information or partition shape pattern information by debinarizing the binary bit strings included in the obtained bitstream. According to an embodiment, the image decoding device 100 can obtain a set of binary bit strings corresponding to the syntax element to be decoded, and can decode each bit by using probability information. Furthermore, the image decoding device 100 can repeat this process until the binary bit string composed of these decoded bits becomes the same as one of the previously obtained binary bit strings. The image decoding device 100 can determine the syntax element by performing debinarization on the binary bit strings.

[0226] According to an embodiment, the image decoding device 100 can determine the syntax for a binary bit string by performing decoding processing of adaptive binary arithmetic encoding, and the image decoding device 100 can update the probability model for the binary bits obtained by the bit stream acquirer 110. (Refer to...) Figure 17 According to an embodiment, the bitstream acquirer 110 of the image decoding device 100 can acquire a bitstream indicating binary code representing segmentation shape pattern information. The image decoding device 100 can determine the syntax for the segmentation shape pattern information using the acquired 1-bit or 2-bit binary code. To determine the syntax for the segmentation shape pattern information, the image decoding device 100 can update the probability for each bit of the 2-bit binary code. That is, depending on whether the value of the first bit of the 2-bit binary code is 0 or 1, the image decoding device 100 can update the probability that the value of the next bit is 0 or 1 when the next bit is decoded.

[0227] According to the embodiment, in the process of determining the syntax, the image decoding device 100 may update the probability for the binary bits. In the process of decoding the binary bits of the binary bit string for the syntax, and for a predetermined bit in the binary bit string, the image decoding device 100 may not update the probability and may determine that the probabilities are the same.

[0228] Reference Figure 17In the process of determining the syntax by using a bit string representing the partition shape pattern information for non-square coding units, when a non-square coding unit is not partitioned, the image decoding device 100 can determine the syntax for the partition shape pattern information by using a bit with a value of 0. That is, when the block shape information indicates that the current coding unit has a non-square shape, the first bit of the bit string for the partition shape pattern information can be 0 when the non-square coding unit is not partitioned, and the first bit of the bit string for the partition shape pattern information can be 1 when the non-square coding unit is partitioned into two or three coding units. Therefore, the probability that the first bit of the bit string for the partition shape pattern information for non-square coding units is 0 can be 1 / 3, and the probability that the first bit of the bit string for the partition shape pattern information for non-square coding units is 1 can be 2 / 3. As described above, since the partition shape pattern information indicating that a non-square coding unit has not been partitioned can be represented by only using a 1-bit bit string with a value of 0, the image decoding device 100 can determine the syntax for the partition shape pattern information by determining whether the second bit is 0 or 1 only when the first bit of the partition shape pattern information is 1. According to an embodiment, when the first binary bit of the shape pattern information is 1, the image decoding device 100 can assume that the probability of the second binary bit being 0 is the same as the probability of the second binary bit being 1, and can decode the binary bit.

[0229] According to an embodiment, in the process of determining the bits of a binary bit string for dividing shape pattern information, the image decoding device 100 may use various probabilities for each bit. According to an embodiment, the image decoding device 100 may determine the probability of the bits for dividing shape pattern information differently based on the orientation of the non-square block. According to an embodiment, the image decoding device 100 may determine the probability of the bits for dividing shape pattern information differently based on the width of the current encoding unit or the length of the longer side. According to an embodiment, the image decoding device 100 may determine the probability of the bits for dividing shape pattern information differently based on at least one of the shape of the current encoding unit and the length of the longer side.

[0230] According to an embodiment, the image decoding device 100 can determine that the probabilities of binary bits for dividing shape pattern information are the same for encoding units with a size equal to or greater than a predetermined size. For example, the image decoding device 100 can determine that the probabilities of binary bits for dividing shape pattern information are the same for encoding units with a size equal to or greater than 64 samples, based on the length of the longer side of the encoding unit.

[0231] According to an embodiment, the image decoding device 100 may determine the initial probability of the binary bits that make up the binary bit string that divides the shape pattern information based on the strip type (e.g., I strip, P strip, or B strip).

[0232] Figure 19 A block diagram of an image encoding and decoding system that performs loop filtering is shown.

[0233] The image encoding and decoding system 1900 has an encoding end 1910 that transmits an encoded bitstream of the image, and a decoding end 1950 that receives the bitstream and decodes it to output a reconstructed image. Here, the encoding end 1910 may have a similar configuration to the image encoding device 200 described below, and the decoding end 1950 may have a similar configuration to the image decoding device 100.

[0234] At the encoding end 1910, the predictive encoder 1915 outputs predicted data via inter-frame prediction and intra-frame prediction, and the transformer and quantizer 1920 outputs quantized transform coefficients of the residual data between the predicted data and the current input image. The entropy encoder 1925 encodes and transforms the quantized transform coefficients and outputs them as a bitstream. The quantized transform coefficients are reconstructed into spatial domain data via the inverse quantizer and inverse transformer 1930, and the reconstructed spatial domain data is output as a reconstructed image via the deblocking filter 1935 and the loop filter 1940. The reconstructed image can be used as a reference image for the next input image via the predictive encoder 1915.

[0235] The encoded image data in the bitstream received by the decoder 1950 is reconstructed into spatial domain residual data via the entropy decoder 1955, dequantizer, and inverse transformer 1960. The prediction data and residual data output from the prediction decoder 1975 can be combined to construct spatial domain image data, and the deblocking filter 1965 and loop filter 1970 can filter the spatial domain image data to output a reconstructed image for the current original image. The reconstructed image can be used as a reference image for the next original image via the prediction decoder 1975.

[0236] The loop filter 1940 of the encoder 1910 performs loop filtering by using filter information input according to user input or system settings. The filter information used by the loop filter 1940 is output to the entropy encoder 1925 and sent to the decoder 1950 along with the encoded image data. The loop filter 1970 of the decoder 1950 can perform loop filtering based on the filter information input from the decoder 1950.

[0237] The various embodiments described above describe operations related to the image decoding method performed by the image decoding device 100. Hereinafter, the operations of the image encoding device 200, which performs an image encoding method corresponding to the inverse processing of the image decoding method, are described according to various embodiments.

[0238] Figure 2 This is a block diagram of an image encoding device 200 according to an embodiment, capable of encoding an image based on at least one of block shape information and segmentation shape pattern information.

[0239] Image encoding device 200 may include encoder 220 and bitstream generator 210. Encoder 220 may receive and encode an input image. Encoder 220 may obtain at least one syntax element by encoding the input image. Syntax element may include at least one of skip flag, prediction mode, motion vector difference, motion vector prediction method (or index), transform quantization coefficient, coding block style, coding block flag, intra-prediction mode, direct flag, merge flag, differential QP, reference index, prediction direction, and transform index. Encoder 220 may determine a context model based on block shape information including at least one of the following: shape, orientation, width-to-height ratio, or size of coding units.

[0240] Bitstream generator 210 can generate a bitstream based on an encoded input image. For example, bitstream generator 210 can generate a bitstream by entropy encoding of syntax elements based on a context model. Furthermore, image encoding device 200 can send the bitstream to image decoding device 100.

[0241] According to an embodiment, the encoder 220 of the image encoding device 200 can determine the shape of the encoding unit. For example, the encoding unit may have a square shape or a non-square shape, and information indicating a square shape or a non-square shape may be included in the block shape information.

[0242] According to an embodiment, encoder 220 can determine which shape the encoding unit will be divided into. Encoder 220 can determine the shape of at least one encoding unit included in the encoding unit, and bitstream generator 210 can generate a bitstream including division shape pattern information, which includes information about the shape of the encoding unit.

[0243] According to an embodiment, encoder 220 can determine whether to divide the coding unit. When encoder 220 determines that only one coding unit is included in the coding unit or the coding unit is not divided, bitstream generator 210 can generate a bitstream including division shape pattern information indicating that the coding unit is not divided. Furthermore, encoder 220 can divide the coding unit into multiple coding units, and bitstream generator 210 can generate a bitstream including division shape pattern information indicating that the coding unit is divided into multiple coding units.

[0244] According to an embodiment, information indicating how many coding units a coding unit will be divided into or in which direction the coding units will be divided can be included in the division shape pattern information. For example, the division shape pattern information may indicate that the coding units are divided in at least one of the vertical and horizontal directions, or it may indicate that the coding units are not divided.

[0245] Image encoding device 200 can determine information regarding the partition shape pattern based on the partitioning shape pattern of the encoding unit. Image encoding device 200 can determine a context model based on at least one of the shape, orientation, width-to-height ratio, or size of the encoding unit. Furthermore, image encoding device 200 can generate a bitstream based on the context model using information regarding the partition shape pattern used to partition the encoding unit.

[0246] To determine the context model, the image coding apparatus 200 may obtain an arrangement for corresponding the shape, orientation, width-to-height ratio, or size of the coding unit with an index for the context model. The image coding apparatus 200 may obtain the index for the context model from this arrangement based on at least one of the shape, orientation, width-to-height ratio, or size of the coding unit. The image coding apparatus 200 may determine the context model based on the index for the context model.

[0247] To determine the context model, the image coding device 200 may also determine the context model based on block shape information, including at least one of the shape, orientation, width-to-height ratio, or size of neighboring coding units adjacent to the coding unit. Furthermore, neighboring coding units may include at least one coding unit located to the lower left, left, upper left, upper, upper right, right, and lower right of the coding unit.

[0248] Furthermore, the image coding device 200 can compare the width of the upper neighboring coding unit with the width of the coding unit to determine the context model. Additionally, the image coding device 200 can compare the heights of the left and right neighboring coding units with the height of the coding unit. Furthermore, the image coding device 200 can determine the context model based on the comparison results.

[0249] The operation of the image encoding device 200 includes reference Figures 3 to 19 The operation of the image decoding device 100 described herein is similar in aspect, and therefore will not be described in detail.

[0250] Hereinafter, embodiments based on the technical concept of this disclosure will be described in detail.

[0251] Figure 20 This is a block diagram of the components of an image decoding device 2000 according to an embodiment.

[0252] Reference Figure 20 The image decoding device 2000 may include an acquirer 2010, a block determiner 2030, a predictive decoder 2050, and a reconstructor 2070. Figure 20 The acquirer 2010 shown may correspond to Figure 1 The bitstream acquirer 110 shown, and the block determiner 2030, predictive decoder 2050, and reconstructor 2070 can correspond to Figure 1 The decoder 120 shown.

[0253] The acquirer 2010, block determiner 2030, predictive decoder 2050, and reconstructor 2070 according to the embodiments can be implemented as at least one processor. The image decoding device 2000 may include at least one memory 2090 storing input and output data of the acquirer 2010, block determiner 2030, predictive decoder 2050, and reconstructor 2070. Furthermore, the image decoding device 2000 may also include a memory controller for controlling the data input and data output of the memory 2090.

[0254] The acquirer 2010 can receive a bitstream generated as a result of encoding an image. The acquirer 2010 can obtain syntax elements for decoding the image from the bitstream. The binary values ​​corresponding to the syntax elements can be included in the bitstream according to the hierarchical structure of the image. The acquirer 2010 can obtain the syntax elements by entropy encoding the binary values ​​included in the bitstream.

[0255] Figure 21 This is an example diagram of the structure of the bitstream 2100 generated based on the layered structure of the image.

[0256] Reference Figure 21 The bitstream 2100 may include a sequence parameter set 2110, a picture parameter set 2120, a group header 2130, and a block parameter set 2140.

[0257] Each of the sequence parameter set 2110, the picture parameter set 2120, the group header 2130, and the block parameter set 2140 includes information used in each layer according to the layered structure of the image.

[0258] In detail, the sequence parameter set 2110 includes information applied to or used for image sequences (e.g., encoded video sequences) that include one or more images.

[0259] Picture parameter set 2120 includes information used in a picture (e.g., one or more individual pictures within an encoded video sequence) or a picture (e.g., one or more individual pictures within an encoded video sequence). Picture parameter set 2120 may refer to sequence parameter set 2110.

[0260] Group header 2130 includes information applied to or used in a defined block group within an image, and may refer to image parameter set 2120 and sequence parameter set 2110. Group header 2130 may be a strip header.

[0261] In addition, the block parameter set 2140 includes information used in the blocks identified in the image and can be referenced to the group header 2130, the picture parameter set 2120, and the sequence parameter set 2110.

[0262] According to an embodiment, the block parameter set 2140 can be identified as at least one of the parameter sets of the CTU, the coding unit, the prediction unit, and the transformation unit, based on the hierarchical structure of the blocks determined in the image.

[0263] The acquirer 2010 can obtain information for decoding the image from the bitstream 2100 according to the hierarchical structure of the image, and the block determiner 2030, the predictive decoder 2050 and the reconstructor 2070, which will be described below, can perform the required operations by using the information obtained by the acquirer 2010.

[0264] Figure 21 The structure of bitstream 2100 shown is merely an example, and Figure 21 One or more parameter sets shown in the parameter set may be omitted in bitstream 2100, or parameter sets not shown (e.g., video parameter sets) may be included in bitstream 2100.

[0265] The block determiner 2030 can divide the current image into blocks and configure block groups that include at least one block in the current image. Here, a block may correspond to a parallel block, and a block group may correspond to a strip. A strip may be referred to as a parallel block group.

[0266] The predictive decoder 2050 can perform inter-frame prediction or intra-frame prediction on the lower-level blocks in the blocks divided from the current image to obtain prediction samples corresponding to the lower-level blocks. Here, the lower-level block can be at least one of a CTU, a coding unit, and a transform unit.

[0267] In the following description, blocks are limited to parallel blocks and block groups are limited to stripes. However, this is merely an example, and when there exists a block B that includes a set of blocks A, each block A may correspond to a block, and block B may correspond to a block group. For example, when a set of CTUs corresponds to parallel blocks, a CTU may be a block, and a parallel block may be a block group.

[0268] For reference Figures 3 to 16 As described, the block determiner 2030 can divide the current image to determine transform units, coding units, CTUs, parallel blocks, stripes, etc.

[0269] Figure 22 The stripes, parallel blocks, and CTUs identified in the current image 2200 are shown.

[0270] The current image 2200 is divided into multiple CTUs. The size of the CTU can be determined based on information obtained from the bitstream. CTUs can have a square shape with the same size.

[0271] A parallel block consists of one or more CTUs. A parallel block can be square or rectangular in shape.

[0272] A strip consists of one or more parallel blocks. Strips can be square or non-square in shape.

[0273] According to an embodiment, the block determiner 2030 can divide the current image 2200 into multiple CTUs based on information obtained from the bit stream, and can configure parallel blocks including at least one CTU and stripes including at least one parallel block in the current image 2200.

[0274] According to an embodiment, the block determiner 2030 can divide the current image 2200 into multiple parallel blocks based on information obtained from the bitstream, and each parallel block can be divided into one or more CTUs. Furthermore, the block determiner 2030 can configure stripes comprising at least one parallel block in the current image 2200.

[0275] According to an embodiment, the block determiner 2030 can divide the current image 2200 into one or more stripes based on information obtained from the bitstream, and can divide each strip into one or more parallel blocks. Furthermore, the block determiner 2030 can divide each parallel block into one or more CTUs.

[0276] The block determiner 2030 can use the address information of the stripes obtained from the bitstream to configure stripes in the current image 2200. The block determiner 2030 can configure stripes comprising one or more parallel blocks in the current image 2200 based on the address information of the stripes obtained from the bitstream. The address information of the stripes can be obtained from the video parameter set, sequence parameter set, picture parameter set, or group header of the bitstream.

[0277] Reference Figure 23 and Figure 24 This describes a method for configuring stripes in the current image 2200, executed by the block determiner 2030.

[0278] Figure 23 and Figure 24 This is a diagram illustrating a method for configuring stripes in the current image 2200.

[0279] When configuring parallel blocks in the current image 2200, the block determiner 2030 can configure a stripe including at least one parallel block in the current image 2200 based on the stripe address information obtained from the bit stream.

[0280] For reference Figure 23 The description states that stripes 2310, 2320, 2330, 2340 and 2350 can be determined in the current image 2200 according to the raster scanning direction 2300, and stripes 2310, 2320, 2330, 2340 and 2350 can be decoded sequentially according to the raster scanning direction 2300.

[0281] According to an embodiment, the address information may include the identifier value of the lower right parallel block located at the lower right end of the parallel blocks included in each of stripes 2310, 2320, 2330, 2340 and 2350.

[0282] Specifically, the address information for stripes 2310, 2320, 2330, 2340, and 2350 may include 9 (identification value of the lower right parallel block of the first stripe 2310), 7 (identification value of the lower right parallel block of the second stripe 2320), 11 (identification value of the lower right parallel block of the third stripe 2330), 12 (identification value of the lower right parallel block of the fourth stripe 2340), and 15 (identification value of the lower right parallel block of the fifth stripe 2350). According to an embodiment, when the fourth stripe 2340 is configured in the current image 2200, the fifth stripe 2350, as the last stripe, can be automatically identified, and therefore, the address information of the fifth stripe 2350 may not be included in the bitstream.

[0283] In order to configure the first stripe 2310, the block determiner 2030 can identify the upper left parallel block from the parallel blocks of the current image 2200, that is, the parallel block with an identifier value of 0. In addition, the block determiner 2030 can determine the region including parallel block 0 and parallel block 9 identified from the address information as the first stripe 2310.

[0284] Next, in order to configure the second stripe 2320, the block determiner 2030 can determine the parallel block with the smallest identifier value (i.e., parallel block 2) among the parallel blocks not included in the previous stripe (i.e., the first stripe 2310) as the upper left parallel block of the second stripe 2320. In addition, the block determiner 2030 can determine the region including parallel block 2 and parallel block 7 identified from the address information as the second stripe 2320.

[0285] Similarly, to designate the third stripe 2330, the block determiner 2030 can identify the parallel block with the smallest identifier value (i.e., parallel block 10) among the parallel blocks not included in the previous stripes (i.e., the first stripe 2310 and the second stripe 2320) as the upper left parallel block of the third stripe 2330. Furthermore, the block determiner 2030 can identify the region including parallel block 10 and parallel block 11 identified from the address information as the third stripe 2330.

[0286] In other words, according to the embodiment, stripes can be configured in the current image 2200 by using only the identification information of the lower right parallel block included in the bitstream.

[0287] According to another embodiment, as address information for determining stripes, the acquirer 2010 can obtain the identifier values ​​of the upper left parallel block and the lower right parallel block included in each stripe, and the block determiner 2030 can configure stripes in the current image 2200 based on the information obtained by the acquirer 2010. Because the upper left parallel block and the lower right parallel block included in each stripe can be identified from the address information, the block determiner 2030 can configure the region including the upper left parallel block and the lower right parallel block identified from the address information as a stripe.

[0288] According to another embodiment, as address information for configuring stripes, the acquirer 2010 can obtain the identifier value of the upper left parallel block included in each stripe, the width of each stripe, and the height of each stripe, and the block determiner 2030 can configure stripes in the current image 2200 based on the information obtained by the acquirer 2010.

[0289] For example, Figure 23 The address information of the second stripe 2320 may include 2 as the identifier value of the upper left parallel block, 2 as the width of the stripe, and 2 as the height of the stripe. Here, the indication that the width and height are 2 indicates that there are two parallel block rows and two parallel block columns in the width and height directions of the second stripe 2320.

[0290] According to the embodiment, the upper left parallel block of the first strip 2310 is fixed as parallel block 0, and therefore, the identifier value of the upper left parallel block of the first strip 2310 may not be included in the bit stream.

[0291] According to another embodiment, the width and height of the stripe obtained from the bitstream can be values ​​obtained by dividing the number of parallel block rows and columns arranged in the width and height directions of the stripe by a predetermined scaling factor. In other words, when Figure 23 When the address information of the second strip 2320 indicates 2 as the identifier value of the upper left parallel block, 1 as the width of the strip, and 1 as the height of the strip, the block determiner 2030 can multiply 1 as the width of the strip and 1 as the height of the strip by a predetermined scaling factor (e.g., 2) in order to identify that there are two parallel block rows and two parallel block columns in the width and height directions of the strip.

[0292] The block determiner 2030 can determine the first stripe 2310 to the fifth stripe 2350 in the current image 2200 based on the address information of the first stripe 2310 to the fifth stripe 2350. When the address information is used to determine up to the fourth stripe 2340 in the current image 2200, the fifth stripe 2350 can be determined automatically, and therefore, the address information of the last stripe may not be included in the bitstream.

[0293] According to another embodiment, in addition to the identifier value of the upper left parallel block of the corresponding strip, the width of the strip, and the height of the strip, the address information of the strip to be determined in the current image 2200, including the parallel block located in the first row or the parallel block located in the first column, may also include a value indicating the number of strips subsequently present in the right-hand or downward direction of the corresponding strip. The value indicating the number of strips subsequently present in the right-hand or downward direction of the strip may be replaced by a value indicating the number of strips arranged in the width or height direction of the strip.

[0294] The address information of the first stripe 2310 may include information about the existence of a stripe in the right direction (i.e., the second stripe 2320) and a stripe in the downward direction (i.e., the fourth stripe 2340). Because the first stripe 2310 includes both parallel blocks located in the first row and parallel blocks located in the first column of the image 2200, the address information of the first stripe 2310 may include values ​​indicating the number of stripes subsequently existing in the right direction and values ​​indicating the number of stripes subsequently existing in the downward direction.

[0295] Because the second stripe 2320 only includes parallel blocks located in the first row, the address information of the second stripe 2320 may include a value indicating the number of stripes that subsequently exist in the direction below the stripe.

[0296] Because the value indicating the number of stripes subsequently present in the right and / or down directions is included in the address information, the last stripe in the width direction of the current image 2200 ( Figure 23 The address information for the second stripe 2320 and / or the fifth stripe 2350 in the image can omit the stripe width, and the last stripe in the height direction of the current image 2200 ( Figure 23 The address information for the fourth stripe 2340 and / or the fifth stripe 2350 in the bitstream can omit the stripe height. Because the block determiner 2030 may already know that the first stripe 2310 has a subsequent stripe existing in the width direction of the current image 2200, the block determiner 2030 can deduce the width of the subsequent stripe of the first stripe 2310 by considering the width of the current image 2200, even when the value indicating the width of the subsequent stripe is not included in the bitstream. Figure 23 In this context, because there are four parallel blocks in the width direction of the current image 2200 and two parallel blocks in the width direction of the first stripe 2310, it is possible to identify two parallel blocks in the width direction of the second stripe 2320 that subsequently exists relative to the first stripe 2310. Similarly, because the block determiner 2030 may know that the first stripe 2310 has a subsequent stripe in the height direction of the current image 2200, the block determiner 2030 can deduce the height of the subsequent stripe of the first stripe 2310 even when the value indicating the height of the subsequent stripe is not included in the bitstream.

[0297] According to another embodiment, the acquirer 2010 can obtain segmentation information from the bitstream for dividing the current image 2200 into stripes, and the block determiner 2030 can divide the current image 2200 into stripes according to the segmentation information. Here, the segmentation information may indicate, for example, a four-segment division, a two-segment division in height, a two-segment division in width, etc.

[0298] The block determiner 2030 can divide each strip in the strips obtained when initially dividing the current image 2200 according to the division information, and can obtain smaller strips in layers.

[0299] like Figure 24 As shown, the block determiner 2030 can determine two regions 2410 and 2420 by dividing the width of the current image 2200 into two regions 2410 and 2420 according to the division information, and can determine two regions 2412 and 2414 by dividing the height of the left region 2410 into two regions 2412 and 2414 according to the division information of the left region 2410. When the division information of the right region 2420 indicates no division, and the regions 2412 and 2414 divided from the left region 2410 are not further divided, the block determiner 2030 can configure the upper left region 2412 as the first strip, the right region 2420 as the second strip, and the lower left region 2414 as the third strip.

[0300] According to another embodiment, the block determiner 2030 can configure stripes in the current image 2200 according to pre-configured mapping information, and can further divide at least one stripe in the current image 2200 or merge two or more stripes according to correction information obtained from the bitstream to configure the final stripes. The mapping information may include address information of the stripes located in the image. For example, the block determiner 2030 can initially configure the stripes in the image 2200 according to mapping information obtained from the video parameter set or sequence parameter set of the bitstream, and can finally configure the stripes in the image 2200 according to correction information obtained from the picture parameter set.

[0301] When parallel blocks and stripes are determined in the current image, the block determiner 2030 can perform inter-frame prediction on at least one coding unit among the coding units included in the parallel block. Here, a method for configuring a list of reference images for inter-frame prediction is described.

[0302] Reference Figure 20 The predictive decoder 2050 performs predictive decoding on coding units included in parallel blocks determined in the current image. The predictive decoder 2050 can perform predictive decoding on coding units via inter-frame prediction or intra-frame prediction. According to inter-frame prediction, predicted samples of the coding units are obtained based on reference blocks in the reference image indicated by motion vectors, and reconstructed samples of the coding units are obtained based on the predicted samples and residual data obtained from the bitstream. Depending on the prediction mode, the residual data may not be included in the bitstream, and in this case, the predicted samples may be determined as reconstructed samples.

[0303] For inter-frame prediction, it may be necessary to construct a list of reference images, including reference images. According to an embodiment, the acquirer 2010 may obtain information indicating multiple first reference image lists from a set of sequence parameters of the bitstream. The information indicating multiple first reference image lists may include the display order (output order) of the decoded images and / or the processing order of the decoded images, such as the frame order count (POC) related value of the reference images. Multiple first reference image lists are used in image sequences including the current image.

[0304] According to an embodiment, the information indicating a plurality of first reference image lists may include the number of first reference image lists. In this case, the predictive decoder 2050 may construct a first reference image list corresponding to the number of first reference image lists identified from the bitstream. In this case, the predictive decoder 2050 may construct the first reference image list according to the same method performed by the image encoding device 3300.

[0305] When encoding coding units included in a predetermined slice, using the plurality of first reference image lists for an image sequence may be inappropriate depending on the characteristics of the image. Therefore, when no reference image list exists in the plurality of first reference image lists that can be used for inter-frame prediction of coding units in the current slice, a new reference image list can be obtained from the group header. However, in this case, the bit rate may increase because the new reference image list is included in the group header. Therefore, a method is needed for constructing an optimal reference image list to be used for the current slice by using a plurality of first reference image lists transmitted via a sequence parameter set signal.

[0306] According to an embodiment, the acquirer 2010 can obtain from the group header of the bitstream an indicator indicating at least one of a plurality of first reference image lists for the image sequence. Furthermore, the predictive decoder 2050 can obtain a second reference image list modified and improved from the first reference image list indicated by the indicator.

[0307] A second reference image list can be obtained by replacing at least one reference image in the first reference image list indicated by the indicator with another reference image, by changing the order of one or more reference images, or by adding a new reference image to the first reference image list.

[0308] To construct a second reference image list, the acquirer 2010 may obtain modification and improvement information from the group header of the bitstream. The modification and improvement information may include the POC-related values ​​of reference images to be removed from the first reference image list indicated by the indicator, the POC-related values ​​of reference images to be added to the second reference image list, the difference between the POC-related values ​​of the reference images to be removed from the first reference image list and the POC-related values ​​of the reference images to be added to the second reference image list, information for changing the order of the images, etc. According to an embodiment, in addition to the group header of the bitstream, the modification and improvement information may also be obtained from a parameter set (e.g., a picture parameter set).

[0309] When a second list of reference images is obtained, the prediction decoder 2050 can perform prediction decoding on the coding units included in the strip based on at least one reference image from the reference images included in the second list of reference images to obtain prediction samples of the coding units.

[0310] The predictive decoder 2050 can predictively decode coding units included in the next strip by using one of the multiple first reference image lists for the image sequence, excluding the first reference image list indicated by the indicator, and using a second reference image list. In other words, the second reference image list obtained for the current strip can also be used for the next strip. Specifically, an indicator can be newly obtained indicating the reference image list to be used in the next strip from one of the multiple first reference image lists, excluding the first reference image list indicated by the indicator obtained for the current strip, and the coding units included in the next strip can be predicted and decoded based on the reference image list indicated by the indicator or a reference image list modified and improved from the reference image list indicated by the indicator. Therefore, even when a new reference image list is not sent via a sequence parameter set or group header signal, a suitable reference image list for predictively decoding coding units of the strip can be constructed simply by updating the previous reference image list.

[0311] In the following text, refer to Figures 25 to 30 Describe a method for obtaining a second list of reference images obtained by modifying and improving a first list of reference images.

[0312] Figure 25 This is an example diagram showing a list of multiple first reference images 2510, 2520 and 2530 obtained from the sequence parameter set.

[0313] Figure 25 Three first reference image lists, 2510, 2520, and 2530, are shown. This is merely an example, and the number of first reference image lists obtained from the sequence parameter set can be modified differently.

[0314] Reference Figure 25 The first reference image list 2510, 2520, and 2530 may include a short-term type reference image or a long-term type reference image. The short-term type reference image indicates an image designated as a short-term type in the reconstructed image stored in the decoded picture buffer (DPB), and the long-term type reference image indicates an image designated as a long-term type in the reconstructed image stored in the DPB.

[0315] The reference images included in the first reference image lists 2510, 2520, and 2530 can be specified by POC-related values. Specifically, the short-term type reference image can be specified by the difference (i.e., the difference value) between the POC of the current image and the POC of the short-term type reference image, and the long-term type reference image can be specified by the least significant bit (LSB) of the POC of the long-term type reference image. The long-term type reference image can also be specified by the most significant bit (MSB) of the POC of the long-term type reference image.

[0316] According to embodiments, the first reference image lists 2510, 2520, and 2530 may include only short-term type reference images or only long-term type reference images. That is, Figure 25 All reference images shown can be either short-term or long-term type reference images. Furthermore, according to embodiments, some of the first reference image lists 2510, 2520, and 2530 may include only short-term type reference images, and other first reference image lists may include only long-term type reference images.

[0317] Figure 26 This is a diagram illustrating a method for obtaining a second list of reference images.

[0318] The predictive decoder 2050 can obtain a second reference image list 2600 by changing at least one reference image in the first reference image list 2510 indicated by an indicator to another reference image. (Reference) Figure 26 It can identify that the short-term type reference image with a difference value of -1, the long-term type reference image with an LSB of 10, and the short-term type reference image with a difference value of -3 in the first reference image list 2510 are replaced by the short-term type reference image with a difference value of -2, the long-term type reference image with an LSB of 8, and the short-term type reference image with a difference value of -5 in the second reference image list 2600, respectively. Figure 26 This illustrates how all reference images in the first reference image list 2510 can be replaced by other reference images. However, this is merely an example, and only one or more reference images in the first reference image list 2510 may be replaced by other reference images.

[0319] According to an embodiment, the predictive decoder 2050 may replace only reference images of a specific type (e.g., long-term type reference images) included in the first reference image list 2510 with another long-term type reference image. That is, short-term type reference images in the first reference image list 2510 may be retained unchanged in the second reference image list 2600, and only long-term type reference images may be replaced with another long-term type reference image based on information obtained from the bitstream. (Refer to...) Figure 26In the first reference image list 2510, only reference images of a specific type (i.e., long-term type reference images with an LSB of 10) can be replaced in the second reference image list 2600 by long-term type reference images with an LSB of 8. According to an embodiment, long-term type reference images in the first reference image list 2510 can be retained unchanged in the second reference image list 2600, and only short-term type reference images in the first reference image list 2510 can be replaced by another short-term type reference image.

[0320] In order to replace the reference image, the acquirer 2010 may obtain the POC correlation value of the new reference image from the group header of the bitstream, and the predictive decoder 2050 may include the reference image indicated by the POC correlation value obtained by the acquirer 2010 in the second reference image list 2600.

[0321] In order to specify the reference image to be replaced by the new reference image (i.e., the reference image to be removed) from the reference images included in the first reference image list 2510, the obtainr 2010 may also obtain the index of the reference image to be removed from the first reference image list 2510 from the bitstream. When all reference images included in the first reference image list 2510 are to be removed, the index of the reference image to be removed from the first reference image list 2510 may not be included in the bitstream.

[0322] As described above, when a reference image of a certain type is predetermined to be removed from the first reference image list 2510, the index of the reference image to be removed may not be included in the bitstream, and the prediction decoder 2050 may remove the predetermined reference image from the reference images included in the first reference image list 2510, and may include the reference image indicated by the POC correlation value obtained from the bitstream in the second reference image list 2600.

[0323] According to an embodiment, the information indicating a new reference image to be included in the second reference image list 2600 may be the difference between the POC correlation value of the new reference image and the POC correlation value of a reference image to be removed from the first reference image list 2510. For example, in Figure 26 In the first reference image list 2510, a reference image with an LSB of 10 is replaced in the second reference image list 2600 by a reference image with an LSB of 8. Therefore, the information indicating the new reference image may include 2 (i.e., 2 = 10 - 8). The predictive decoder 2050 may deduce the POC correlation value of the reference image to be newly included in the second reference image list 2600 based on the difference between the POC correlation values ​​and the POC correlation value of the reference image removed from the first reference image list 2510.

[0324] According to an embodiment, new reference images can be added to the second reference image list 2600 in the order in which reference images are removed from the first reference image list 2510 indicated by the indicator. For example... Figure 26 As shown, when a long-term type reference image assigned index 1 is removed from the first reference image list 2510, index 1 can also be assigned to a new reference image.

[0325] Figure 27 This is an illustration used to describe another method for obtaining a second list of reference images.

[0326] The predictive decoder 2050 can obtain a second reference image list 2700 by excluding reference images of a specific type from the reference images in the first reference image list 2510 indicated by an indicator from a plurality of first reference image lists used for the image sequence. (Refer to...) Figure 27 It can be identified that long-term type reference images in the first reference image list 2510 indicated by the indicator are not included in the second reference image list 2700.

[0327] According to an embodiment, the predictive decoder 2050 may also obtain a second reference image list 2700 that excludes short-term type reference images from the reference images in the first reference image list 2510.

[0328] Figure 28 This is an illustration used to describe another method for obtaining a second list of reference images.

[0329] The predictive decoder 2050 can also obtain a second reference image list 2800 by changing the order of reference images in the first reference image list 2510 indicated by the indicator according to modification and improvement information obtained from the group header of the bitstream. Here, the order of all reference images in the first reference image list 2510, or the order of one or more reference images in the first reference image list 2510, can be changed according to the modification and improvement information.

[0330] For example, modification and improvement information obtained from the group header of the bitstream may include indices of reference images in the first reference image list 2510, arranged according to the order in which the reference images will be changed. More specifically, in Figure 28In the second reference image list 2800, when the reference image with index 0, index 1, and index 2 in the first reference image list 2510 are respectively changed to reference image with index 1, index 2, and index 0, the bitstream header may include (2, 0, 1) as modification and improvement information. The predictive decoder 2050 may assign index 0 to the reference image assigned index 2 in the first reference image list 2510, assign index 1 to the reference image assigned index 0, and assign index 2 to the reference image assigned index 1 to construct the second reference image list 2800.

[0331] As another example, modification and improvement information obtained from the group header of the bitstream may include the indices of reference images whose order in the first reference image list 2510 must be changed. Specifically, in Figure 28 In the first reference image list 2510, when the order of the reference image with index 1 and the reference image with index 2 is to be changed, the group header of the bitstream may include (1, 2) as modification and improvement information. The predictive decoder 2050 may assign index 2 to the reference image in the first reference image list 2510 that is assigned index 1, and assign index 1 to the reference image that is assigned index 2, to construct the second reference image list 2800.

[0332] Figure 29 This is an illustration used to describe another method for obtaining a second list of reference images.

[0333] The number of first reference images indicated by the indicator in the multiple first reference image lists used for an image sequence can be multiple. That is, such as... Figure 29 As shown, the indicator can indicate a first reference image list 2910 that includes only short-term type reference images and a first reference image list 2920 that includes only long-term type reference images.

[0334] The predictive decoder 2050 can obtain a second reference image list 2930, which includes short-term type reference images and long-term type reference images included in the first reference image lists 2910 and 2920 indicated by indicators. Here, in the second reference image list 2930, long-term type reference images can be assigned higher indices than short-term type reference images. Conversely, in the second reference image list 2930, short-term type reference images can be assigned higher indices than long-term type reference images.

[0335] According to an embodiment, the acquirer 2010 can obtain the order information of the short-term type reference image and the long-term type reference image from the bit stream, and the predictive decoder 2050 can assign an index to the short-term type reference image and the long-term type reference image included in the second reference image list 2930 according to the obtained order information.

[0336] According to another embodiment, the first reference image list 2910 and the first reference image list 2920 may include at least one reference image, regardless of the type of the reference image. In this case, when a short-term type reference image exists in the first reference image list 2910 indicated by the indicator and a long-term type reference image exists in the first reference image list 2920 indicated by the indicator, the prediction decoder 2950 may obtain a second reference image list 2930 that includes the short-term type reference image included in the first reference image list 2910 and the long-term type reference image included in the first reference image list 2920. Optionally, when a long-term type reference image exists in the first reference image list 2910 indicated by the indicator and a short-term type reference image exists in the first reference image list 2920 indicated by the indicator, the prediction decoder 2950 may obtain a second reference image list 2930 that includes the long-term type reference image included in the first reference image list 2910 and the short-term type reference image included in the first reference image list 2920.

[0337] Figure 30 This is an illustration used to describe another method for obtaining a second list of reference images.

[0338] The first reference image list 3010 indicated by the indicator may include only short-term type reference images. According to an embodiment, the first reference image list 3010 indicated by the indicator may include only long-term type reference images.

[0339] When the first reference image list 3010 includes only short-term type reference images, the acquirer 2010 can obtain the POC correlation values ​​of the long-term type reference images to be included in the second reference image list 3030 from the bitstream, and can construct a second reference image list 3030 including the long-term type reference images indicated by the obtained POC correlation values ​​and the short-term type reference images included in the first reference image list 3010. That is, the first reference image list 3010, which includes only short-term type reference images, can be sent by signaling via a sequence parameter set, and the POC correlation values ​​of the long-term type reference images can be sent by signaling via a group header.

[0340] When the reference image list is sent via the sequence parameter set instead of the group header, it is not necessary to send the reference image list for each block group, and therefore, the compression ratio can be improved due to the reduction in overhead. For example, when the prediction structure repeats for each group of pictures (GOP), the reference list may need to be sent repeatedly for each GOP. Sending a reference image list that may be sent frequently via the sequence parameter set can further reduce the bit rate.

[0341] Here, the availability of the sequence parameter set can vary depending on the type of reference image (i.e., whether it is a long-term or short-term type). Short-term type reference images are related to the pattern of prediction structure repetition, as in the example above, while long-term type reference images are highly correlated with the correlation between the current frame and long-term reference images. For example, although the prediction structure repeats for each GOP, when the long-term type reference image is no longer valid because the image content has completely changed due to screen transitions, etc., a reference list for short-term type reference images can be obtained from the sequence parameter set, and long-term type reference images can be sent separately via the group header, thus avoiding sending the entire reference list via the group header.

[0342] According to an embodiment, when only long-term type reference images are included in the first reference image list, the acquirer 2010 can obtain the POC correlation values ​​of short-term type reference images to be included in the second reference image list from the bitstream, and can construct a second reference image list including the short-term type reference images indicated by the POC correlation values ​​and the long-term type reference images included in the first reference image list.

[0343] When constructing the second reference image list 3030, the reference images indicated by the POC correlation values ​​obtained from the group header of the bitstream can be assigned a higher or lower index than the reference images included in the first reference image list 3010.

[0344] As described above, when the second reference image list is fully constructed, the prediction decoder 2050 can perform inter-frame prediction of the coding unit based on the reference images included in the second reference image list. As a result of the inter-frame prediction, prediction samples corresponding to the coding unit can be obtained.

[0345] The reconstructor 2070 obtains reconstructed samples of the coding unit by using predicted samples. According to an embodiment, the reconstructor 2070 can obtain reconstructed samples of the coding unit by adding the residual data obtained from the bitstream to the predicted samples.

[0346] The reconstructor 2070 can perform luminance mapping on the predicted samples of the coding unit before obtaining the reconstructed samples.

[0347] A luminance mapping modifies the luminance values ​​of predicted samples based on parameters obtained from the bitstream, and may, for example, correspond to a tone mapping.

[0348] According to an embodiment, the acquirer 2010 may obtain parameters for luminance mapping from one or more sets of post-processing parameters of the bitstream. Each of the one or more sets of post-processing parameters may include parameters for luminance mapping or adaptive loop filtering, which will be described below.

[0349] Parameters used for brightness mapping may include, for example, the range of brightness values ​​to be changed, the difference in brightness values ​​to be applied to the predicted samples, etc.

[0350] Figure 31 This is a diagram showing a bitstream that includes multiple sets of post-processing parameters for luminance mapping or adaptive loop filtering.

[0351] In addition to the Sequence Parameter Set (SPS) 3110, Picture Parameter Set (PPS) 3120, Group Header (GH) 3130, and Block Parameter Set (BPS) 3140 mentioned above, the bitstream 3100 may also include multiple post-processing parameter sets 3150a, 3150b, and 3150c. Unlike SPS 3110, PPS 3120, GH 3130, and BPS 3140, post-processing parameter sets 3150a, 3150b, and 3150c can be included in the bitstream regardless of the image's layered structure.

[0352] Identifiers can be assigned to each of the post-processing parameter sets 3150a, 3150b, and 3150c to identify the post-processing parameter sets 3150a, 3150b, and 3150c. According to an embodiment, identifiers 0, 1, and 2 can be assigned to post-processing parameter set A 3150a, post-processing parameter set B 3150b, and post-processing parameter set C 3150c, respectively.

[0353] One or more of the post-processing parameter sets 3150a, 3150b, and 3150c include parameters for luminance mapping, and the other post-processing parameter sets include parameters for adaptive loop filtering. For example, post-processing parameter sets A and C may include parameters for luminance mapping, and post-processing parameter set B may include parameters for adaptive loop filtering.

[0354] The acquirer 2010 can obtain from the PPS 3120, GH 3130, or BPS 3140 an identifier indicating one of the multiple post-processing parameter sets 3150a, 3150b, and 3150c used for brightness mapping of the predicted samples. The reconstructor 2070 can change the brightness value of the predicted samples by using parameters obtained from the post-processing parameter set indicated by the identifier.

[0355] When the acquirer 2010 obtains an identifier from the PPS 3120, the set of post-processing parameters indicated by that identifier is used for the predicted samples derived in the current image. Similarly, when the acquirer 2010 obtains an identifier from the GH 3130, the set of post-processing parameters indicated by that identifier is used for the predicted samples derived in the current strip. Furthermore, when the acquirer 2010 obtains an identifier from the BPS 3140, the set of post-processing parameters indicated by that identifier is used for the predicted samples derived in the current block.

[0356] According to an embodiment, the acquirer 2010 can obtain from the bitstream an identifier indicating any one of a plurality of post-processing parameter sets 3150a, 3150b, and 3150c, as well as correction information. Here, the correction information may include information for changing the parameters included in the post-processing parameter set indicated by the identifier. For example, the correction information may include the difference between the values ​​of the parameters included in the post-processing parameter set indicated by the identifier and the values ​​of the parameters to be changed.

[0357] The reconstructor 2070 can correct the parameters of the post-processing parameter set indicated by the identifier based on the correction information, and can change the brightness value of the predicted sample by using the corrected parameters.

[0358] According to another embodiment, the identifier obtained from the bitstream can indicate multiple sets of post-processing parameters. In this case, the reconstructor 2070 can construct a new set of parameters by combining one or more parameters included in each of the post-processing parameter sets indicated by the identifier, and can perform brightness mapping on the predicted samples by using the newly constructed set of parameters.

[0359] The reconstructor 2070 obtains reconstructed samples corresponding to the current coding unit by using prediction samples generated as a result of prediction decoding or prediction samples that have undergone luminance mapping. Once the reconstructed samples are obtained, the reconstructor 2070 may apply adaptive loop filtering to the reconstructed samples.

[0360] Adaptive loop filtering refers to one-dimensional filtering performed on the sample values ​​of reconstructed samples by using filter coefficients transmitted via a bitstream signal. Adaptive loop filtering can be performed separately for luminance and chrominance values. Filter coefficients can include those for a one-dimensional filter. Each filter coefficient of a one-dimensional filter can be represented as the difference between consecutive filter coefficients, and this difference can be transmitted via a bitstream signal.

[0361] As described above, one or more post-processing parameter sets in the post-processing parameter set include parameters for luminance mapping, and other post-processing parameter sets include parameters for adaptive loop filtering (e.g., filter coefficients). For example, post-processing parameter set A 3150a and post-processing parameter set B 3150b may include parameters for adaptive loop filtering, and post-processing parameter set C 3150c may include parameters for luminance mapping.

[0362] Acquirer 2010 can obtain identifiers from PPS 3120, GH 3130, or BPS 3140 indicating post-processing parameter sets used for adaptive loop filtering of reconstructed samples from multiple post-processing parameter sets 3150a, 3150b, and 3150c. Reconstructor 2070 can filter the reconstructed samples using parameters obtained from the post-processing parameter sets indicated by the identifiers. When acquirer 2010 obtains an identifier from PPS, the post-processing parameter set indicated by that identifier is used for reconstructed samples derived in the current image, and when acquirer 2010 obtains an identifier from GH, the post-processing parameter set indicated by that identifier is used for reconstructed samples derived in the current strip. Furthermore, when acquirer 2010 obtains an identifier from BPS, the post-processing parameter set indicated by that identifier is used for reconstructed samples derived in the current block.

[0363] According to an embodiment, the acquirer 2010 can obtain from the bitstream an identifier indicating any one of a plurality of post-processing parameter sets 3150a, 3150b, and 3150c, as well as correction information. Here, the correction information may include information for changing the filter coefficients included in the post-processing parameter set indicated by the identifier. For example, the correction information may include the difference between the values ​​of the filter coefficients included in the post-processing parameter set indicated by the identifier and the values ​​of the filter coefficients to be changed.

[0364] The reconstructor 2070 can correct the filter coefficients of the post-processing parameter set indicated by the identifier based on the correction information, and can filter the reconstructed samples by using the corrected filter coefficients.

[0365] According to another embodiment, the identifier obtained from the bitstream can indicate multiple sets of post-processing parameters. In this case, the reconstructor 2070 can construct a new set of filter coefficients by combining one or more filter coefficients included in each set of post-processing parameter sets indicated by the identifier, and can filter the reconstructed samples by using the newly constructed set of filter coefficients.

[0366] According to another embodiment, when an identifier obtained from the bitstream indicates multiple sets of post-processing parameters, the reconstructor 2070 can filter the luminance value of the reconstructed sample by using filter coefficients included in any one of the post-processing parameter sets indicated by the identifier, and can filter the chrominance value of the reconstructed sample by using filter coefficients included in another set of post-processing parameters indicated by the identifier.

[0367] According to another embodiment, the acquirer 2010 can obtain from the bitstream an identifier indicating any set of post-processing parameters and filter coefficient information. In this case, the reconstructor 2070 can combine one or more filter coefficients included in the set of post-processing parameters indicated by the identifier with filter coefficients transmitted by signal via the bitstream, and can filter the reconstructed samples by using the combined set of filter coefficients.

[0368] According to an embodiment, the reconstructor 2070 may additionally perform deblocking filtering on the reconstructed samples for which adaptive loop filtering has been performed.

[0369] As described above, the predictive decoder 2050 can decode the coding units included in the current stripe via inter-frame prediction. According to an embodiment, when the coding units are decoded, the boundary of the current stripe can be regarded as the frame boundary.

[0370] According to an embodiment, in the decoder-side motion vector correction (DMVR) mode where the decoder directly derives the motion vector of the coding unit, the predictive decoder 2050 can limit the search range to the boundary of the region in the reference image that is located at the same position as the current strip when deriving the motion vector of the current coding unit.

[0371] According to an embodiment, when the motion vector of the current coding unit, transmitted via a bitstream signal, indicates a block outside the boundary of a region in the reference image that is located at the same position as the current strip, prediction samples can be obtained by filling in the region that is located at the same position as the current strip.

[0372] According to an embodiment, the predictive decoder 2050 can treat the boundaries of stripes as the boundaries of the frame in a bidirectional optical flow (BIO) processing mode, and can perform predictive decoding on the current coding unit. The BIO processing mode indicates sample-by-sample motion vector improvement processing performed for block-by-block motion compensation for bidirectional prediction.

[0373] When the acquirer 2010 performs entropy encoding on the binary values ​​included in the bitstream based on CABAC, the acquirer 2010 can selectively apply wavefront parallel processing (WPP) by taking into account the number of parallel blocks included in the stripe. WPP instructs the processing of the current CTU to be performed after the processing of the CTU on the upper right side is completed, for parallel encoding / decoding. Specifically, WPP configures the probabilistic model of the first CTU in each row using probabilistic information obtained by processing the second CTU in the upper row.

[0374] When the strip consists of only one parallel block, the acquirer 2010 can configure a probabilistic model for the CTUs included in the parallel block based on WPP, and when the strip consists of multiple parallel blocks, the acquirer 2010 may not apply WPP to the CTUs included in the parallel blocks.

[0375] Figure 32 This is a diagram used to describe an image decoding method according to an embodiment.

[0376] In operation S3210, the image decoding device 2000 obtains information from the SPS of the bitstream indicating a plurality of first reference image lists for an image sequence including the current image. The plurality of first reference image lists may include at least one of short-term type reference images and long-term type reference images.

[0377] In operation S3220, the image decoding device 2000 configures blocks and block groups, including at least one block, in the current image. The block can be a parallel block, and the block group can be a stripe.

[0378] According to an embodiment, the image decoding device 2000 can divide the current image into multiple CTUs based on information obtained from the bitstream, and can configure parallel blocks including at least one CTU and stripes including at least one parallel block in the current image.

[0379] According to an embodiment, the image decoding device 2000 can divide the current image into multiple parallel blocks based on information obtained from the bitstream, and each parallel block can be divided into one or more CTUs. Furthermore, the block determiner 2030 can configure stripes in the current image that include at least one parallel block.

[0380] According to an embodiment, the image decoding device 2000 can divide the current image into one or more stripes based on information obtained from the bitstream, and can divide each stripe into one or more parallel blocks. Furthermore, the block determiner 2030 can divide each parallel block into one or more CTUs.

[0381] As described above, the image decoding device 2000 can configure stripes in the current image based on the address information obtained from the bitstream.

[0382] In operation S3230, the image decoding device 2000 can obtain an indicator for a current block group including the current block in the current image from the GH of the bitstream, and can obtain a second reference image list based on the first reference image list obtained from the indicator. The image decoding device 2000 can also obtain modification and improvement information for obtaining the second reference image list and an indicator from the bitstream. The modification and improvement information may include at least one of the following: the POC correlation value of a reference image to be removed from the first reference image list indicated by the indicator; the POC correlation value of a reference image to be added to the second reference image list; the difference between the POC correlation value of a reference image to be removed from the first reference image list and the POC correlation value of a reference image to be added to the second reference image list; and information for changing the order of the images.

[0383] In operation S3240, the image decoding device 2000 performs predictive decoding on the lower-level block of the current block based on the reference image included in the second reference image list.

[0384] When a predicted sample corresponding to a lower-level block is obtained as a result of predictive decoding, the image decoding device 2000 can specify a set of post-processing parameters for brightness mapping of the predicted sample according to an identifier indicating at least one set of post-processing parameters from a plurality of post-processing parameter sets. Furthermore, the image decoding device 2000 can change the brightness value of the predicted sample by using parameters included in the set of post-processing parameters indicated by the identifier.

[0385] According to an embodiment, the image decoding device 2000 can obtain reconstructed samples based on predicted samples obtained as a result of predictive decoding or predicted samples after brightness mapping, and can perform adaptive loop filtering on the reconstructed samples. To this end, the image decoding device 2000 can specify a set of post-processing parameters for adaptive loop filtering according to an identifier indicating at least one set of post-processing parameters from a plurality of post-processing parameter sets. Furthermore, the image decoding device 2000 can filter the reconstructed samples using parameters included in the set of post-processing parameters indicated by the identifier.

[0386] Figure 33 This is a diagram illustrating the components of an image encoding device 3300 according to an embodiment.

[0387] Reference Figure 33 The image encoding device 3300 includes a block determiner 3310, a predictive encoder 3330, a reconstructor 3350, and a generator 3370. Figure 33 The generator 3370 shown can correspond to Figure 2 The bitstream generator 210 shown, and the block determiner 3310, predictive encoder 3330, and reconstructor 3350 can correspond to Figure 2 The encoder 220 shown.

[0388] The block determiner 3310, predictive encoder 3330, reconstructor 3350, and generator 3370 according to the embodiment can be implemented as at least one processor. The image encoding device 3300 may include at least one memory 3390 storing input and output data of the block determiner 3310, predictive encoder 3330, reconstructor 3350, and generator 3370. Furthermore, the image encoding device 3300 may include a memory controller for controlling the data input and data output of the memory 3390.

[0389] The block determiner 3310 can divide the current image into blocks and configure block groups comprising at least one block in the current image. Here, the block may correspond to a parallel block, and the block group may correspond to a strip. A strip may be referred to as a parallel block group.

[0390] For reference Figures 3 to 16 The block determiner 3310 can determine transform units, coding units, CTUs, parallel blocks, stripes, etc. by dividing the current image.

[0391] According to an embodiment, the block determiner 3310 can divide the current image into multiple CTUs and can configure parallel blocks including at least one CTU and stripes including at least one parallel block in the current image.

[0392] According to an embodiment, the block determiner 3310 can divide the current image into multiple parallel blocks, and each parallel block can be divided into one or more CTUs. Furthermore, the block determiner 3310 can configure stripes comprising at least one parallel block in the current image.

[0393] According to an embodiment, the block determiner 3310 can divide the current image into one or more stripes, and each stripe can be divided into one or more parallel blocks. Furthermore, the block determiner 3310 can divide each parallel block into one or more CTUs.

[0394] The predictive encoder 3330 performs inter-frame or intra-frame prediction on the lower-level blocks divided from the current image to obtain prediction samples corresponding to the lower-level blocks. Here, the lower-level block can be at least one of a CTU, a coding unit, and a transform unit.

[0395] The predictive encoder 3330 can predictively encode coding units through inter-frame prediction or intra-frame prediction. According to inter-frame prediction, the predicted samples of the current coding unit can be obtained based on reference blocks in the reference image indicated by motion vectors, and the residual data corresponding to the difference between the predicted samples and the current coding unit can be sent to the image decoding device 2000 via a bitstream. Depending on the prediction mode, the residual data may not be included in the bitstream.

[0396] The following section describes a method for constructing a list of reference images for inter-frame prediction.

[0397] According to an embodiment, the predictive encoder 3330 can construct multiple first reference image lists for an image sequence including the current image. The predictive encoder 3330 selects at least one first reference image list from the multiple first reference image lists for the image sequence. The predictive encoder 3330 can select a first reference image list for the current strip from the multiple first reference image lists. Furthermore, the predictive encoder 3330 obtains a second reference image list modified and improved from the selected first reference image list.

[0398] A second reference image list can be obtained by replacing at least one reference image in the first reference image list with another reference image, by changing the order of one or more reference images, or by adding a new reference image to the first reference image list.

[0399] When the second reference image list is obtained, the prediction encoder 3330 can encode the coding units included in the strip by inter-frame prediction using at least one reference image from the reference images included in the second reference image list.

[0400] The predictive encoder 3330 can predictively encode coding units included in the next strip by using a first reference image list (excluding the first reference image list selected for the current strip) and a second reference image list from a plurality of first reference image lists for the image sequence. In other words, the second reference image list obtained for the current strip can also be used for the next strip.

[0401] The following describes a method for obtaining a second list of reference images modified and improved from the first list of reference images.

[0402] According to an embodiment, the predictive encoder 3330 can obtain a second reference image list by changing at least one reference image in the first reference image list to another reference image.

[0403] According to an embodiment, the predictive encoder 3330 may replace only reference images of a specific type (e.g., long-term type reference images) in the reference images included in the first reference image list with another long-term type reference image. That is, short-term type reference images in the reference images included in the first reference image list may be kept unchanged in the second reference image list, and only long-term type reference images may be replaced by another long-term type reference image.

[0404] According to an embodiment, regardless of the type of the reference images included in the first reference image list, at least one reference image in the first reference image list can be replaced by another reference image. According to an embodiment, new reference images can be added to the second reference image list in the order in which they are removed from the first reference image list. That is, when a long-term type reference image assigned index 1 is removed from the first reference image list, index 1 can also be assigned to a new reference image.

[0405] According to an embodiment, the predictive encoder 3330 can obtain a second reference image list by excluding reference images of a specific type from reference images in a first reference image list selected for the current strip in a plurality of first reference image lists for an image sequence.

[0406] According to an embodiment, the predictive encoder 3330 can obtain a second reference image list by changing the order of one or more reference images in a first reference image list for the current strip selection from a plurality of first reference image lists for an image sequence.

[0407] According to an embodiment, the predictive encoder 3330 can obtain a second reference image list by using a first reference image list that includes only short-term type reference images and a first reference image list that includes only long-term type reference images. For example, the predictive encoder 3330 can include short-term type reference images included in the first reference image list and long-term type reference images included in the first reference image list in the second reference image list.

[0408] Furthermore, according to an embodiment, when the first reference image list includes only short-term type reference images, the predictive encoder 3330 can obtain a second reference image list that includes new long-term type reference images and short-term type reference images included in the first reference image list. Conversely, when the first reference image list includes only long-term type reference images, the predictive encoder 3330 can obtain a second reference image list that includes new short-term type reference images and long-term type reference images included in the first reference image list.

[0409] Once the second reference image list is constructed, the predictive encoder 3330 can perform inter-frame prediction on the coding units based on the reference images included in the second reference image list. As a result of the inter-frame prediction, prediction samples corresponding to the coding units can be obtained.

[0410] The reconstructor 3350 obtains reconstructed samples of the coding unit by using predicted samples. The reconstructed image including the reconstructed samples can be stored in the DPB as a reference image for subsequent images.

[0411] According to an embodiment, the reconstructor 3350 can perform luminance mapping on the predicted samples of the coding unit before obtaining the reconstructed samples. The reconstructor 3350 can obtain parameters for luminance mapping from multiple sets of post-processing parameters.

[0412] Each of the plurality of post-processing parameter sets may include parameters for luminance mapping or adaptive loop filtering, as described below. In other words, some post-processing parameter sets include parameters for luminance mapping, and others include parameters for adaptive loop filtering. For example, at least one parameter set may include parameters for luminance mapping, and others may include parameters for adaptive loop filtering. The reconstructor 3350 may generate a plurality of post-processing parameter sets including parameters for luminance mapping or parameters for adaptive loop filtering. As described above, the plurality of post-processing parameter sets can be transmitted as signals to the image decoding device 2000 via a bitstream.

[0413] The reconstructor 3350 can obtain parameters from a set of post-processing parameters selected from the plurality of post-processing parameter sets, and can change the brightness value of the predicted sample points by using the obtained parameters.

[0414] According to an embodiment, the reconstructor 3350 can correct the parameters of a set of post-processing parameters selected from a plurality of post-processing parameter sets, and can change the brightness value of the predicted sample points by means of the corrected parameters.

[0415] Furthermore, according to an embodiment, the reconstructor 3350 can construct a new parameter set by combining one or more parameters included in at least two of the plurality of post-processing parameter sets, and can change the brightness value of the predicted sample points by using the parameters of the newly constructed parameter set.

[0416] The reconstructor 3350 obtains reconstructed samples corresponding to the current coding unit by using prediction samples generated as a result of prediction decoding or prediction samples that have undergone luminance mapping. Once the reconstructed samples are obtained, the reconstructor 3350 may apply adaptive loop filtering to the reconstructed samples.

[0417] As described above, some post-processing parameter sets in the post-processing parameter set may include parameters for brightness mapping, and other post-processing parameter sets may include parameters for adaptive loop filtering (e.g., filter coefficients). The reconstructor 3350 can filter the reconstructed samples using parameters obtained from at least one of the plurality of post-processing parameter sets.

[0418] According to an embodiment, the reconstructor 3350 can correct parameters obtained from any one of the plurality of post-processing parameter sets, and can filter the reconstructed samples by using the corrected parameters.

[0419] Furthermore, according to an embodiment, the reconstructor 3350 can construct a new parameter set by combining one or more parameters included in at least two of the plurality of post-processing parameter sets, and can filter the reconstructed samples by using the parameters of the newly constructed parameter set.

[0420] Furthermore, according to an embodiment, the reconstructor 3350 can filter the luminance values ​​of the reconstructed sample points by using any one of the plurality of post-processing parameter sets, and can filter the chrominance values ​​of the reconstructed sample points by using another set of post-processing parameters.

[0421] When the predictive encoder 3330 performs inter-frame prediction on the coding units included in the current strip, the predictive encoder 3330 can regard the boundary of the current strip as the picture boundary.

[0422] When the predictive encoder 3330 derives the motion vector of the current coding unit, the predictive encoder 3330 can limit the search range to the boundary of the region in the reference image that is in the same position as the current strip.

[0423] According to an embodiment, the predictive encoder 3330 can treat the boundary of a strip as the boundary of a frame in BIO processing mode, and can perform predictive encoding on the current encoding unit.

[0424] Generator 3370 generates a bitstream that includes information for encoding the image. As described above, the bitstream may include SPS, PPS, GH, BPS, and at least one set of post-processing parameters.

[0425] The above description of the image decoding device 2000 includes information contained in the bitstream generated by generator 3370, therefore its detailed description is omitted.

[0426] The generator 3370 can entropy encode the binary values ​​corresponding to syntax elements based on CABAC. Here, the generator 3370 can selectively apply WPP by considering the number of parallel blocks included in the strip. When the strip includes only one parallel block, the generator 3370 can configure a probabilistic model for the CTUs included in the parallel block based on WPP, and when the strip includes multiple parallel blocks, the generator 3370 can choose not to apply WPP to the CTUs included in the parallel blocks.

[0427] Figure 34 This is an illustration used to describe an image encoding method according to an embodiment.

[0428] In operation S3410, the image encoding device 3300 constructs a plurality of first reference image lists for an image sequence including the current image. The plurality of first reference image lists may include at least one of short-term type reference images and long-term type reference images.

[0429] In operation S3420, the image encoding device 3300 configures blocks and block groups comprising at least one block in the current image. The block can be a parallel block, and the block group can be a stripe.

[0430] According to an embodiment, the image encoding device 3300 can divide the current image into multiple CTUs and can configure parallel blocks including at least one CTU and stripes including at least one parallel block in the current image.

[0431] According to an embodiment, the image encoding device 3300 can divide the current image into multiple parallel blocks, and each parallel block can be divided into one or more CTUs. Furthermore, the image encoding device 3300 can configure stripes comprising at least one parallel block in the current image.

[0432] According to an embodiment, the image encoding device 3300 can divide the current image into one or more stripes, and each stripe can be divided into one or more parallel blocks. Furthermore, the image encoding device 3300 can divide each parallel block into one or more CTUs.

[0433] In operation S3430, the image encoding device 3300 can select a first reference image list for the current block group including the current block in the current image from a plurality of first reference image lists, and can obtain a second reference image list based on the selected first reference image list.

[0434] In operation S3440, the image encoding device 3300 performs predictive encoding on the lower-level block included in the current block based on the reference image included in the second reference image list.

[0435] When a predicted sample corresponding to the lower block is obtained as a result of predictive coding, the image coding device 3300 can change the brightness value of the predicted sample by using parameters included in at least one of a plurality of post-processing parameter sets.

[0436] According to an embodiment, the image coding device 3300 can obtain reconstructed samples based on predicted samples obtained as a result of predictive coding or predicted samples that have undergone brightness mapping, and can perform adaptive loop filtering on the reconstructed samples. To this end, the image coding device 3300 can filter the reconstructed samples using parameters included in at least one of the plurality of post-processing parameter sets.

[0437] Furthermore, the embodiments described above can be written as computer executable programs that can be stored in a medium.

[0438] The medium can sustainably store computer-executable programs or temporarily store computer-executable programs or instructions for execution or download. Furthermore, the medium can be any of a variety of recording or storage media that combine single or multiple hardware components, and is not limited to media directly connected to a computer system but can be distributed across a network. Examples of media include magnetic media (such as hard disks, floppy disks, and magnetic tapes) configured to store program instructions, optical recording media (such as CD-ROMs and DVDs), magneto-optical media (such as floppy disks), and ROM, RAM, and flash memory. Other examples of media include recording and storage media managed by application stores that distribute applications or by websites, servers, etc., that provide or distribute various other types of software.

[0439] While one or more embodiments of this disclosure have been described with reference to the accompanying drawings, those skilled in the art will understand that various changes in form and detail may be made in this disclosure without departing from the spirit and scope defined by the claims.

Claims

1. An image decoding method, comprising: Obtain the address information for the current slice included in the current frame from the slice header of the bitstream, wherein the slice header includes information for the current slice; Based on the address information, obtain the current stripe including the current sample point; Use the reference screen list to obtain reference screens for the current screen; Based on the reference image and the motion vector for the current sample point, the predicted sample point for the current sample point is obtained. Information is obtained indicating a set of post-processing parameters from a plurality of post-processing parameter sets, the set of post-processing parameters being applied to the brightness mapping of the predicted sample points; The luminance mapping parameters to be applied to the luminance values ​​of the predicted samples are obtained from the post-processing parameter set of the bitstream, wherein the post-processing parameter set of the bitstream is different from the strip header of the bitstream; Modified prediction samples are obtained by changing the brightness value of the prediction sample based on the brightness mapping parameters obtained from the post-processing parameter set of the bitstream. Obtain the residual sample points included in the current sample point in the current frame; and Based on the modified predicted samples and the residual samples, reconstructed samples are obtained that include the current samples in the current frame.

2. An image decoding device, comprising: The processor is configured as follows: Obtain the address information for the current slice included in the current frame from the slice header of the bitstream, wherein the slice header includes information for the current slice; Based on the address information, obtain the current stripe including the current sample point; Use the reference screen list to obtain reference screens for the current screen; Based on the reference image and the motion vector for the current sample point, a predicted sample point included in the current image is obtained. Information is obtained indicating a set of post-processing parameters from a plurality of post-processing parameter sets, the set of post-processing parameters being applied to the brightness mapping of the predicted sample points; The luminance mapping parameters to be applied to the luminance values ​​of the predicted samples are obtained from the post-processing parameter set of the bitstream, which is different from the strip header of the bitstream; Modified prediction samples are obtained by changing the brightness value of the prediction sample based on the brightness mapping parameters obtained from the post-processing parameter set of the bitstream. Obtain the residual sample points included in the current sample point in the current frame; and Based on the modified predicted samples and the residual samples, reconstructed samples are obtained that include the current samples in the current frame.

3. An image encoding method, comprising: Obtain the current stripe including the current sample point; Use the reference screen list to obtain reference screens for the current screen; Based on the reference image and the motion vector for the current sample point, the predicted sample point for the current sample point is obtained; One of a plurality of post-processing parameter sets is selected and applied to the brightness mapping of the predicted sample points, wherein the post-processing parameter set is different from the strip header; Identify the brightness mapping parameters that will be applied to the brightness values ​​of the predicted sample points; Modified prediction samples are obtained by changing the prediction samples based on one or more brightness mapping parameters obtained from the post-processing parameter set; Based on the modified prediction sample points, obtain residual sample points that include the current sample points in the current frame; The address information of the current strip included in the current frame, the brightness mapping parameters, and the information about the residual samples are encoded; and A bitstream is generated that includes the encoded address information, the encoded luminance mapping parameters, and the encoded information, wherein the encoded address information is included in the slice header of the bitstream, wherein the slice header includes information for the current slice, and wherein the luminance mapping parameters are included in the post-processing parameter set of the bitstream.

4. A method for transmitting a bit stream, comprising: Obtain the current stripe including the current sample point; Use the reference screen list to obtain reference screens for the current screen; Based on the reference image and the motion vector for the current sample point, the predicted sample point for the current sample point is obtained; One of a plurality of post-processing parameter sets is selected and applied to the brightness mapping of the predicted sample points, wherein the post-processing parameter set is different from the strip header; Identify the brightness mapping parameters that will be applied to the brightness values ​​of the predicted sample points; Modified prediction samples are obtained by changing the prediction samples based on one or more brightness mapping parameters obtained from the post-processing parameter set; Based on the modified prediction sample points, obtain residual sample points that include the current sample points in the current frame; The address information of the current strip included in the current image, the brightness mapping parameters, and the information about the residual sample points are encoded. Generate a bitstream comprising encoded address information, encoded luminance mapping parameters, and encoded information, wherein the encoded address information is included in a slice header of the bitstream, wherein the slice header includes information for the current slice, and wherein the luminance mapping parameters are included in the post-processing parameter set of the bitstream; and The bitstream is sent from the image encoding device to the image decoding device.

Images