Encoding methods and apparatus and decoding methods and apparatus
By determining the affine parameter set of the current block based on the decoding state of adjacent blocks during video encoding and decoding, the problem of low video encoding efficiency in the prior art is solved, and more efficient video data transmission and quality preservation are achieved.
Patent Information
- Application Number
- CN202311172854.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2017-09-28
- Filing Date
- 2018-09-28
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2038-09-28
AI Technical Summary
Existing video coding methods struggle to effectively reduce image quality degradation while improving compression rates, and inter-frame prediction methods involve a large amount of encoded information, resulting in low video data transmission efficiency.
The affine parameter set of the current block is determined based on whether the adjacent blocks of the current block have been decoded, thereby improving the coding efficiency of the affine mode. Video decoding and encoding are performed using video decoding and encoding devices, and the processor is used to determine the affine parameter set and reconstruct the current block.
It improves the efficiency of video encoding and decoding, reduces the amount of encoded information, and enhances the quality and efficiency of video data transmission.
Smart Images

Figure CN116996669B_ABST
Abstract
Description
[0001] This application is a divisional application of the invention patent application filed on September 28, 2018, with application number "201880063174.9" and titled "Encoding Method and Apparatus and Decoding Method and Apparatus". Technical Field
[0002] This disclosure relates to a video encoding method and a decoding method, and more specifically, to an inter-frame prediction method for an image encoding / decoding method and device. Background Technology
[0003] High-quality video requires a large amount of data during encoding. However, the bandwidth allowed to transmit video data is limited, and therefore the data rate applied during video data transmission may be restricted. Therefore, to efficiently transmit video data, encoding and decoding methods that minimize image quality degradation while increasing compression rates are needed.
[0004] Video data can be compressed by removing spatial and temporal redundancy between pixels. Since adjacent pixels typically share common characteristics, encoded information is transmitted as pixel data units to eliminate redundancy between adjacent pixels.
[0005] Instead of directly transmitting the pixel values of the pixels included in the data unit, the method required to obtain the pixel values is transmitted. For each data unit, a prediction method is determined to predict pixel values similar to the original values, and encoded information about this prediction method is sent from the encoder to the decoder. Furthermore, because the predicted values are not exactly the same as the original values, residual data about the difference between the original and predicted values is sent from the encoder to the decoder.
[0006] As predictions become more accurate, the amount of encoded information required to specify the prediction method increases, but the size of the residual data decreases. Therefore, the prediction method is determined by considering both the encoded information and the size of the residual data. Specifically, data units divided from the frame have various sizes, and as the size of the data unit increases, the prediction accuracy is very likely to decrease, but the encoded information decreases. Therefore, the size of the blocks is determined based on the characteristics of the frame.
[0007] Furthermore, the prediction methods include intra-frame prediction and inter-frame prediction. Intra-frame prediction is a method of predicting pixels in a block based on neighboring pixels. Inter-frame prediction is a method of predicting pixels by referring to pixels in another frame that includes a block. Therefore, spatial redundancy is removed via intra-frame prediction and temporal redundancy is removed via inter-frame prediction.
[0008] As the number of prediction methods increases, the amount of encoded information used to indicate these methods also increases. Therefore, the size of the encoded information can be reduced by predicting the encoded information applied to another block from one block.
[0009] Because video data loss is permissible to a degree imperceptible to human vision, the amount of residual data can be reduced by lossy compression of the residual data based on transform and quantization processing. Summary of the Invention
[0010] Technical issues
[0011] A video encoding method is provided to determine the motion vector resolution of a current block based on its encoding order and whether adjacent blocks have been decoded. Furthermore, a video decoding method is provided to determine the motion vector resolution of a current block based on its encoding order and whether adjacent blocks have been decoded. Additionally, a computer-readable recording medium is provided containing programs for executing the video encoding and video decoding methods according to embodiments of this disclosure on a computer.
[0012] Solution
[0013] A video decoding method is provided, the video decoding method comprising: obtaining affine parameter group candidates for the current block based on whether neighboring blocks of the current block have been decoded; determining the affine parameter group of the current block from the affine parameter group candidates according to the affine parameter information of the current block; and reconstructing the current block based on one or more affine parameters included in the affine parameter group.
[0014] A video decoding device is provided, the video decoding device comprising: a processor configured to obtain affine parameter group candidates for a current block based on whether neighboring blocks of the current block have been decoded, determine an affine parameter group for the current block from the affine parameter group candidates based on affine parameter information of the current block, and reconstruct the current block based on one or more affine parameters included in the affine parameter group.
[0015] A video encoding method is provided, the video encoding method comprising: determining candidate affine parameter sets for the current block based on whether neighboring blocks of the current block have been decoded; determining the optimal affine parameter set for predicting the current block; and determining affine parameter information indicating the optimal affine parameter set of the current block.
[0016] A video encoding apparatus is provided, the video encoding apparatus comprising: a processor configured to determine affine parameter group candidates for a current block based on whether neighboring blocks of the current block have been decoded, determine an optimal affine parameter group for predicting the current block, and determine affine parameter information indicating the optimal affine parameter group of the current block.
[0017] A non-transitory computer-readable recording medium is provided, which records programs for performing the video encoding method and the video decoding method.
[0018] The technical problems solved by this disclosure are not limited to the above-described technical features, and other technical problems can be derived from the following embodiments.
[0019] Beneficial effects
[0020] The coding efficiency of affine mode can be improved by considering whether the adjacent blocks of the current block have been decoded to determine the affine parameter set. Attached Figure Description
[0021] Figure 1 This is a schematic block diagram of an image decoding device according to an embodiment.
[0022] Figure 2 This is a flowchart of an image decoding method according to an embodiment.
[0023] Figure 3 The illustration shows a process performed by an image decoding device according to an embodiment to determine at least one coding unit by dividing the current coding unit.
[0024] Figure 4 The illustration shows a process performed by an image decoding device according to an embodiment to determine at least one coding unit by dividing a non-square coding unit.
[0025] Figure 5 The illustration shows a process performed by an image decoding device according to an embodiment, which divides encoding units based on at least one of block shape information and segmentation shape pattern information.
[0026] Figure 6 A method for determining a predetermined coding unit from an odd number of coding units, performed by an image decoding device according to an embodiment, is shown.
[0027] Figure 7 This illustrates the order in which the image decoding device processes the plurality of coding units when it determines the plurality of coding units by dividing the current coding unit, according to an embodiment.
[0028] Figure 8 The illustration shows a process performed by an image decoding device according to an embodiment, in which the current coding unit is determined to be divided into an odd number of coding units when the coding units cannot be processed in a predetermined order.
[0029] Figure 9 The illustration shows a process performed by an image decoding device according to an embodiment to determine at least one coding unit by dividing a first coding unit.
[0030] Figure 10The embodiment shows that when a second coding unit with a non-square shape is determined during the division of a first coding unit by an image decoding device, the shape into which the second coding unit can be divided is limited, provided that a predetermined condition is met.
[0031] Figure 11 The illustration shows a process performed by an image decoding device according to an embodiment, whereby the division shape pattern information cannot indicate that a square coding unit is divided into four square coding units, for dividing a square coding unit.
[0032] Figure 12 This illustrates that, according to an embodiment, the processing order among multiple coding units can be changed based on the process of dividing the coding units.
[0033] Figure 13 The illustration shows a process for determining the depth of a coding unit as the shape and size of the coding unit change when the coding unit is recursively divided such that multiple coding units are determined, according to an embodiment.
[0034] Figure 14 The diagram illustrates a depth that can be determined based on the shape and size of the coding unit, and a partial index (PID) used to distinguish the coding unit, according to an embodiment.
[0035] Figure 15 The illustration shows how multiple encoding units are determined based on multiple predetermined data units included in the image, according to an embodiment.
[0036] Figure 16 The diagram illustrates a processing block, according to an embodiment, used as a standard for determining the order of reference coding units included in a frame.
[0037] Figure 17 This illustrates a video decoding apparatus based on an embodiment relating to the division of the current block and the determination of the encoding order of the resulting lower-level blocks.
[0038] Figures 18A to 18C The basic coding order according to the embodiment is shown.
[0039] Figure 19A and Figure 19B The examples show the cases where the coding unit is encoded in the positive direction and the cases where the coding unit is encoded in the negative direction.
[0040] Figure 20 This shows a tree structure describing the encoding order of the largest coding unit and the coding units included in the largest coding unit.
[0041] Figure 21A and Figure 21B It is used to describe how to change the encoding order of three or more blocks arranged in a vertical or horizontal direction based on the encoding order flag.
[0042] Figure 22 It is a method for describing in detail the derivation of motion vectors applied to sample points of the current block in affine mode.
[0043] Figure 23 This is an embodiment used to describe the determination of affine parameter set candidates from several sample point positions of the current block.
[0044] Figure 24 This is an embodiment used to describe determining a candidate set of affine parameters from a neighboring block decoded according to an affine pattern.
[0045] Figure 25 This is an embodiment used to describe determining candidates for an affine parameter group from multiple adjacent blocks decoded according to an affine pattern when the left block of the current block is decoded and the right block is not decoded.
[0046] Figure 26 This is an embodiment used to describe determining the motion vector of the current block used in the affine mode from multiple adjacent blocks decoded according to the affine mode when the right block of the current block is decoded and the left block is not decoded.
[0047] Figure 27 This is an embodiment used to describe determining the motion vector of the current block, which is used in the affine mode, from multiple adjacent blocks decoded according to the affine mode when the right and left blocks of the current block have not been decoded.
[0048] Figure 28 It is a method used to describe how to determine the motion vector of the current block in affine mode.
[0049] Figure 29 This illustrates a video decoding method based on an embodiment related to dividing the current block and decoding the resulting lower-level blocks.
[0050] Figure 30 This illustrates a video encoding apparatus according to an embodiment relating to dividing the current block and encoding the resulting lower-level blocks.
[0051] Figure 31 This illustrates a video coding method based on an embodiment relating to dividing the current block and encoding the resulting lower-level blocks. Detailed Implementation
[0052] Best practice
[0053] A video decoding method is provided, the video decoding method comprising: obtaining affine parameter group candidates for the current block based on whether neighboring blocks of the current block have been decoded; determining the affine parameter group of the current block from the affine parameter group candidates according to the affine parameter information of the current block; and reconstructing the current block based on one or more affine parameters included in the affine parameter group.
[0054] Disclosed implementation methods
[0055] The advantages and features of one or more embodiments, as well as methods for implementing said one or more embodiments, can be more readily understood by referring to the examples and accompanying drawings. In this regard, embodiments of this disclosure may take different forms and should not be construed as limited to the description set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the embodiments of this disclosure to those skilled in the art.
[0056] The terminology used in this specification will be briefly defined, and the embodiments will be described in detail.
[0057] All terms used herein, including descriptive or technical terms, shall be interpreted as having meanings obvious to a person skilled in the art. However, these terms may have different meanings depending on the intent of a person skilled in the art, precedent, or the emergence of new technologies. Furthermore, some terms may be arbitrarily chosen by the applicant, and in such cases, the meaning of the chosen term will be described in detail in the specific description of this disclosure. Therefore, the terms used herein must be defined based on their meanings in conjunction with the description throughout the specification.
[0058] In the following description, unless the context clearly indicates otherwise, the singular form includes the plural form.
[0059] When a component "comprises" or "includes" an element, the component may also include other elements, without excluding other elements, unless there is a specific description to the contrary.
[0060] In the following description, terms such as “unit” refer to software or hardware components and “units” perform specific functions. However, “unit” is not limited to software or hardware. A “unit” may be formed in an addressable storage medium or may be formed to operate on one or more processors. Thus, for example, the term “unit” may refer to components such as software components, object-oriented software components, class components, and task components, and may include processes, functions, attributes, programs, subroutines, program code snippets, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, or variables. The functionality provided by components and “units” may be associated with a smaller number of components and “units”, or may be divided into additional components and “units.”
[0061] According to embodiments of this disclosure, a "unit" may include a processor and memory. The term "processor" should be broadly interpreted to include general-purpose processors, central processing units (CPUs), microprocessors, digital signal processors (DSPs), controllers, microcontrollers, state machines, etc. In some cases, "processor" may refer to application-specific integrated circuits (ASICs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), etc. The term "processor" may refer to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors combined with a DSP core, or any other combination of such configurations.
[0062] The term "memory" should be broadly interpreted to include any electronic component capable of storing electronic information. The term "memory" can refer to various types of processor-readable media, such as Random Access Memory (RAM), Read-Only Memory (ROM), Non-Volatile Random Access Memory (NVRAM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable PROM (EEPROM), Flash Memory, magnetic or optical data storage devices, registers, etc. Memory is said to be in an electronically communicating state with the processor when the processor can read information from and / or write information to the memory. Memory integrated into the processor is in an electronically communicating state with the processor.
[0063] In the following text, “image” can be a still image, such as a video, or a moving image, such as a motion picture (that is, the video itself).
[0064] In the following text, a "sample" refers to data assigned to a sampling location in an image (i.e., the data to be processed). For example, pixel values in the spatial domain and transform coefficients in the transform domain can be samples. A unit that includes at least one such sample can be defined as a block.
[0065] In the following, embodiments will be described in detail with reference to the accompanying drawings, enabling those skilled in the art to readily implement the embodiments. In the drawings, details irrelevant to the description are omitted to clearly illustrate this disclosure.
[0066] In the following text, reference will be made to Figures 1 to 16 This describes an image encoding apparatus and an image decoding apparatus, as well as an image encoding method and an image decoding method, according to embodiments. (Refer to...) Figures 3 to 16 A method for determining data units of an image according to an embodiment is described.
[0067] In the following text, reference will be made to Figure 1 and Figure 2Methods and apparatus for adaptively selecting a context model based on coding units of various shapes, according to embodiments of the present disclosure, are described.
[0068] Figure 1 This is a schematic block diagram of an image decoding device 100 according to an embodiment.
[0069] Image decoding device 100 may include receiver 110 and decoder 120. Receiver 110 and decoder 120 may include at least one processor. In addition, receiver 110 and decoder 120 may include memory storing instructions to be executed by at least one processor.
[0070] Receiver 110 can receive a bitstream. The bitstream includes information about an image encoded by the image encoding device described later. Furthermore, the bitstream can be transmitted from the image encoding device. The image encoding device and image decoding device 100 can be connected via wired or wireless means, and receiver 110 can receive the bitstream via wired or wireless means. Receiver 110 can receive the bitstream from a storage medium (such as an optical medium or a hard disk). Decoder 120 can reconstruct the image based on information obtained from the received bitstream. Decoder 120 can obtain syntax elements for reconstructing the image from the bitstream. Decoder 120 can reconstruct the image based on the syntax elements.
[0071] Reference Figure 2 The operation of the image decoding device 100 is described in detail.
[0072] Figure 2 This is a flowchart of an image decoding method according to an embodiment.
[0073] According to an embodiment of this disclosure, receiver 110 receives a bit stream.
[0074] Image decoding device 100 obtains a binary bit string corresponding to the partition shape pattern of the coding unit from the bit stream (operation 210). Image decoding device 100 determines the partitioning rule of the coding unit (operation 220). Furthermore, based on at least one of the binary bit string corresponding to the partition shape pattern and the partitioning rule, image decoding device 100 divides the coding unit into multiple coding units (operation 230). According to the width-to-height ratio of the coding unit, image decoding device 100 can determine a first permissible range of the size of the coding unit in order to determine the partitioning rule. According to the partition shape pattern of the coding unit, image decoding device 100 can determine a second permissible range of the size of the coding unit in order to determine the partitioning rule.
[0075] The division of coding units will be described in detail below based on embodiments of the present disclosure.
[0076] First, a frame can be divided into one or more stripes. A strip can be a sequence of one or more maximum coding units (coding tree units (CTUs)). Conceptually, there exists a maximum coding block (coding tree block (CTB) in contrast to a maximum coding unit (CTU).
[0077] A maximum coding block (CTB) represents an N×N block containing N×N samples (N is an integer). Each color component can be divided into one or more maximum coding blocks.
[0078] When the image has three sample arrays (sample arrays for the Y, Cr, and Cb components), the Maximum Coding Unit (CTU) includes a maximum coding block for the luma samples, two corresponding maximum coding blocks for the chroma samples, and a syntax structure for encoding the luma and chroma samples. When the image is monochrome, the CTU includes a maximum coding block for the monochrome samples and a syntax structure for encoding the monochrome samples. When the image is encoded in color planes separated according to the color components, the CTU includes a syntax structure for encoding the image and its samples.
[0079] A maximum code block (CTB) can be divided into an M×N code block consisting of M×N samples (M and N are integers).
[0080] When the image has sample arrays for the Y, Cr, and Cb components, the encoding unit (CU) includes an encoding block for the luminance samples, two corresponding encoding blocks for the chrominance samples, and a syntax structure for encoding the luminance and chrominance samples. When the image is monochrome, the encoding unit includes an encoding block for the monochrome samples and a syntax structure for encoding the monochrome samples. When the image is encoded in a color plane separated according to the color components, the encoding unit includes a syntax structure for encoding the image and its samples.
[0081] As described above, the maximum coding block and the maximum coding unit are conceptually distinguished from each other, and the coding block and the coding unit are conceptually distinguished from each other. That is, the (maximum) coding unit represents a data structure including the (maximum) coding block containing the corresponding samples and the syntax structure corresponding to the (maximum) coding block. However, since those skilled in the art understand that the (maximum) coding unit or the (maximum) coding block represents a block of a predetermined size including a predetermined number of samples, the references to maximum coding block and maximum coding unit or coding block and coding unit in the following specification need not be distinguished unless otherwise described.
[0082] An image can be divided into maximum coding units (CTUs). The size of each CTU can be determined based on information obtained from the bitstream. The shape of each CTU can be a square of the same size. However, the embodiments are not limited to this.
[0083] For example, information about the maximum size of a luminance-coded block can be obtained from the bitstream. For example, the maximum size of a luminance-coded block, indicated by the information about the maximum size of the luminance-coded block, can be one of 4×4, 8×8, 16×16, 32×32, 64×64, 128×128, and 256×256.
[0084] For example, information about the luma block size difference and the maximum size of the luma coding block that can be divided into two can be obtained from the bitstream. The information about the luma block size difference represents the size difference between the maximum luma coding unit and the maximum luma coding block that can be divided into two. Therefore, when the information about the maximum size of the luma coding block that can be divided into two and the information about the luma block size difference obtained from the bitstream are combined, the size of the maximum luma coding unit can be determined. The size of the maximum chroma coding unit can be determined using the size of the maximum luma coding unit. For example, when the Y:Cb:Cr ratio is 4:2:0 according to the color format, the size of the chroma block can be half the size of the luma block, and the size of the maximum chroma coding unit can be half the size of the maximum luma coding unit.
[0085] According to the embodiment, since information about the maximum size of a divisible luminance coding block is obtained from the bitstream, the maximum size of the divisible luminance coding block can be determined variably. Conversely, the maximum size of a tridivisible luminance coding block can be fixed. For example, the maximum size of a tridivisible luminance coding block in an I-strip can be 32×32, and the maximum size of a tridivisible luminance coding block in a P-strip or B-strip can be 64×64.
[0086] Furthermore, the largest coding unit can be hierarchically divided into coding units based on the partition shape pattern information obtained from the bit stream. At least one of the following information obtained from the bit stream—information indicating whether a quad partition is performed, information indicating whether a multi-partition is performed, partition direction information, and partition type information—can be used as partition shape pattern information.
[0087] For example, information indicating whether a quad partitioning is performed can indicate whether the current coding unit has been quad partitioned (QUAD_SPLIT).
[0088] When the current coding unit is not quad-partitioned, the information indicating whether to perform multi-partitioning can indicate whether the current coding unit will not be partitioned again (NO_SPLIT) or will be partitioned into two or three parts.
[0089] When the current coding unit is divided into two or three parts, the division direction information indicates that the current coding unit is divided along one of the horizontal and vertical directions.
[0090] When the current coding unit is divided along the horizontal or vertical direction, the division type information indicates whether the current coding unit is divided into two or three parts.
[0091] The partitioning mode of the current coding unit can be determined based on the partitioning direction and partitioning type information. The partitioning mode when the current coding unit is partitioned horizontally in two directions can be determined as a two-horizontal partitioning mode (SPLIT_BT_HOR), the partitioning mode when the current coding unit is partitioned horizontally in three directions can be determined as a three-horizontal partitioning mode (SPLIT_TT_HOR), the partitioning mode when the current coding unit is partitioned vertically in two directions can be determined as a two-vertical partitioning mode (SPLIT_BT_VER), and the partitioning mode when the current coding unit is partitioned vertically in three directions can be determined as a three-vertical partitioning mode (SPLIT_TT_VER).
[0092] Image decoding device 100 can obtain partitioning shape pattern information from a binary bit string in a bit stream. The bit stream received by image decoding device 100 may be in the form of fixed-length binary code, unary code, truncated unary code, predetermined binary code, etc. The binary bit string is information about binary numbers. The binary bit string may include at least one bit. Image decoding device 100 can obtain partitioning shape pattern information corresponding to the binary bit string based on partitioning rules. Image decoding device 100 can determine, based on a binary bit string, whether to perform four partitions on the coding unit, whether not to partition the coding unit, the partitioning direction, and the partitioning type.
[0093] A coding unit can be smaller than or the same as the maximum coding unit. For example, since the maximum coding unit is the coding unit with the largest size, it is one of the coding units. When the partitioning shape pattern information regarding the maximum coding unit indicates that partitioning is not performed, the coding units determined within the maximum coding unit have the same size as the maximum coding unit. When the partitioning shape pattern information regarding the maximum coding unit indicates that partitioning is performed, the maximum coding unit can be partitioned into coding units. Furthermore, when the partitioning shape pattern information regarding the coding units indicates that partitioning is performed, the coding unit can be partitioned into smaller coding units. However, the partitioning of an image is not limited to this, and the maximum coding unit may not be distinguished from the coding units. (Refer to...) Figures 3 to 16 Describe in detail the division of coding units.
[0094] Furthermore, one or more prediction blocks for prediction can be determined from the coding unit. The prediction blocks may be the same as or smaller than the coding unit. Additionally, one or more transform blocks for transform can be determined from the coding unit. The transform blocks may be the same as or smaller than the coding unit.
[0095] The shapes and sizes of the transform blocks and the prediction blocks can be independent of each other.
[0096] In another embodiment, prediction can be performed by using the coding unit as a prediction unit. Furthermore, transformation can be performed by using the coding unit as a transform block.
[0097] The current block and peripheral blocks in this disclosure can refer to one of the maximum coding unit, coding unit, prediction block, and transform block. Furthermore, the current block of the current coding unit is the block currently being decoded or encoded, or the block currently being partitioned. Peripheral blocks can be blocks reconstructed prior to the current block. Peripheral blocks can be spatially or temporally adjacent to the current block. Peripheral blocks can be located at one of the following positions: lower left, left side, upper left, above, upper right, right, or lower right of the current block.
[0098] Figure 3 The illustration shows a process performed by an image decoding device 100 according to an embodiment, which involves dividing the current coding unit to determine at least one coding unit.
[0099] The block shape may include 4N×4N, 4N×2N, 2N×4N, 4N×N, N×4N, 32N×N, N×32N, 16N×N, N×16N, 8N×N, or N×8N. Here, N can be a positive integer. Block shape information is information indicating at least one of the following: the shape, orientation, width-to-height ratio, or size of the coding unit.
[0100] The shape of the encoding unit can be square or non-square. When the width and height of the encoding unit are the same (i.e., when the block shape of the encoding unit is 4N×4N), the image decoding device 100 can determine the block shape information of the encoding unit as square. The image decoding device 100 can also determine the shape of the encoding unit as non-square.
[0101] When the width and height of the encoding unit are different from each other (i.e., when the block shape of the encoding unit is 4N×2N, 2N×4N, 4N×N, N×4N, 32N×N, N×32N, 16N×N, N×16N, 8N×N, or N×8N), the image decoding device 100 can determine the block shape information of the encoding unit as a non-square shape. When the shape of the encoding unit is non-square, the image decoding device 100 can determine the width-to-height ratio in the block shape information of the encoding unit as at least one of 1:2, 2:1, 1:4, 4:1, 1:8, 8:1, 1:16, 16:1, 1:32, and 32:1. Furthermore, the image decoding device 100 can determine whether the encoding unit is horizontal or vertical based on its width and height. Additionally, the image decoding device 100 can determine the size of the encoding unit based on at least one of its width, height, or area.
[0102] According to an embodiment, the image decoding device 100 can determine the shape of the coding unit by using block shape information, and can determine the coding unit partitioning method by using partition shape pattern information. That is, the coding unit partitioning method indicated by the partition shape pattern information can be determined based on the block shape indicated by the block shape information used by the image decoding device 100.
[0103] Image decoding device 100 can obtain partition shape pattern information from the bitstream. However, embodiments are not limited to this, and image decoding device 100 and image encoding device can determine pre-agreed partition shape pattern information based on block shape information. Image decoding device 100 can determine pre-agreed partition shape pattern information for the largest or smallest coding unit. For example, image decoding device 100 can determine the partition shape pattern information for the largest coding unit as a four-part partition. Furthermore, image decoding device 100 can determine the partition shape pattern information for the smallest coding unit as "no partitioning". Specifically, image decoding device 100 can determine the size of the largest coding unit as 256×256. Image decoding device 100 can determine the pre-agreed partition shape pattern information as a four-part partition. A four-part partition is a partition shape pattern that divides the width and height of the coding unit into two equal parts. Image decoding device 100 can obtain a 128×128 coding unit from the 256×256-sized largest coding unit based on the partition shape pattern information. Furthermore, image decoding device 100 can determine the size of the smallest coding unit as 4×4. Image decoding device 100 can obtain partitioning shape pattern information indicating "do not perform partitioning" for the smallest coding unit.
[0104] According to an embodiment, the image decoding device 100 can use block shape information indicating that the current coding unit has a square shape. For example, the image decoding device 100 can determine whether to divide the square coding unit, whether to divide the square coding unit vertically, whether to divide the square coding unit horizontally, or whether to divide the square coding unit into four coding units based on the division shape pattern information. (Refer to...) Figure 3 When the block shape information of the current coding unit 300 indicates a square shape, the decoder 120 may determine, based on the partition shape pattern information indicating that partitioning is not performed, not to partition the coding unit 310a with the same size as the current coding unit 300, or may determine the coding units 310b, 310c, 310d, 310e or 310f partitioned based on the partition shape pattern information indicating a predetermined partitioning method.
[0105] Reference Figure 3 According to an embodiment, the image decoding device 100 can determine two coding units 310b obtained by dividing the current coding unit 300 in a vertical direction based on the division shape pattern information indicating that division is performed in a vertical direction. The image decoding device 100 can determine two coding units 310c obtained by dividing the current coding unit 300 in a horizontal direction based on the division shape pattern information indicating that division is performed in a horizontal direction. The image decoding device 100 can determine four coding units 310d obtained by dividing the current coding unit 300 in both vertical and horizontal directions based on the division shape pattern information indicating that division is performed in both vertical and horizontal directions. According to an embodiment, the image decoding device 100 can determine three coding units 310e obtained by dividing the current coding unit 300 in a vertical direction based on the division shape pattern information indicating that division is performed in a three-way direction. The image decoding device 100 can determine three coding units 310f obtained by dividing the current coding unit 300 in a horizontal direction based on the division shape pattern information indicating that division is performed in a three-way direction. However, the method for dividing square coding units is not limited to the method described above, and the division shape pattern information can indicate various methods. The predetermined division method for dividing square coding units will be described in detail below with respect to various embodiments.
[0106] Figure 4 The illustration shows a process performed by an image decoding device 100 according to an embodiment to determine at least one coding unit by dividing a non-square coding unit.
[0107] According to an embodiment, the image decoding device 100 can use block shape information indicating that the current coding unit has a non-square shape. Based on the partitioning shape pattern information, the image decoding device 100 can determine whether to partition the non-square current coding unit or to partition the non-square current coding unit using a predetermined partitioning method. (Refer to...) Figure 4 When the block shape information of the current encoding unit 400 or 450 indicates a non-square shape, the image decoding device 100 may determine, based on the partitioning shape pattern information indicating that partitioning is not performed, that encoding unit 410 or encoding unit 460 with the same size as the current encoding unit 400 or 450 will not be partitioned, or determine that the encoding units 420a and 420b, 430a to 430c, 470a and 470b, or 480a to 480c partitioned based on the partitioning shape pattern information indicating a predetermined partitioning method. The predetermined partitioning method for partitioning non-square encoding units will be described in detail below with respect to various embodiments.
[0108] According to an embodiment, the image decoding device 100 can determine the partitioning method of the coding units by using partitioning shape pattern information, and in this case, the partitioning shape pattern information can indicate the number of one or more coding units generated by partitioning the coding units. (Refer to...) Figure 4 When the partitioning shape pattern information indicates that the current coding unit 400 or 450 is divided into two coding units, the image decoding device 100 can determine the two coding units 420a and 420b or 470a and 470b included in the current coding unit 400 or 450 by partitioning the current coding unit 400 or 450 based on the partitioning shape pattern information.
[0109] According to an embodiment, when the image decoding device 100 divides a non-square current coding unit 400 or 450 based on division shape pattern information, the image decoding device 100 may consider the position of the long side of the non-square current coding unit 400 or 450 to divide the current coding unit. For example, the image decoding device 100 may determine multiple coding units by dividing the long side of the current coding unit 400 or 450 by considering the shape of the current coding unit 400 or 450.
[0110] According to an embodiment, when the partitioning shape pattern information indicates that the coding unit is divided (tripartitely divided) into an odd number of blocks, the image decoding device 100 can determine the odd number of coding units included in the current coding unit 400 or 450. For example, when the partitioning shape pattern information indicates that the current coding unit 400 or 450 is divided into three coding units, the image decoding device 100 can divide the current coding unit 400 or 450 into three coding units 430a, 430b and 430c or 480a, 480b and 480c.
[0111] According to an embodiment, the width-to-height ratio of the current encoding unit 400 or 450 can be 4:1 or 1:4. When the width-to-height ratio is 4:1, since the width is longer than the height, the block shape information can be horizontal. When the width-to-height ratio is 1:4, since the width is shorter than the height, the block shape information can be vertical. The image decoding device 100 can determine whether to divide the current encoding unit into an odd number of blocks based on the division shape pattern information. Furthermore, the image decoding device 100 can determine the division direction of the current encoding unit 400 or 450 based on the block shape information of the current encoding unit 400 or 450. For example, when the current encoding unit 400 is vertical, the image decoding device 100 can determine encoding units 430a to 430c by dividing the current encoding unit 400 horizontally. Similarly, when the current encoding unit 450 is horizontal, the image decoding device 100 can determine encoding units 480a to 480c by dividing the current encoding unit 450 vertically.
[0112] According to an embodiment, the image decoding device 100 may determine an odd number of coding units included in the current coding unit 400 or 450, and not all determined coding units may have the same size. For example, a predetermined coding unit 430b or coding unit 480b in the determined odd number of coding units 430a, 430b, and 430c, or 480a, 480b, and 480c, may have a different size than the other coding units 430a and 430c or 480a and 480c. That is, the coding units determined by dividing the current coding unit 400 or 450 may have multiple sizes, and in some cases, all coding units in the odd number of coding units 430a, 430b, and 430c, or 480a, 480b, and 480c, may have different sizes.
[0113] According to an embodiment, when the partitioning shape pattern information indicates that the coding unit is divided into an odd number of blocks, the image decoding device 100 can determine the odd number of coding units included in the current coding unit 400 or 450, and additionally, a predetermined restriction can be imposed on at least one of the odd number of coding units generated by partitioning the current coding unit 400 or 450. (Refer to...) Figure 4The image decoding device 100 may configure the decoding process for encoding unit 430b or encoding unit 480b to differ from the decoding processes for other encoding units 430a and 430c or 480a and 480c, wherein encoding unit 430b or encoding unit 480b is located at the center position of the three encoding units 430a, 430b and 430c or 480a, 480b and 480c generated when dividing the current encoding unit 400 or 450. For example, unlike other encoding units 430a and 430c or 480a and 480c, the image decoding device 100 may restrict the encoding unit 430b or encoding unit 480b at the center position from being divided or from being divided only a predetermined number of times.
[0114] Figure 5 The illustration shows a process performed by an image decoding device 100 according to an embodiment, which divides the encoding unit based on at least one of block shape information and division shape pattern information.
[0115] According to an embodiment, the image decoding device 100 can determine whether to divide the square first coding unit 500 into coding units or not to divide the square first coding unit 500 based on at least one of block shape information and partition shape pattern information. According to an embodiment, when the partition shape pattern information indicates that the first coding unit 500 should be divided along a horizontal direction, the image decoding device 100 can determine the second coding unit 510 by dividing the first coding unit 500 along a horizontal direction. The terms first coding unit, second coding unit, and third coding unit used in the embodiment are terms used to understand the relationship before and after the coding unit is divided. For example, the second coding unit can be determined by dividing the first coding unit, and the third coding unit can be determined by dividing the second coding unit. It will be understood that the structures of the first coding unit, second coding unit, and third coding unit follow the above description.
[0116] According to an embodiment, the image decoding device 100 can determine, based on the division shape pattern information, whether to divide the determined second coding unit 510 into coding units or not to divide the determined second coding unit 510. (Refer to...) Figure 5The image decoding device 100 can divide a non-square second coding unit 510, determined by dividing the first coding unit 500, into one or more third coding units 520a or 520b, 520c, and 520d based on the division shape pattern information, or it can choose not to divide the non-square second coding unit 510. The image decoding device 100 can obtain the division shape pattern information and can obtain multiple second coding units (e.g., 510) of various shapes by dividing the first coding unit 500 based on the obtained division shape pattern information. It can also divide the second coding unit 510 based on the division shape pattern information using the division method of the first coding unit 500. According to an embodiment, when the first coding unit 500 is divided into a second coding unit 510 based on the division shape pattern information of the first coding unit 500, the second coding unit 510 can also be divided into third coding units 520a or 520b, 520c, and 520d based on the division shape pattern information of the second coding unit 510. In other words, coding units can be recursively divided based on the partition shape pattern information of each coding unit. Therefore, square coding units can be determined by dividing non-square coding units, and non-square coding units can be determined by recursively dividing square coding units.
[0117] Reference Figure 5 The predetermined coding units (e.g., coding units located at the center or square coding units) in an odd number of third coding units 520b, 520c, and 520d, determined by dividing the non-square second coding unit 510, can be recursively divided. According to an embodiment, the square third coding unit 520c in the odd number of third coding units 520b, 520c, and 520d can be divided into multiple fourth coding units along the horizontal direction. The non-square fourth coding units 530b or 530d in the multiple fourth coding units 530a, 530b, 530c, and 530d can be further divided into multiple coding units. For example, the non-square fourth coding units 530b or 530d can be further divided into an odd number of coding units. Methods for recursively dividing coding units will be described below with respect to various embodiments.
[0118] According to an embodiment, the image decoding device 100 can divide each of the third coding units 520a or 520b, 520c, and 520d into coding units based on the division shape pattern information. Furthermore, the image decoding device 100 can determine, based on the division shape pattern information, not to divide the second coding unit 510. According to an embodiment, the image decoding device 100 can divide the non-square second coding unit 510 into an odd number of third coding units 520b, 520c, and 520d. The image decoding device 100 can impose predetermined restrictions on predetermined third coding units among the odd number of third coding units 520b, 520c, and 520d. For example, the image decoding device 100 can restrict the third coding unit 520c at the center position among the odd number of third coding units 520b, 520c, and 520d to not be divided again or to be divided a set number of times.
[0119] Reference Figure 5 The image decoding device 100 may restrict the third coding unit 520c at the center position among the odd number of third coding units 520b, 520c, and 520d included in the non-square second coding unit 510 to no longer be divided, to be divided by using a predetermined division method (e.g., divided into only four coding units or divided by using the division method of the second coding unit 510), or to be divided only a predetermined number of times (e.g., divided only n times (where n > 0)). However, the restriction on the third coding unit 520c at the center position is not limited to the above examples and may include various restrictions for decoding the third coding unit 520c at the center position differently from the other third coding units 520b and 520d.
[0120] According to an embodiment, the image decoding device 100 can obtain partitioning shape pattern information for partitioning the current coding unit from a predetermined position in the current coding unit.
[0121] Figure 6 This illustrates a method performed by an image decoding device 100 according to an embodiment to determine a predetermined coding unit from an odd number of coding units.
[0122] Reference Figure 6 The partitioning shape pattern information of the current encoding unit 600 or 650 can be obtained from a sample at a predetermined position among multiple sample points included in the current encoding unit 600 or 650 (e.g., sample point 640 or 690 at the center position). However, the predetermined position in the current encoding unit 600 where at least one partitioning shape pattern information can be obtained is not limited to... Figure 6The image decoding device 100 can obtain the division shape pattern information from the predetermined position and can determine whether to divide the current encoding unit into encoding units of various shapes and sizes or not to divide the current encoding unit.
[0123] According to an embodiment, when the current coding unit is divided into a predetermined number of coding units, the image decoding device 100 can select one coding unit from the coding units. Various methods described below with respect to various embodiments can be used to select one coding unit from a plurality of coding units.
[0124] According to an embodiment, the image decoding device 100 can divide the current encoding unit into multiple encoding units and determine the encoding unit at a predetermined position.
[0125] According to an embodiment, the image decoding device 100 can use information indicating the positions of an odd number of coding units to determine the coding unit at the center position among the odd number of coding units. (See also...) Figure 6 The image decoding device 100 can determine an odd number of coding units 620a, 620b, and 620c or an odd number of coding units 660a, 660b, and 660c by dividing the current coding unit 600 or the current coding unit 650. The image decoding device 100 can determine an intermediate coding unit 620b or an intermediate coding unit 660b by using information about the positions of the odd number of coding units 620a, 620b, and 620c or the odd number of coding units 660a, 660b, and 660c. For example, the image decoding device 100 can determine the coding unit 620b at the center position by determining the positions of coding units 620a, 620b, and 620c based on information indicating the positions of predetermined samples included in coding units 620a, 620b, and 620c. In detail, the image decoding device 100 can determine the position of the encoding units 620a, 620b and 620c based on the information indicating the position of the upper left sample points 630a, 630b and 630c of the encoding units 620a, 620b and 620c, and then determine the encoding unit 620b at the center position.
[0126] According to an embodiment, the information indicating the positions of the top-left samples 630a, 630b, and 630c included in encoding units 620a, 620b, and 620c respectively may include information about the positions or coordinates of the encoding units 620a, 620b, and 620c in the image. According to an embodiment, the information indicating the positions of the top-left samples 630a, 630b, and 630c included in encoding units 620a, 620b, and 620c respectively may include information indicating the width or height of the encoding units 620a, 620b, and 620c included in the current encoding unit 600, and the width or height may correspond to information indicating the difference between the coordinates of the encoding units 620a, 620b, and 620c in the image. That is, the image decoding device 100 can determine the encoding unit 620b at the center position by directly using information about the positions or coordinates of the encoding units 620a, 620b, and 620c in the image, or by using information about the width or height of the encoding units corresponding to the difference between the coordinates.
[0127] According to an embodiment, the information indicating the position of the upper left sample 630a of the upper encoding unit 620a may include coordinates (xa, ya), the information indicating the position of the upper left sample 630b of the middle encoding unit 620b may include coordinates (xb, yb), and the information indicating the position of the upper left sample 630c of the lower encoding unit 620c may include coordinates (xc, yc). The image decoding device 100 can determine the middle encoding unit 620b by using the coordinates of the upper left samples 630a, 630b, and 630c included in the encoding units 620a, 620b, and 620c, respectively. For example, when the coordinates of the upper left samples 630a, 630b, and 630c are sorted in ascending or descending order, the encoding unit 620b including the coordinates (xb, yb) of the sample 630b at the center position can be determined as the encoding unit at the center position among the encoding units 620a, 620b, and 620c determined by dividing the current encoding unit 600. However, the coordinates indicating the positions of the top-left samples 630a, 630b, and 630c may include coordinates indicating their absolute positions in the image, or coordinates (dxb, dyb) indicating the relative position of the top-left sample 630b of the intermediate encoding unit 620b relative to the top-left sample 630a of the upper encoding unit 620a, and coordinates (dxc, dyc) indicating the relative position of the top-left sample 630c of the lower encoding unit 620c relative to the top-left sample 630a of the upper encoding unit 620a. The method of determining the encoding unit at a predetermined position by using the coordinates of the samples included in the encoding unit as information indicating the position of the samples is not limited to the methods described above, and may include various arithmetic methods capable of using the coordinates of the samples.
[0128] According to an embodiment, the image decoding device 100 can divide the current encoding unit 600 into a plurality of encoding units 620a, 620b, and 620c, and can select one of the encoding units 620a, 620b, and 620c based on a predetermined criterion. For example, the image decoding device 100 can select encoding unit 620b from the encoding units 620a, 620b, and 620c that has a size different from the sizes of the other encoding units.
[0129] According to an embodiment, the image decoding device 100 can determine the width or height of each of the encoding units 620a, 620b, and 620c by using coordinates (xa, ya) indicating the position of the upper left sample point 630a of the upper encoding unit 620a, coordinates (xb, yb) indicating the position of the upper left sample point 630b of the middle encoding unit 620b, and coordinates (xc, yc) indicating the position of the upper left sample point 630c of the lower encoding unit 620c. The image decoding device 100 can determine the respective dimensions of the encoding units 620a, 620b, and 620c by using coordinates (xa, ya), (xb, yb), and (xc, yc) indicating the positions of the encoding units 620a, 620b, and 620c. According to an embodiment, the image decoding device 100 can determine the width of the upper encoding unit 620a as the width of the current encoding unit 600. The image decoding device 100 can determine the height of the upper encoding unit 620a as yb-ya. According to an embodiment, the image decoding device 100 can determine the width of the intermediate encoding unit 620b as the width of the current encoding unit 600. The image decoding device 100 can determine the height of the intermediate encoding unit 620b as yc-yb. According to an embodiment, the image decoding device 100 can determine the width or height of the lower encoding unit 620c by using the width or height of the current encoding unit 600 or the width or height of the upper encoding unit 620a and the intermediate encoding unit 620b. The image decoding device 100 can determine encoding units with dimensions different from the dimensions of other encoding units based on the determined widths and heights of the encoding units 620a to 620c. (Refer to...) Figure 6 The image decoding device 100 can determine an intermediate encoding unit 620b, which has a size different from that of the upper encoding unit 620a and the lower encoding unit 620c, as an encoding unit at a predetermined position. However, the method described above by the image decoding device 100 for determining an encoding unit with a size different from that of other encoding units corresponds only to the example of determining an encoding unit at a predetermined position by using the size of the encoding unit determined based on the coordinates of sample points. Therefore, various methods can be used to determine an encoding unit at a predetermined position by comparing the sizes of encoding units determined based on the coordinates of predetermined sample points.
[0130] The image decoding device 100 can determine the width or height of each of the coding units 660a, 660b, and 660c by using coordinates (xd, yd) indicating the position of the upper left sample 670a of the left coding unit 660a, coordinates (xe, ye) indicating the position of the upper left sample 670b of the middle coding unit 660b, and coordinates (xf, yf) indicating the position of the upper left sample 670c of the right coding unit 660c. The image decoding device 100 can also determine the individual dimensions of the coding units 660a, 660b, and 660c by using coordinates (xd, yd), (xe, ye), and (xf, yf) indicating the positions of the coding units 660a, 660b, and 660c.
[0131] According to an embodiment, the image decoding device 100 can determine the width of the left encoding unit 660a as xe-xd. The image decoding device 100 can determine the height of the left encoding unit 660a as the height of the current encoding unit 650. According to an embodiment, the image decoding device 100 can determine the width of the middle encoding unit 660b as xf-xe. The image decoding device 100 can determine the height of the middle encoding unit 660b as the height of the current encoding unit 650. According to an embodiment, the image decoding device 100 can determine the width or height of the right encoding unit 660c by using the width or height of the current encoding unit 650 or the width or height of the left encoding unit 660a and the middle encoding unit 660b. The image decoding device 100 can determine encoding units with dimensions different from the dimensions of other encoding units based on the determined widths and heights of the encoding units 660a to 660c. (Refer to...) Figure 6 The image decoding device 100 can determine an intermediate encoding unit 660b, which has a size different from that of the left encoding unit 660a and the right encoding unit 660c, as an encoding unit at a predetermined position. However, the method described above by the image decoding device 100 for determining an encoding unit with a size different from that of other encoding units corresponds only to the example of determining an encoding unit at a predetermined position by using the size of the encoding unit determined based on the coordinates of sample points. Therefore, various methods can be used to determine an encoding unit at a predetermined position by comparing the sizes of encoding units determined based on the coordinates of predetermined sample points.
[0132] However, the position of the sample points considered in determining the position of the coding unit is not limited to the aforementioned upper left position, and information about any position of the sample points included in the coding unit can be used.
[0133] According to an embodiment, the image decoding device 100 may consider the shape of the current coding unit and select a coding unit at a predetermined position from an odd number of coding units determined by dividing the current coding unit. For example, when the current coding unit has a non-square shape with a width greater than its height, the image decoding device 100 may determine a coding unit at a predetermined position in the horizontal direction. That is, the image decoding device 100 may determine one coding unit from the coding units at different positions in the horizontal direction and impose restrictions on that coding unit. When the current coding unit has a non-square shape with a height greater than its width, the image decoding device 100 may determine a coding unit at a predetermined position in the vertical direction. That is, the image decoding device 100 may determine one coding unit from the coding units at different positions in the vertical direction and impose restrictions on that coding unit.
[0134] According to an embodiment, the image decoding device 100 can use information indicating the positions of each of an even number of coding units to determine a coding unit at a predetermined position among the even number of coding units. The image decoding device 100 can determine the even number of coding units by dividing the current coding unit (bipartitioning), and can determine the coding unit at the predetermined position by using information about the positions of the even number of coding units. The associated operations are similar to those already described above regarding... Figure 6 The operation of the coding unit at a predetermined position (e.g., the center position) in an odd number of coding units is described in detail, and therefore its detailed description is not provided here.
[0135] According to an embodiment, when a non-square current coding unit is divided into multiple coding units, predetermined information about the coding unit at a predetermined position can be used in the division operation to determine the coding unit at the predetermined position among the multiple coding units. For example, the image decoding device 100 can use at least one of the block shape information and the division shape pattern information stored in the samples included in the intermediate coding unit during the division operation to determine the coding unit at the center position among the multiple coding units determined by dividing the current coding unit.
[0136] Reference Figure 6The image decoding device 100 can divide the current coding unit 600 into multiple coding units 620a, 620b, and 620c based on the division shape pattern information, and can determine the coding unit 620b at the center position among the multiple coding units 620a, 620b, and 620c. Furthermore, the image decoding device 100 can determine the coding unit 620b at the center position by considering the location where the division shape pattern information is obtained. That is, the division shape pattern information of the current coding unit 600 can be obtained from the sample point 640 at the center position of the current coding unit 600, and when the current coding unit 600 is divided into multiple coding units 620a, 620b, and 620c based on the division shape pattern information, the coding unit 620b including the sample point 640 can be determined as the coding unit at the center position. However, the information used to determine the coding unit at the center position is not limited to the division shape pattern information, and various types of information can be used to determine the coding unit at the center position.
[0137] According to an embodiment, predetermined information for identifying the coding unit at a predetermined location can be obtained from predetermined samples included in the coding unit to be determined. (Refer to...) Figure 6 The image decoding device 100 can use partitioning shape pattern information obtained from samples at predetermined positions in the current encoding unit 600 (e.g., samples at the center of the current encoding unit 600) to determine the encoding unit at a predetermined position (e.g., the encoding unit at the center of the partitioned encoding units) among a plurality of encoding units 620a, 620b, and 620c determined by partitioning the current encoding unit 600. That is, the image decoding device 100 can determine the samples at predetermined positions by considering the block shape of the current encoding unit 600, determine the encoding unit 620b including samples from the plurality of encoding units 620a, 620b, and 620c determined by partitioning the current encoding unit 600, and can impose predetermined constraints on the encoding unit 620b. (Refer to...) Figure 6 According to an embodiment, during the decoding operation, the image decoding device 100 can determine a sample 640 at the center position of the current encoding unit 600 as a sample from which predetermined information can be obtained, and can impose a predetermined restriction on the encoding unit 620b including the sample 640. However, the position of the sample from which predetermined information can be obtained is not limited to the above-described position, and can be included at any position of the sample included in the encoding unit 620b to be determined for restriction.
[0138] According to an embodiment, the location of a sample point from which predetermined information can be obtained can be determined based on the shape of the current encoding unit 600. According to an embodiment, block shape information can indicate whether the current encoding unit has a square shape or a non-square shape, and the location of the sample point from which predetermined information can be obtained can be determined based on this shape. For example, the image decoding device 100 can determine a sample point located on a boundary that bisects at least one of the width and height of the current encoding unit as a sample point from which predetermined information can be obtained by using at least one of information about the width of the current encoding unit and information about the height of the current encoding unit. As another example, when the block shape information of the current encoding unit indicates a non-square shape, the image decoding device 100 can determine one of the sample points adjacent to the boundary that bisects the long side of the current encoding unit as a sample point from which predetermined information can be obtained.
[0139] According to an embodiment, when the current coding unit is divided into multiple coding units, the image decoding device 100 can use partition shape pattern information to determine the coding unit at a predetermined position among the multiple coding units. According to an embodiment, the image decoding device 100 can obtain partition shape pattern information from samples at predetermined positions in the coding units, and can divide the multiple coding units generated by dividing the current coding unit using partition shape pattern information obtained from samples at predetermined positions in each of the multiple coding units. That is, the coding units can be recursively divided based on partition shape pattern information obtained from samples at predetermined positions in each coding unit. The above already relates to... Figure 5 The operation of recursively partitioning the coding unit is described, and therefore its detailed description will not be provided here.
[0140] According to an embodiment, the image decoding device 100 can determine one or more coding units by dividing the current coding unit, and can determine the order of decoding one or more coding units based on a predetermined block (e.g., the current coding unit).
[0141] Figure 7 This illustrates the order in which the image decoding device 100 processes the plurality of coding units when it determines such units by dividing the current coding unit, according to an embodiment.
[0142] According to an embodiment, based on the division shape pattern information, the image decoding device 100 can determine the second coding units 710a and 710b by dividing the first coding unit 700 along the vertical direction, determine the second coding units 730a and 730b by dividing the first coding unit 700 along the horizontal direction, or determine the second coding units 750a to 750d by dividing the first coding unit 700 along both the vertical and horizontal directions.
[0143] Reference Figure 7 The image decoding device 100 can determine to process the second coding units 710a and 710b, determined by dividing the first coding unit 700 along the vertical direction, in a horizontal order 710c. The image decoding device 100 can determine to process the second coding units 730a and 730b, determined by dividing the first coding unit 700 along the horizontal direction, in a vertical order 730c. The image decoding device 100 can determine to process the second coding units 750a to 750d, determined by dividing the first coding unit 700 along both the vertical and horizontal directions, according to a predetermined order (e.g., raster scan order or zigzag scan order 750e), wherein the predetermined order is to process coding units in one row and then process coding units in the next row.
[0144] According to an embodiment, the image decoding device 100 can recursively divide the encoding units. (Refer to...) Figure 7 The image decoding device 100 can determine a plurality of coding units 710a and 710b, 730a and 730b, or 750a to 750d by dividing the first coding unit 700, and recursively divides each of the determined plurality of coding units 710a and 710b, 730a and 730b, or 750a to 750d. The method of dividing the plurality of coding units 710a and 710b, 730a and 730b, or 750a to 750d can correspond to the method of dividing the first coding unit 700. In this way, each of the plurality of coding units 710a and 710b, 730a and 730b, or 750a to 750d can be independently divided into a plurality of coding units. (Refer to...) Figure 7 The image decoding device 100 can determine the second coding units 710a and 710b by dividing the first coding unit 700 along the vertical direction, and can determine whether to divide each of the second coding units 710a and 710b independently or not to divide each of the second coding units 710a and 710b.
[0145] According to an embodiment, the image decoding device 100 can determine the third coding units 720a and 720b by dividing the left second coding unit 710a along the horizontal direction, and may not divide the right second coding unit 710b.
[0146] According to an embodiment, the processing order of coding units can be determined based on the operation of dividing the coding units. In other words, the processing order of the divided coding units can be determined based on the processing order of the coding units immediately preceding the division. The image decoding device 100 can determine the processing order of the third coding units 720a and 720b determined by dividing the left second coding unit 710a independently of the right second coding unit 710b. Since the third coding units 720a and 720b are determined by dividing the left second coding unit 710a horizontally, the third coding units 720a and 720b can be processed in the vertical order 720c. Since the left second coding unit 710a and the right second coding unit 710b are processed in the horizontal order 710c, the right second coding unit 710b can be processed after the third coding units 720a and 720b included in the left second coding unit 710a are processed in the vertical order 720c. The operation of determining the processing order of coding units based on the coding units before they are divided is not limited to the example above, and various methods can be used to process coding units that have been divided and determined to be of various shapes independently in a predetermined order.
[0147] Figure 8 The illustration shows a process performed by an image decoding device 100 according to an embodiment, in which the current coding unit is determined to be divided into an odd number of coding units when the coding units cannot be processed in a predetermined order.
[0148] According to an embodiment, the image decoding device 100 can determine whether the current coding unit is divided into an odd number of coding units based on the obtained partition shape pattern information. (See also...) Figure 8 The square first coding unit 800 can be divided into non-square second coding units 810a and 810b, and the second coding units 810a and 810b can be independently divided into third coding units 820a and 820b, as well as 820c to 820e. According to an embodiment, the image decoding device 100 can determine a plurality of third coding units 820a and 820b by dividing the left second coding unit 810a along the horizontal direction, and can divide the right second coding unit 810b into an odd number of third coding units 820c to 820e.
[0149] According to an embodiment, the image decoding device 100 can determine whether to divide any coding unit into an odd number of coding units by determining whether the third coding units 820a and 820b and 820c to 820e can be processed in a predetermined order. (Refer to...) Figure 8The image decoding device 100 can determine the third coding units 820a and 820b, and 820c to 820e by recursively dividing the first coding unit 800. The image decoding device 100 can determine whether any one of the following coding units is divided into an odd number of coding units based on at least one of block shape information and a division shape pattern: the first coding unit 800, the second coding units 810a and 810b, and the third coding units 820a and 820b, and 820c to 820e. For example, the rightmost second coding unit 810b of the second coding units 810a and 810b can be divided into an odd number of third coding units 820c, 820d, and 820e. The processing order of the plurality of coding units included in the first coding unit 800 may be a predetermined order (e.g., zigzag scanning order 830), and the image decoding device 100 may determine whether the third coding units 820c, 820d, and 820e, which are determined by dividing the right second coding unit 810b into an odd number of coding units, meet the conditions for processing in a predetermined order.
[0150] According to an embodiment, the image decoding device 100 can determine whether third encoding units 820a and 820b, and 820c to 820e, included in the first encoding unit 800, satisfy conditions for processing in a predetermined order, and said conditions are related to whether at least one of the width and height of the second encoding units 810a and 810b will be halved along the boundaries of the third encoding units 820a and 820b, and 820c to 820e. For example, the third encoding units 820a and 820b determined when the height of the non-square-shaped left second encoding unit 810a is halved can satisfy said conditions. Because the boundaries of the third encoding units 820c to 820e determined when the right second encoding unit 810b is divided into three encoding units cannot halve the width or height of the right second encoding unit 810b, it can be determined that the third encoding units 820c to 820e do not satisfy said conditions. When the conditions described above are not met, the image decoding device 100 can determine that the scanning order is discontinuous, and based on the determination result, can determine that the right-side second coding unit 810b will be divided into an odd number of coding units. According to an embodiment, when the coding unit is divided into an odd number of coding units, the image decoding device 100 can impose a predetermined restriction on the coding units at predetermined positions in the divided coding units. The restriction or the predetermined position has been described above with respect to various embodiments, and therefore its detailed description will not be provided here.
[0151] Figure 9 The illustration shows a process performed by an image decoding device 100 according to an embodiment, which involves dividing a first encoding unit 900 to determine at least one encoding unit.
[0152] According to an embodiment, the image decoding device 100 can divide the first coding unit 900 based on the division shape pattern information obtained by the receiver 110. The square first coding unit 900 can be divided into four square coding units, or it can be divided into multiple non-square coding units. For example, referring to… Figure 9 When the partitioning shape pattern information indicates that the first coding unit 900 should be divided into non-square coding units, the image decoding device 100 may divide the first coding unit 900 into a plurality of non-square coding units. Specifically, when the partitioning shape pattern information indicates that an odd number of coding units should be determined by partitioning the first coding unit 900 along a horizontal or vertical direction, the image decoding device 100 may divide the square first coding unit 900 into an odd number of coding units (e.g., second coding units 910a, 910b, and 910c determined by partitioning the square first coding unit 900 along a vertical direction, or second coding units 920a, 920b, and 920c determined by partitioning the square first coding unit 900 along a horizontal direction).
[0153] According to an embodiment, the image decoding device 100 can determine whether second encoding units 910a, 910b, 910c, 920a, 920b, and 920c included in the first encoding unit 900 satisfy conditions for processing in a predetermined order, and said conditions are related to whether at least one of the width and height of the first encoding unit 900 will be halved along the boundaries of the second encoding units 910a, 910b, 910c, 920a, 920b, and 920c. (Refer to...) Figure 9 Because the boundaries of the second coding units 910a, 910b, and 910c, determined by dividing the first coding unit 900 into a square along the vertical direction, do not halve the width of the first coding unit 900, it can be determined that the first coding unit 900 does not meet the conditions for processing in a predetermined order. Furthermore, because the boundaries of the second coding units 920a, 920b, and 920c, determined by dividing the first coding unit 900 into a square along the horizontal direction, do not halve the height of the first coding unit 900, it can be determined that the first coding unit 900 does not meet the conditions for processing in a predetermined order. When the conditions are not met as described above, the image decoding device 100 can determine that the scanning order is discontinuous, and can determine, based on the determination result, that the first coding unit 900 is divided into an odd number of coding units. According to an embodiment, when the coding unit is divided into an odd number of coding units, the image decoding device 100 can impose a predetermined restriction on the coding units at predetermined positions within the divided coding units. The restrictions or predetermined positions have already been described above with respect to various embodiments, and therefore will not be described in detail here.
[0154] According to an embodiment, the image decoding device 100 can determine encoding units of various shapes by dividing the first encoding unit.
[0155] Reference Figure 9 The image decoding device 100 can divide the square first coding unit 900 or the non-square first coding unit 930 or 950 into coding units of various shapes.
[0156] Figure 10 The shape into which the second encoding unit can be divided is limited when the second encoding unit, which has a non-square shape, is determined when the first encoding unit 1000 is divided by the image decoding device 100 according to the embodiment, meets predetermined conditions.
[0157] According to an embodiment, the image decoding device 100 can determine, based on the division shape pattern information obtained by the receiver 110, to divide a square first coding unit 1000 into non-square second coding units 1010a and 1010b or 1020a and 1020b. The second coding units 1010a and 1010b or 1020a and 1020b can be divided independently. In this way, based on the division shape pattern information of each of the second coding units 1010a and 1010b or 1020a and 1020b, the image decoding device 100 can determine whether to divide each of the second coding units 1010a and 1010b or 1020a and 1020b into multiple coding units or not to divide each of the second coding units 1010a and 1010b or 1020a and 1020b. According to an embodiment, the image decoding device 100 can determine the third coding units 1012a and 1012b by dividing the non-square left second coding unit 1010a, which is determined by dividing the first coding unit 1000 along the vertical direction, along the horizontal direction. However, when the left second coding unit 1010a is divided along the horizontal direction, the image decoding device 100 can restrict the right second coding unit 1010b from being divided along the horizontal direction in which the left second coding unit 1010a is divided. When the third coding units 1014a and 1014b are determined by dividing the right second coding unit 1010b along the same direction, since the left second coding unit 1010a and the right second coding unit 1010b are divided independently along the horizontal direction, the third coding units 1012a and 1012b or 1014a and 1014b can be determined. However, this situation works in the same way as when the image decoding device 100 divides the first encoding unit 1000 into four squares, namely the second encoding units 1030a, 1030b, 1030c and 1030d, based on the division shape pattern information, and may be inefficient in terms of image decoding.
[0158] According to an embodiment, the image decoding device 100 can determine the third encoding units 1022a and 1022b or 1024a and 1024b by dividing the non-square second encoding units 1020a or 1020b, which are determined by dividing the first encoding unit 1000 along the horizontal direction, along the vertical direction. However, when the second encoding unit (e.g., the upper second encoding unit 1020a) is divided along the vertical direction, for the reasons described above, the image decoding device 100 can restrict another second encoding unit (e.g., the lower second encoding unit 1020b) from being divided along the vertical direction in which the upper second encoding unit 1020a is divided.
[0159] Figure 11 The illustration shows the process performed by the image decoding device 100 according to an embodiment, when the division shape pattern information cannot indicate that the square coding unit is divided into four square coding units, for dividing the square coding unit.
[0160] According to an embodiment, the image decoding device 100 can determine second coding units 1110a and 1110b, or 1120a and 1120b, etc., by dividing the first coding unit 1100 based on division shape pattern information. The division shape pattern information may include information about various methods of dividing the coding unit, but may not include information about dividing the coding unit into four square coding units. Based on such division shape pattern information, the image decoding device 100 may not divide the square first coding unit 1100 into four square second coding units 1130a, 1130b, 1130c, and 1130d. The image decoding device 100 can determine non-square second coding units 1110a and 1110b, or 1120a and 1120b, etc., based on the division shape pattern information.
[0161] According to an embodiment, the image decoding device 100 can independently divide non-square second coding units 1110a and 1110b or 1120a and 1120b, etc. Each of the second coding units 1110a and 1110b or 1120a and 1120b, etc., can be recursively divided in a predetermined order, and this division method can correspond to the method of dividing the first coding unit 1100 based on the division shape pattern information.
[0162] For example, the image decoding device 100 can determine the third coding units 1112a and 1112b of the square by dividing the left second coding unit 1110a horizontally, and can determine the third coding units 1114a and 1114b of the square by dividing the right second coding unit 1110b horizontally. Furthermore, the image decoding device 100 can determine the third coding units 1116a, 1116b, 1116c, and 1116d of the square by dividing both the left second coding unit 1110a and the right second coding unit 1110b horizontally. In this case, coding units with the same shape as the second coding units 1130a, 1130b, 1130c, and 1130d of the four squares divided from the first coding unit 1100 can be determined.
[0163] As another example, the image decoding device 100 can determine the square third coding units 1122a and 1122b by dividing the upper second coding unit 1120a vertically, and can determine the square third coding units 1124a and 1124b by dividing the lower second coding unit 1120b vertically. Furthermore, the image decoding device 100 can determine the square third coding units 1126a, 1126b, 1126c, and 1126d by dividing both the upper and lower second coding units 1120a and 1120b vertically. In this case, coding units with the same shape as the four squares divided from the first coding unit 1100, namely the second coding units 1130a, 1130b, 1130c, and 1130d, can be determined.
[0164] Figure 12 This illustrates that, according to an embodiment, the processing order among multiple coding units can be changed based on the process of dividing the coding units.
[0165] According to an embodiment, the image decoding device 100 can divide the first encoding unit 1200 based on partition shape pattern information. When the block shape indicates a square shape and the partition shape pattern information indicates that the first encoding unit 1200 is divided along at least one direction, either horizontal or vertical, the image decoding device 100 can determine the second encoding units 1210a and 1210b or 1220a and 1220b by dividing the first encoding unit 1200. (Refer to...) Figure 12The non-square second coding units 1210a and 1210b, or 1220a and 1220b, determined by dividing the first coding unit 1200 only along the horizontal or vertical direction, can be independently divided based on the division shape pattern information of each coding unit. For example, the image decoding device 100 can determine third coding units 1216a, 1216b, 1216c, and 1216d by dividing the second coding units 1210a and 1210b generated by dividing the first coding unit 1200 along the vertical direction along the horizontal direction, and can determine third coding units 1226a, 1226b, 1226c, and 1226d by dividing the second coding units 1220a and 1220b generated by dividing the first coding unit 1200 along the horizontal direction along the vertical direction. The above already relates to... Figure 11 The operation of dividing the second coding units 1210a and 1210b or 1220a and 1220b is described, so its detailed description will not be provided here.
[0166] According to an embodiment, the image decoding device 100 can process the encoding units in a predetermined order. The above has already mentioned... Figure 7 The operation of processing the encoded units in a predetermined order is described, therefore its detailed description will not be provided here. (See reference...) Figure 12 The image decoding device 100 can determine the third encoding units 1216a, 1216b, 1216c, and 1216d, as well as 1226a, 1226b, 1226c, and 1226d of the four squares by dividing the first encoding unit 1200 of the square. According to an embodiment, the image decoding device 100 can determine the processing order of the third encoding units 1216a, 1216b, 1216c, and 1216d, as well as 1226a, 1226b, 1226c, and 1226d, based on the division method of the first encoding unit 1200.
[0167] According to an embodiment, the image decoding device 100 can determine third coding units 1216a, 1216b, 1216c, and 1216d by dividing second coding units 1210a and 1210b, which are generated by dividing the first coding unit 1200 in the vertical direction, in the horizontal direction. The third coding units 1216a, 1216b, 1216c, and 1216d can be processed in the following processing order 1217: firstly, the third coding units 1216a and 1216c, which are included in the left second coding unit 1210a, are processed in the vertical direction, and then the third coding units 1216b and 1216d, which are included in the right second coding unit 1210b, are processed in the vertical direction.
[0168] According to an embodiment, the image decoding device 100 can determine third coding units 1226a, 1226b, 1226c, and 1226d by dividing second coding units 1220a and 1220b, which are generated by dividing the first coding unit 1200 in the horizontal direction, in the vertical direction. The third coding units 1226a, 1226b, 1226c, and 1226d can be processed in the following processing order 1227: firstly, the third coding units 1226a and 1226b, which are included in the upper second coding unit 1220a, are processed in the horizontal direction, and then the third coding units 1226c and 1226d, which are included in the lower second coding unit 1220b, are processed in the horizontal direction.
[0169] Reference Figure 12 The third coding units 1216a, 1216b, 1216c, and 1216d, as well as 1226a, 1226b, 1226c, and 1226d of the square can be determined by dividing the second coding units 1210a and 1210b and 1220a and 1220b respectively. Although the second coding units 1210a and 1210b, determined by dividing the first coding unit 1200 vertically, are different from the second coding units 1220a and 1220b, determined by dividing the first coding unit 1200 horizontally, the third coding units 1216a, 1216b, 1216c, and 1216d, as well as 1226a, 1226b, 1226c, and 1226d, derived from the second coding units 1210a and 1210b and 1220a and 1220b, ultimately show coding units of the same shape as those derived from the first coding unit 1200. In this way, by recursively dividing the coding units in different ways based on the division shape pattern information, even if the coding units are ultimately determined to have the same shape, the image decoding device 100 can process multiple coding units in different orders.
[0170] Figure 13 The illustration shows a process for determining the depth of a coding unit as the shape and size of the coding unit change when recursively dividing the coding unit to determine multiple coding units, according to an embodiment.
[0171] According to an embodiment, the image decoding device 100 can determine the depth of the coding unit based on a predetermined criterion. For example, the predetermined criterion could be the length of the long side of the coding unit. When the length of the long side of the coding unit before it was divided is 2n (n > 0) times the length of the long side of the divided current coding unit, the image decoding device 100 can determine that the depth of the current coding unit is increased by n compared to the depth of the coding unit before it was divided. In the following description, the coding unit with the increased depth is referred to as a deeper coding unit.
[0172] Reference Figure 13 According to an embodiment, the image decoding device 100 can determine deeper second and third coding units 1302 and 1304 by dividing a first coding unit 1300 of a square based on block shape information indicating the shape of the square (e.g., the block shape information may be represented as "0:SQUARE"). Assuming the size of the first coding unit 1300 of the square is 2N×2N, the second coding unit 1302, determined by dividing the width and height of the first coding unit 1300 by half, can have a size of N×N. Furthermore, the third coding unit 1304, determined by dividing the width and height of the second coding unit 1302 by half, can have a size of N / 2×N / 2. In this case, the width and height of the third coding unit 1304 are 1 / 4 of the width and height of the first coding unit 1300. When the depth of the first coding unit 1300 is D, the depth of the second coding unit 1302, whose width and height are half of the width and height of the first coding unit 1300, can be D+1, and the depth of the third coding unit 1304, whose width and height are one-quarter of the width and height of the first coding unit 1300, can be D+2.
[0173] According to an embodiment, the image decoding device 100 can determine a deeper second encoding unit 1312 or 1322 and a third encoding unit 1314 or 1324 by dividing the non-square first encoding unit 1310 or 1320 based on block shape information indicating a non-square shape (for example, the block shape information may be represented as "1:NS_VER" indicating a non-square shape with a height longer than its width, or "2:NS_HOR" indicating a non-square shape with a width longer than its height).
[0174] The image decoding device 100 can determine the second encoding unit 1302, 1312, or 1322 by dividing at least one of the width and height of the first encoding unit 1310, which has a size of N×2N. That is, the image decoding device 100 can determine the second encoding unit 1302 or the second encoding unit 1322 with a size of N×N by dividing the first encoding unit 1310 along the horizontal direction, or it can determine the second encoding unit 1312 with a size of N / 2×N by dividing the first encoding unit 1310 along both the horizontal and vertical directions.
[0175] According to an embodiment, the image decoding device 100 can determine the second encoding unit 1302, 1312, or 1322 by dividing at least one of the width and height of the first encoding unit 1320, which has a size of 2N×N. That is, the image decoding device 100 can determine the second encoding unit 1302 or the second encoding unit 1312 with a size of N×N by dividing the first encoding unit 1320 along the vertical direction, or it can determine the second encoding unit 1322 with a size of N×N / 2 by dividing the first encoding unit 1320 along both the horizontal and vertical directions.
[0176] According to an embodiment, the image decoding device 100 can determine a third encoding unit 1304, 1314, or 1324 by dividing at least one of the width and height of the second encoding unit 1302, which has a size of N×N. That is, the image decoding device 100 can determine a third encoding unit 1304 with a size of N / 2×N / 2, a third encoding unit 1314 with a size of N / 4×N / 2, or a third encoding unit 1324 with a size of N / 2×N / 4 by dividing the second encoding unit 1302 along the vertical and horizontal directions.
[0177] According to an embodiment, the image decoding device 100 can determine the third encoding unit 1304, 1314, or 1324 by dividing at least one of the width and height of the second encoding unit 1312, which has a size of N / 2 × N. That is, the image decoding device 100 can determine the third encoding unit 1304 or the third encoding unit 1324 with a size of N / 2 × N / 4 by dividing the second encoding unit 1312 along the horizontal direction, or it can determine the third encoding unit 1314 with a size of N / 4 × N / 2 by dividing the second encoding unit 1312 along both the vertical and horizontal directions.
[0178] According to an embodiment, the image decoding device 100 can determine a third encoding unit 1304, 1314, or 1324 by dividing at least one of the width and height of a second encoding unit 1322 with a size of N×N / 2. That is, the image decoding device 100 can determine a third encoding unit 1304 with a size of N / 2×N / 2 or a third encoding unit 1314 with a size of N / 4×N / 2 by dividing the second encoding unit 1322 along the vertical direction, or it can determine a third encoding unit 1324 with a size of N / 2×N / 4 by dividing the second encoding unit 1322 along both the vertical and horizontal directions.
[0179] According to an embodiment, the image decoding device 100 can divide the square coding units 1300, 1302, or 1304 along a horizontal or vertical direction. For example, the image decoding device 100 can determine a first coding unit 1310 of size N×2N by dividing the first coding unit 1300 of size 2N×2N along a vertical direction, or it can determine a first coding unit 1320 of size 2N×N by dividing the first coding unit 1300 along a horizontal direction. According to an embodiment, when the depth is determined based on the length of the longest side of the coding unit, the depth of the coding unit determined by dividing the first coding unit 1300 of size 2N×2N along a horizontal or vertical direction can be the same as the depth of the first coding unit 1300.
[0180] According to an embodiment, the width and height of the third encoding unit 1314 or 1324 can be 1 / 4 of the width and height of the first encoding unit 1310 or 1320. When the depth of the first encoding unit 1310 or 1320 is D, the depth of the second encoding unit 1312 or 1322, whose width and height are 1 / 2 of the width and height of the first encoding unit 1310 or 1320, can be D+1, and the depth of the third encoding unit 1314 or 1324, whose width and height are 1 / 4 of the width and height of the first encoding unit 1310 or 1320, can be D+2.
[0181] Figure 14 The diagram illustrates a depth that can be determined based on the shape and size of the coding unit, and a partial index (PID) used to distinguish the coding unit, according to an embodiment.
[0182] According to an embodiment, the image decoding device 100 can determine second coding units of various shapes by dividing a square first coding unit 1400. (See also...) Figure 14 The image decoding device 100 can determine the second encoding units 1402a and 1402b, 1404a and 1404b, and 1406a, 1406b, 1406c and 1406d by dividing the first encoding unit 1400 along at least one of the vertical and horizontal directions based on the division shape pattern information. In other words, the image decoding device 100 can determine the second encoding units 1402a and 1402b, 1404a and 1404b, and 1406a, 1406b, 1406c and 1406d based on the division shape pattern information of the first encoding unit 1400.
[0183] According to an embodiment, the depths of the second coding units 1402a and 1402b, second coding units 1404a and 1404b, and second coding units 1406a, 1406b, 1406c, and 1406d, determined based on the division shape pattern information of the square first coding unit 1400, can be determined based on the length of their longer sides. For example, since the length of the side of the square first coding unit 1400 is equal to the length of the longer side of the non-square second coding units 1402a and 1402b and 1404a and 1404b, the first coding unit 1400 and the non-square second coding units 1402a and 1402b and 1404a and 1404b can have the same depth (e.g., D). However, when the image decoding device 100 divides the first encoding unit 1400 into four square second encoding units 1406a, 1406b, 1406c and 1406d based on the division shape pattern information, the depth of the second encoding units 1406a, 1406b, 1406c and 1406d can be D+1, which is 1 deeper than the depth D of the first encoding unit 1400.
[0184] According to an embodiment, the image decoding device 100 can determine a plurality of second encoding units 1412a and 1412b, as well as 1414a, 1414b, and 1414c, by dividing a first encoding unit 1410, whose height is longer than its width, along the horizontal direction based on division shape pattern information. According to an embodiment, the image decoding device 100 can determine a plurality of second encoding units 1422a and 1422b, as well as 1424a, 1424b, and 1424c, by dividing a first encoding unit 1420, whose width is longer than its height, along the vertical direction based on division shape pattern information.
[0185] According to an embodiment, the depths of the second coding units 1412a and 1412b, and the second coding units 1414a, 1414b and 1414c, or the second coding units 1422a and 1422b, and the second coding units 1424a, 1424b and 1424c, determined based on the division shape pattern information of the non-square first coding unit 1410 or 1422b, can be determined based on the length of their longer sides. For example, since the length of the side of the square second coding units 1412a and 1412b is half the length of the longer side of the non-square first coding unit 1410, which has a height longer than its width, the depth of the square second coding units 1412a and 1412b is D+1 deeper than the depth D of the non-square first coding unit 1410.
[0186] Furthermore, the image decoding device 100 can divide the non-square first coding unit 1410 into an odd number of second coding units 1414a, 1414b, and 1414c based on the division shape pattern information. The odd number of second coding units 1414a, 1414b, and 1414c may include the non-square second coding units 1414a and 1414c and the square second coding unit 1414b. In this case, since the length of the longer side of the non-square second coding units 1414a and 1414c and the length of the side of the square second coding unit 1414b are half the length of the longer side of the first coding unit 1410, the depth of the second coding units 1414a, 1414b, and 1414c can be D+1, which is 1 greater than the depth D of the non-square first coding unit 1410. The image decoding device 100 can determine the depth of the coding unit divided from the first coding unit 1420, which has a width longer than its height, by using the method described above for determining the depth of the coding unit divided from the first coding unit 1410.
[0187] According to an embodiment, when the odd number of divided coding units do not have equal sizes, the image decoding device 100 can determine the PID used to identify the divided coding units based on the size ratio between the coding units. (Refer to...) Figure 14 The width of the central coding unit 1414b among the odd-numbered coding units 1414a, 1414b, and 1414c can be equal to the width of the other coding units 1414a and 1414c, and its height can be twice the height of the other coding units 1414a and 1414c. That is, in this case, the central coding unit 1414b may include two other coding units 1414a or 1414c. Therefore, when the PID of the central coding unit 1414b is 1 based on the scan order, the PID of the coding unit 1414c adjacent to coding unit 1414b can be increased by 2 and thus can be 3. That is, there may be discontinuous PID values. According to an embodiment, the image decoding device 100 can determine whether the odd-numbered coding units do not have equal sizes based on whether there is a discontinuity in the PIDs used to identify the divided coding units.
[0188] According to an embodiment, the image decoding device 100 may determine whether to use a specific partitioning method based on PID values used to identify multiple coding units determined by partitioning the current coding unit. (See also...) Figure 14The image decoding device 100 can determine an even number of coding units 1412a and 1412b or an odd number of coding units 1414a, 1414b, and 1414c by dividing a first coding unit 1410 having a rectangular shape with a height longer than its width. The image decoding device 100 can use a PID to identify each coding unit. According to an embodiment, the PID can be obtained from a sample point at a predetermined position of each coding unit (e.g., the upper left sample point).
[0189] According to an embodiment, the image decoding device 100 can determine the coding unit at a predetermined position among the divided coding units by using a PID for distinguishing coding units. According to an embodiment, when the division shape pattern information of a first coding unit 1410 having a rectangular shape with a height longer than its width indicates that the coding unit should be divided into three coding units, the image decoding device 100 can divide the first coding unit 1410 into three coding units 1414a, 1414b, and 1414c. The image decoding device 100 can assign a PID to each of the three coding units 1414a, 1414b, and 1414c. The image decoding device 100 can compare the PIDs of the divided odd-numbered coding units to determine the coding unit at the center position among the divided odd-numbered coding units. The image decoding device 100 can determine the coding unit 1414b, which has a PID corresponding to the median value among the PIDs of the coding unit, as the coding unit at the center position among the coding units determined by dividing the first coding unit 1410. According to an embodiment, when the divided coding units do not have equal sizes, the image decoding device 100 can determine a PID for distinguishing the divided coding units based on the size ratio between the coding units. (Refer to...) Figure 14The width of the coding unit 1414b generated by dividing the first coding unit 1410 can be equal to the width of the other coding units 1414a and 1414c, and the height of the coding unit 1414b can be twice the height of the other coding units 1414a and 1414c. In this case, when the PID of the coding unit 1414b at the center position is 1, the PID of the coding unit 1414c adjacent to the coding unit 1414b can be increased by 2 and therefore can be 3. When the PID does not increase uniformly as described above, the image decoding device 100 can determine that the coding unit is divided into a plurality of coding units including coding units with different sizes from the other coding units. According to an embodiment, when the division shape pattern information indicates that the coding unit is divided into an odd number of coding units, the image decoding device 100 can divide the current coding unit in such a way that the coding unit at a predetermined position in the odd number of coding units (e.g., the coding unit at the center position) has a size different from the other coding units. In this case, the image decoding device 100 can determine the coding unit at the center position with different sizes by using the PID of the coding unit. However, the PID of the encoding unit at the predetermined position and its size or position are not limited to the examples above, and various PIDs of the encoding unit as well as various positions and sizes can be used.
[0190] According to an embodiment, the image decoding device 100 may use a predetermined data unit, in which encoding units are recursively divided.
[0191] Figure 15 The illustration shows how multiple encoding units are determined based on multiple predetermined data units included in the image, according to an embodiment.
[0192] According to an embodiment, a predetermined data unit can be defined as a data unit that recursively divides coding units by using division shape pattern information. That is, the predetermined data unit may correspond to a coding unit used to determine the highest depth of a plurality of coding units divided from the current frame. In the following description, for ease of explanation, the predetermined data unit is referred to as a reference data unit.
[0193] According to an embodiment, the reference data unit may have a predetermined size and a predetermined shape. According to an embodiment, the reference data unit may include M×N sample points. Here, M and N may be equal to each other and may be integers represented as powers of 2. That is, the reference data unit may have a square shape or a non-square shape and may be divided into an integer number of encoding units.
[0194] According to an embodiment, the image decoding device 100 can divide the current frame into multiple reference data units. According to an embodiment, the image decoding device 100 can divide the multiple reference data units from the current frame using the division shape pattern information of each reference data unit. The operation of dividing the reference data units can correspond to a division operation using a quadtree structure.
[0195] According to an embodiment, the image decoding device 100 can predetermine the minimum allowed size of the reference data units included in the current frame. Therefore, the image decoding device 100 can determine various reference data units with sizes equal to or greater than the minimum size, and can determine one or more coding units by referring to the determined reference data units and using partitioning shape pattern information.
[0196] Reference Figure 15 The image decoding device 100 may use a square reference coding unit 1500 or a non-square reference coding unit 1502. According to an embodiment, the shape and size of the reference coding unit may be determined based on various data units (e.g., sequences, frames, stripes, strip segments, maximum coding units, etc.) that can include one or more reference coding units.
[0197] According to an embodiment, the receiver 110 of the image decoding device 100 can obtain from the bitstream at least one of reference coding unit shape information and reference coding unit size information for each of the various data units. The above has already discussed... Figure 3 The operation of dividing the current coding unit 300 describes the operation of dividing the square reference coding unit 1500 into one or more coding units, and the above has already been discussed... Figure 4 The operation of dividing the current coding unit 400 or 450 describes the operation of dividing the non-square reference coding unit 1502 into one or more coding units. Therefore, its detailed description will not be provided here.
[0198] According to an embodiment, the image decoding device 100 can determine the size and shape of a reference coding unit using a PID (PID for identifying the size and shape of a reference coding unit) based on some data units previously determined based on predetermined conditions. That is, the receiver 110 can obtain from the bitstream only the PID used to identify the size and shape of the reference coding unit for each slice, slice segment, or maximum coding unit, wherein each slice, slice segment, or maximum coding unit is a data unit among various data units (e.g., sequence, frame, slice, slice segment, maximum coding unit, etc.) that satisfies predetermined conditions (e.g., a data unit with a size equal to or smaller than the slice). The image decoding device 100 can determine the size and shape of the reference data unit for each data unit that satisfies the predetermined conditions by using the PID. When obtaining and using reference coding unit shape information and reference coding unit size information from the bitstream based on each data unit with a relatively small size, the efficiency of using the bitstream may be low; therefore, only the PID can be obtained and used, rather than directly obtaining the reference coding unit shape information and reference coding unit size information. In this case, at least one of the size and shape of the reference coding unit corresponding to the PID used to identify the size and shape of the reference coding unit can be predetermined. In other words, the image decoding device 100 can determine at least one of the dimensions and shapes of the reference coding unit included in the data unit used as the unit for obtaining the PID by selecting at least one of the dimensions and shapes of the reference coding unit based on the PID.
[0199] According to embodiments, the image decoding device 100 may use one or more reference coding units included in the maximum coding unit. That is, the maximum coding unit divided from the image may include one or more reference coding units, and the coding unit can be determined by recursively dividing each reference coding unit. According to embodiments, at least one of the width and height of the maximum coding unit may be an integer multiple of at least one of the width and height of the reference coding unit. According to embodiments, the size of the reference coding unit can be obtained by dividing the maximum coding unit n times based on a quadtree structure. That is, according to various embodiments, the image decoding device 100 may determine the reference coding unit by dividing the maximum coding unit n times based on a quadtree structure, and may divide the reference coding unit based on at least one of block shape information and division shape pattern information.
[0200] Figure 16 The diagram illustrates a processing block, according to an embodiment, used as a standard for determining the order of reference coding units included in frame 1600.
[0201] According to an embodiment, the image decoding device 100 can determine one or more processing blocks divided from the image. A processing block is a data unit divided from the image that includes one or more reference coding units, and the one or more reference coding units included in the processing block can be determined according to a specific order. That is, the determination order of the one or more reference coding units determined in each processing block can correspond to one of various types of orders used to determine reference coding units, and can vary depending on the processing block. The determination order of the reference coding units determined for each processing block can be one of various orders (e.g., raster scan order, zigzag scan, N-shaped scan, upper right diagonal scan, horizontal scan, and vertical scan), but is not limited to the scan orders mentioned above.
[0202] According to an embodiment, the image decoding device 100 can obtain processing block size information and determine the size of one or more processing blocks included in the image. The image decoding device 100 can obtain processing block size information from a bitstream and determine the size of one or more processing blocks included in the image. The size of a processing block can be a predetermined size of a data unit indicated by the processing block size information.
[0203] According to an embodiment, the receiver 110 of the image decoding device 100 can obtain processing block size information from the bitstream based on each specific data unit. For example, processing block size information can be obtained from the bitstream according to data units such as images, sequences, frames, stripes, or strip segments. That is, the receiver 110 can obtain processing block size information from the bitstream based on each data unit among various data units, and the image decoding device 100 can determine the size of one or more processing blocks divided from the frame by using the obtained processing block size information. The size of the processing block can be an integer multiple of the size of the reference coding unit.
[0204] According to an embodiment, the image decoding device 100 can determine the sizes of processing blocks 1602 and 1612 included in the frame 1600. For example, the image decoding device 100 can determine the size of the processing blocks based on processing block size information obtained from the bitstream. (Refer to...) Figure 16 According to an embodiment, the image decoding device 100 may determine the width of processing blocks 1602 and 1612 to be four times the width of the reference coding unit, and may determine the height of processing blocks 1602 and 1612 to be four times the height of the reference coding unit. The image decoding device 100 may determine the determination order of one or more reference coding units in one or more processing blocks.
[0205] According to an embodiment, the image decoding device 100 may determine processing blocks 1602 and 1612 included in the frame 1600 based on the size of the processing blocks, and may determine the determination order of one or more reference coding units in processing blocks 1602 and 1612. According to an embodiment, the operation of determining the reference coding units may include determining the size of the reference coding units.
[0206] According to an embodiment, the image decoding device 100 can obtain determination order information of one or more reference coding units included in one or more processing blocks from a bitstream, and can determine a determination order for one or more reference coding units based on the obtained determination order information. The determination order information can be defined as the order or direction of the reference coding units in a processing block. That is, the determination order of the reference coding units can be determined independently for each processing block.
[0207] According to an embodiment, the image decoding device 100 can obtain deterministic order information of reference coding units from the bitstream based on each specific data unit. For example, the receiver 110 can obtain deterministic order information of reference coding units from the bitstream based on each data unit (such as an image, sequence, frame, strip, strip fragment, or processing block). Because the deterministic order information of the reference coding units indicates the order of the reference coding units used to determine the processing block, deterministic order information can be obtained for each specific data unit comprising an integer number of processing blocks.
[0208] According to an embodiment, the image decoding device 100 may determine one or more reference coding units based on a determined order.
[0209] According to an embodiment, receiver 110 can obtain the determined order information of reference coding units from the bitstream as information related to processing blocks 1602 and 1612, and image decoding device 100 can determine the determined order of one or more reference coding units including processing blocks 1602 and 1612, and determine one or more reference coding units included in frame 1600 based on the determined order. (Refer to...) Figure 16The image decoding device 100 can determine the determination order 1604 and 1614 of one or more reference coding units in processing blocks 1602 and 1612, respectively. For example, when the determination order information of reference coding units is obtained for each processing block, different types of determination order information of reference coding units can be obtained for processing blocks 1602 and 1612. When the determination order 1604 of reference coding units in processing block 1602 is a raster scan order, the reference coding units included in processing block 1602 can be determined according to the raster scan order. Conversely, when the determination order 1614 of reference coding units in another processing block 1612 is a reverse raster scan order, the reference coding units included in processing block 1612 can be determined according to the reverse raster scan order.
[0210] According to an embodiment, the image decoding device 100 can decode one or more determined reference coding units. The image decoding device 100 can decode an image based on the reference coding units determined as described above. Methods for decoding the reference coding units may include various image decoding methods.
[0211] According to an embodiment, the image decoding device 100 can obtain block shape information indicating the shape of the current coding unit or partition shape pattern information indicating the partitioning method of the current coding unit from the bitstream, and can use the obtained information. The partition shape pattern information can be included in the bitstream associated with various data units. For example, the image decoding device 100 can use the partition shape pattern information included in the sequence parameter set, picture parameter set, video parameter set, strip header, or strip segment header. Furthermore, the image decoding device 100 can obtain syntax elements corresponding to the block shape information or partition shape pattern information from the bitstream according to each maximum coding unit, each reference coding unit, or each processing block, and can use the obtained syntax elements.
[0212] The method for determining partitioning rules according to embodiments of the present disclosure will be described in detail below.
[0213] Image decoding device 100 can determine the segmentation rules of an image. The segmentation rules can be predetermined between image decoding device 100 and image encoding device. Image decoding device 100 can determine the segmentation rules based on information obtained from a bitstream. Image decoding device 100 can determine the segmentation rules based on information obtained from at least one of a sequence parameter set, a picture parameter set, a video parameter set, a strip header, or a strip segment header. Image decoding device 100 can determine different segmentation rules based on frames, stripes, time layers, maximum coding units, or coding units.
[0214] Image decoding device 100 may determine partitioning rules based on block shape information of coding units. Block shape may include the size, shape, width-to-height ratio, and orientation of the coding unit. Image encoding device and image decoding device 100 may pre-determine partitioning rules based on block shape information of coding units. However, embodiments are not limited to this. Image decoding device 100 may also determine partitioning rules based on information obtained from a bitstream received from image encoding device.
[0215] The shape of the encoding unit may include a square and a non-square shape. When the width and height of the encoding unit are the same, the image decoding device 100 may determine the shape of the encoding unit as a square. Furthermore, when the width and height of the encoding unit are not the same, the image decoding device 100 may determine the shape of the encoding unit as a non-square shape.
[0216] The size of the coding unit can include various sizes such as 4×4, 8×4, 4×8, 8×8, 16×4, 16×8 to 256×256. The size of the coding unit can be classified based on the length of its long side, the length of its short side, or its area. The image decoding device 100 can apply the same classification rules to coding units classified into the same group. For example, the image decoding device 100 can classify coding units with the same long side length as coding units of the same size. Furthermore, the image decoding device 100 can apply the same classification rules to coding units with the same long side length.
[0217] The width-to-height ratio of a coding unit can include 1:2, 2:1, 1:4, 4:1, 1:8, 8:1, 1:16, 16:1, 32:1, 1:32, etc. Furthermore, the orientation of the coding unit can be horizontal or vertical. A horizontal orientation indicates a coding unit whose width is longer than its height. A vertical orientation indicates a coding unit whose width is shorter than its height.
[0218] Image decoding device 100 can adaptively determine partitioning rules based on the size of the coding unit. Image decoding device 100 can also determine different permissible partitioning shape patterns based on the size of the coding unit. For example, image decoding device 100 can determine whether partitioning is permissible based on the size of the coding unit. Image decoding device 100 can determine the partitioning direction based on the size of the coding unit. Image decoding device 100 can determine the permissible partitioning type based on the size of the coding unit.
[0219] The partitioning rule determined based on the size of the coding unit can be a partitioning rule predetermined between the image encoding device and the image decoding device 100. Alternatively, the image decoding device 100 can determine the partitioning rule based on information obtained from the bitstream.
[0220] The image decoding device 100 can adaptively determine the partitioning rules based on the position of the coding units. The image decoding device 100 can adaptively determine the partitioning rules based on the position of the coding units in the image.
[0221] Furthermore, the image decoding device 100 can determine partitioning rules such that encoding units generated via different partitioning paths do not have the same block shape. However, the embodiments are not limited to this, and encoding units generated via different partitioning paths have different decoding processing orders. As already referred to above... Figure 12 The decoding process order has been described, so its details will not be provided further.
[0222] Reference Figures 17 to 28 Describes a method for determining the encoding order of the current block and applying an adaptive motion vector resolution (AMVR) mode based on the encoding order.
[0223] Figure 17 A video decoding device 1700 is shown according to an embodiment relating to dividing the current block and encoding the divided lower-level blocks.
[0224] The video decoding device 1700 includes a processor 1710 and a memory 1720. Figure 17 In this embodiment, processor 1710 and memory 1720 are represented by component units located in a single device, but processor 1710 and memory 1720 do not necessarily need to be physically adjacent to each other. Therefore, according to an embodiment, processor 1710 and memory 1720 can be distributed. Figure 17 In this context, processor 1710 is represented as a single component, but according to embodiments, multiple processors may be included in the video decoding device 1700.
[0225] Depend on Figure 17 The functions performed by the processor 1710 can be controlled by Figure 1 Decoder 120 is executed.
[0226] Processor 1710 can obtain partitioning information indicating whether to partition the current block. The partitioning information indicates whether to partition the current block into two or more smaller blocks. Furthermore, when the partitioning information indicates to partition the current block, processor 1710 partitions the current block into two or more lower-level blocks.
[0227] The current block can be divided into various shapes based on its shape. For example, if the current block is a square, it can be divided into four lower square blocks based on the current block's partitioning information.
[0228] When the shape of the current block allows for two or more partitioning methods, the processor 1710 can select a partitioning method based on the partitioning shape information. Therefore, when the partitioning information indicates that the current block should be partitioned, the processor 1710 can obtain the partitioning shape information indicating the partitioning method for the current block. Furthermore, the processor 1710 can partition the current block according to the partitioning method indicated by the partitioning shape information.
[0229] For example, when the current block is a 2N×2N square, the shape information can indicate which of the following partitioning methods should be applied to the current block: N×N partitioning, N×2N partitioning, 2N×N partitioning, vertical unequal trisectioning, and horizontal unequal trisectioning. N×N partitioning divides the current block into four N×N blocks. N×2N partitioning divides the current block into N×2N blocks. 2N×N partitioning divides the current block into 2N×N blocks. Vertical unequal trisectioning divides the 2N×2N block into three blocks of equal width and a height ratio of 1:2:1. Horizontal unequal trisectioning divides the 2N×2N block into three blocks of equal height and a width ratio of 1:2:1. Additionally, the current block can be partitioned using one of various horizontal or vertical partitioning methods.
[0230] When the current block is an N×2N rectangle that is longer vertically, the shape information indicates the appropriate partitioning method to apply to the current block, either an N×N partition or a vertical unequal trisection. An N×N partition divides the current block into two N×N blocks. A vertical unequal trisection divides the N×2N block into three blocks of equal width with a height ratio of 1:2:1. Alternatively, the current block can be partitioned using one of various horizontal or vertical partitioning methods.
[0231] When the current block is a 2N×N rectangle with a length of 2N×N in the horizontal direction, the partitioning shape information indicates the partitioning method to be applied to the current block in either N×N partitioning or unequal horizontal trisectioning. N×N partitioning divides the current block into two N×N blocks. Unequal horizontal trisectioning divides the 2N×N block into three blocks with the same height and a width ratio of 1:2:1. Alternatively, the current block can be partitioned using one of various horizontal or vertical partitioning methods.
[0232] In addition to the above methods, methods such as dividing the current block according to asymmetrical proportions, dividing it into triangles, and dividing it into other geometric shapes can also be used to divide the current block into square or rectangular blocks.
[0233] When the partitioning information does not indicate that the current block should be partitioned, the processor 1710 does not partition the current block. Furthermore, the processor 1710 decodes the current block.
[0234] When the current block is a coding unit, the processor 1710 determines the current block as the final coding unit. The final coding unit is no longer divided into coding units of greater depth. According to an embodiment, the processor 1710 may divide the current block, which is the final coding unit, into data units other than coding units.
[0235] According to an embodiment, processor 1710 may divide the current block into one or more prediction units according to a hierarchical tree structure. Similarly, processor 1710 may divide the current block into one or more transformation units according to a hierarchical tree structure. Then, processor 1710 may reconstruct the current block based on the prediction results for the prediction units and the transformation results for the transformation units.
[0236] When the current block is a prediction unit, the processor 1710 can perform prediction on the current block. When the current block is a transform unit, the processor 1710 can obtain residual data by performing dequantization and inverse transform on the quantized transform coefficients of the current block.
[0237] Processor 1710 obtains encoding order information indicating the encoding order of lower-level blocks. Furthermore, processor 1710 can determine the decoding order of lower-level blocks based on the obtained encoding order information.
[0238] Encoding order information indicates the encoding order of two or more lower-level blocks included in the current block. The amount of data in the encoding order information is determined based on the number of lower-level blocks and the method used to determine the encoding order.
[0239] For example, when there are two lower-level blocks, the encoding order information can be determined as an indication of the first lower-level block to be encoded. Therefore, the encoding order information can be in the form of a flag with 1 bit of data.
[0240] However, when there are four lower-level blocks, the number of possible encoding orders for these lower-level blocks is 4! = 24. Therefore, 5 bits of data are needed to indicate these 24 encoding orders. In other words, the number of possible encoding orders increases as the number of lower-level blocks increases. Therefore, to reduce the amount of data required for encoding order information, an encoding order determination method can be used whereby the encoding order is determined by deciding whether to swap the encoding orders of some lower-level block pairs according to a predetermined basic encoding order. The encoding order information indicating whether to swap the encoding orders of lower-level block pairs indicates whether the basic encoding order is forward or reverse.
[0241] The current frame, including the current block, is encoded and decoded according to the basic encoding order. At the same level, all blocks and pixels encoded and decoded in the current frame are also encoded and decoded according to the basic encoding order. Therefore, lower-level blocks at the same level derived from the current block are also encoded and decoded according to the basic encoding order. This is described below. Figures 18A to 18C An example of the basic encoding order is shown in the figure.
[0242] Therefore, when encoding lower-level block pairs according to the basic encoding order, it can be said that the lower-level block pairs are encoded in the forward direction. On the other hand, when encoding lower-level block pairs in the reverse order of the basic encoding order, it can be said that the lower-level block pairs are encoded in the backward direction.
[0243] For example, when two lower-level blocks are horizontally adjacent to each other and encoded in the forward direction, the encoding order information can be determined such that the left lower-level block is decoded first. On the other hand, when two horizontally adjacent lower-level blocks are encoded in the reverse direction, the encoding order information can be determined such that the right lower-level block is decoded first.
[0244] Similarly, when two lower-level blocks are vertically adjacent to each other and encoded in the forward direction, the encoding order information can be determined such that the upper lower-level block is decoded first. On the other hand, when two lower-level blocks are vertically adjacent and encoded in the reverse direction, the encoding order information can be determined such that the lower lower-level block is decoded first.
[0245] When the encoding order information only indicates the encoding order of lower-level block pairs, the encoding order information has a data size of 1 bit. Encoding order information with a data size of 1 bit can be defined as an encoding order flag.
[0246] The processor 1710 can obtain encoding order information from the bit stream. The encoding order information can be located after the partitioning information in the bit stream.
[0247] Processor 1710 can implicitly determine encoding order information based on the surrounding environment of the current block. Encoding order information can be determined based on whether neighboring blocks adjacent to the current block are encoded. For example, processor 1710 can determine that a lower-level block with many adjacent neighboring blocks is to be decoded first.
[0248] Regarding the 1710 processor, Figures 18A to 18C The basic decoding sequence according to an embodiment is shown. Figures 18A to 18C The basic encoding order is the zigzag encoding order. According to the zigzag encoding order, data units are encoded from left to right, and after all data units in the current row have been encoded, data units in the rows below the current row are encoded from left to right. This zigzag encoding order is called the raster scan order.
[0249] exist Figure 18A The diagram illustrates the encoding order based on the zigzag encoding order of the largest coding units included in the current frame 1800. The largest coding units are indexed from 0 to 15 according to the zigzag encoding order. Following the zigzag encoding order, the largest coding units in the first row, indexed from 0 to 3, are encoded first, and then the largest coding units in the second row, indexed from 4 to 7, are encoded from left to right. The largest coding units are also internally encoded according to the zigzag encoding order.
[0250] Figure 18B The encoding order of the largest coding unit 1810 with index 6 in the current frame 1800 is shown. Coding units at the final depth, divided according to the zigzag encoding order, can be indexed from 0 to 15. The zigzag encoding order is applied to data units of the same depth. Furthermore, later coding units at depth n are not encoded until all lower-level coding units of a coding unit at depth n are encoded. For example, the coding unit at index 15 is not encoded until all coding units with indices 5 to 14 are encoded. Coding units are also internally encoded according to the zigzag encoding order.
[0251] Figure 18C The reference sample is shown, which is referenced by the coding unit 1824 with index 6 in the coding units included in the largest coding unit 1810. Only the coding units 1812 with index 0 and 1822 with index 5 are reconstructed around the currently being encoded coding unit 1824 with index 6. Therefore, only pixel 1850 of coding unit 1812 and pixel 1860 of coding unit 1822 can be used as reference samples for coding unit 1824.
[0252] It can be applied in different directions depending on the data unit. Figures 18A to 18C The zigzag encoding order can be varied. For example, the zigzag encoding order can be changed to encode data cells in the same row from right to left. Alternatively, the zigzag encoding order can be changed to encode data cells in the row above the current row after all data cells in the current row have been encoded. Furthermore, the zigzag encoding order can be changed to encode data cells in the same column from top to bottom, encoding all data cells in the current column, and then encoding data cells in the column to the right of the current column.
[0253] Regarding the 1710 processor, Figure 19A and Figure 19B The examples 1900, where encoding unit 1910 is encoded in the forward direction, and 1902, where encoding unit 1920 is encoded in the reverse direction, are shown respectively. (Refer to...) Figure 19A and Figure 19B Describe the advantages achieved by changing the encoding order.
[0254] Based on the intra-frame mode along the upper right direction Figure 19A The coding unit 1910 and Figure 19B The coding unit 1920 is used for prediction. Figure 19A and Figure 19B The solid line 1930 represents pixels with constant values arranged in a straight line in the original image. Therefore, when predicting the current coding unit along the direction of the solid line 1930, the prediction accuracy of coding units 1910 and 1920 will be improved.
[0255] In the forward encoding case 1900, the left, upper, and upper right coding units of the current coding unit 1910 are reconstructed before the current coding unit 1910. Therefore, the current coding unit 1910 references the pixel or coding information of the left, upper, and upper right coding units. For example, pixel 1916 located at the lower edge of the upper right coding unit is predicted for the prediction of the current coding unit 1910. Pixel 1916 is spatially separated from the current coding unit 1910, and therefore the prediction accuracy of portion 1914 of the current coding unit 1910 may be low.
[0256] However, in case 1902, where encoding is performed in reverse, since the right, top, and top-left coding units of the current coding unit 1920 are reconstructed before the current coding unit 1920, pixel 1926 located at the left edge of the right coding unit can be used for prediction of the current coding unit 1920 in intra-frame prediction. Because pixel 1926 is adjacent to the current coding unit 1920, the prediction accuracy of a portion 1924 of the current coding unit 1920 can be higher than the prediction accuracy of a portion 1914 of the current coding unit 1910.
[0257] With reference Figure 19A and Figure 19B Similar to the intra-frame prediction implementations described, even in inter-frame prediction, there are many cases where prediction accuracy can be improved by obtaining coding information from blocks located in the reverse direction. When the current coding unit and the coding unit to its right are coding units targeting the same object, the motion information of the current coding unit and the motion information of the right coding unit can be similar. Therefore, coding efficiency can be improved by deriving the motion information of the current coding unit from the motion information of the right coding unit.
[0258] Therefore, the coding efficiency of an image can be improved by comparing the coding efficiency when encoding the current coding unit in the forward direction with the coding efficiency when encoding the current coding unit in the reverse direction to determine the coding order.
[0259] The encoding order information can be configured in the same way as the encoding order information applied to the upper-level blocks of the current block. For example, when the current block is a prediction unit or a transform unit, the processor 1710 can apply the encoding order information applied to the encoding units included in the current block to the current block. As another example, when the current block is an encoding unit, the processor 1710 can apply the encoding order information applied to encoding units with a depth lower than that of the current block to the current block.
[0260] When there are two or more encoding order flags for the current block, the processor 1710 may obtain only one encoding order flag from the bit stream and determine the remaining encoding order flags as associated with the encoding order flag obtained from the bit stream.
[0261] Regarding the determination of the encoding order by the processor 1710, Figure 20 The tree structure of the largest coding unit is shown, which describes the coding order of the largest coding unit and the coding units included in the largest coding unit.
[0262] The largest coding unit 2050 is divided into multiple coding units 2056, 2058, 2060, 2062, 2068, 2070, 2072, 2074, 2080, 2082, 2084, and 2086. The largest coding unit 2050 corresponds to the highest node 2000 in the tree structure. Furthermore, the multiple coding units 2056, 2058, 2060, 2062, 2068, 2070, 2072, 2074, 2080, 2082, 2084, and 2086 correspond to multiple nodes 2006, 2008, 2010, 2012, 2018, 2020, 2022, 2024, 2030, 2032, 2034, and 2036, respectively. In the tree structure, the upper encoding order markers 2002, 2014, and 2026, which indicate the encoding order, correspond to arrows 2052, 2064, and 2076, respectively, and the lower encoding order markers 2004, 2016, and 2028 correspond to arrows 2054, 2066, and 2078, respectively.
[0263] The top encoding order flag indicates the encoding order of the top two encoding units out of four encoding units at the same depth. When the top encoding order flag is 0, encoding is performed in the forward direction. Conversely, when the top encoding order flag is 1, encoding is performed in the reverse direction.
[0264] Similarly, the bottom encoding order flag indicates the encoding order of the two bottom encoding units out of four encoding units at the same depth. When the bottom encoding order flag is 0, encoding is performed in the forward direction. Conversely, when the bottom encoding order flag is 1, encoding is performed in the reverse direction.
[0265] For example, because the upper encoding sequence flag 2014 is 0, the encoding order between encoding units 2068 and 2070 is determined to be forward (i.e., from left to right). Furthermore, because the lower encoding sequence flag 2016 is 1, the encoding order between encoding units 2072 and 2074 is determined to be reverse (i.e., from right to left).
[0266] According to the embodiment, the upper encoding order flag and the lower encoding order flag can be set to the same value. For example, when the upper encoding order flag 2002 is determined to be 1, the lower encoding order flag 2004 corresponding to the upper encoding order flag 2002 can be determined to be 1. Because the values of the upper encoding order flag and the lower encoding order flag are determined to be 1 bit, the amount of information in the encoding order information is reduced.
[0267] According to an embodiment, the upper and lower coding order flags of the current coding unit can be determined by referring to at least one of the upper coding order flags and lower coding order flags applied to coding units with a depth lower than the current coding unit. For example, the upper coding order flag 2026 and lower coding order flag 2028 applied to coding units 2080, 2082, 2084, and 2086 can be determined based on the lower coding order flag 2016 applied to coding units 2072 and 2074. Therefore, the upper coding order flag 2026 and lower coding order flag 2028 can be determined to have the same value as the lower coding order flag 2016. Because the values of the upper and lower coding order flags are determined from the upper coding unit of the current coding unit, coding order information is not obtained from the bitstream. Therefore, the amount of coding order information is reduced.
[0268] Regarding the determination of the encoding order by processor 1710, it will refer to... Figure 21A and Figure 21B Describe how to change the encoding order of three or more blocks arranged in a vertical or horizontal direction based on an encoding order flag.
[0269] Figure 21A The embodiments relate to a method for exchanging coding order based on coding order flags only when the coding orders of spatially adjacent coding units are adjacent to each other.
[0270] Encoding unit 2100 is divided into three encoding units 2110, 2120, and 2130. When the basic encoding order is from left to right, encoding is performed in the order of encoding units 2110, 2120, and 2130. However, the encoding order can be changed according to encoding order flags 2140 and 2150.
[0271] The encoding order flag 2140 indicates the encoding order of encoding units 2110 and 2120. When the encoding order flag 2140 is 0, the encoding order of encoding units 2110 and 2120 is determined to be forward. Therefore, encoding unit 2110 is encoded before encoding unit 2120. However, when the encoding order flag 2140 is 1, the encoding order of encoding units 2110 and 2120 is determined to be reverse, so encoding unit 2120 is encoded before encoding unit 2110.
[0272] Encoding order flag 2150 indicates the encoding order of encoding units 2120 and 2130. When encoding order flag 2140 indicates a forward direction, encoding order flag 2150 is obtained. When encoding order flag 2140 indicates a reverse direction, the encoding orders of encoding units 2120 and 2130 are not adjacent, therefore encoding order flag 2150 is not obtained. When encoding order flag 2150 is 0, the encoding order of encoding units 2120 and 2130 is determined to be forward. Therefore, encoding unit 2120 is encoded before encoding unit 2130. However, when encoding order flag 2150 is 1, the encoding order of encoding units 2120 and 2130 is determined to be reverse, therefore encoding unit 2130 is encoded before encoding unit 2120.
[0273] according to Figure 21A In this embodiment, there are three possible encoding orders for the three encoding units. Therefore, one or two encoding order flags are used to determine the encoding order.
[0274] Figure 21B The embodiments relate to a method for determining the coding order based on a coding order flag 2160 indicating the direction applied to the coding order of the three coding units.
[0275] The encoding order flag 2160 indicates whether the encoding order is forward or reverse. For example, when the encoding order flag 2160 is 0, the encoding order of encoding units 2110, 2120, and 2130 can be determined to be forward. Therefore, when the encoding order flag 2160 is 0, encoding can be performed in the order of encoding units 2110, 2120, and 2130.
[0276] On the other hand, when the encoding sequence flag 2160 is 1, the encoding order of encoding units 2110, 2120 and 2130 can be determined to be in reverse. Therefore, when the encoding sequence flag 2160 is 1, encoding can be performed in the order of encoding unit 2130, encoding unit 2120 and encoding unit 2110.
[0277] according to Figure 21BIn this embodiment, there are two possibilities regarding the encoding order of the three encoding units. Therefore, an encoding order flag is used to determine the encoding order.
[0278] It can be applied to four or more coding units. Figure 21A and Figure 21B The method for determining the encoding order used in the embodiments.
[0279] Processor 1710 can identify encoding order change permission information for upper-level data units of the current block. Encoding order change permission information indicates whether the blocks included in the upper-level data units of the current block are allowed to have their encoding order changed. When the encoding order change permission information does not allow the encoding order to be changed, all blocks of the upper-level data unit are decoded according to the basic encoding order. When the encoding order change permission information indicates that the encoding order information of the current block is encoded, processor 1710 can obtain the encoding order information.
[0280] Encoding order change permission information can be included in the video parameter set, sequence parameter set, picture parameter set, stripe segment header, or maximum coding unit header. Furthermore, when two or more types of encoding order information exist, multiple encoding order change permission messages for multiple encoding order information can be stored in different header information.
[0281] The encoding order change permission information can indicate the depth or block size that provides the encoding order information. For example, processor 1710 can only obtain encoding order information if the depth of the current block is included in the depth indicated by the encoding order change permission information. As another example, processor 1710 can only obtain encoding order information if the depth of the current block is the block size indicated by the encoding order change permission information.
[0282] When the partitioning information does not indicate that the current block should be partitioned, the processor 1710 can determine the prediction method for the current block based on the encoding information of the current block and the decoding of the neighboring blocks of the current block.
[0283] The encoding information of the current block indicates how to predict the current block. Specifically, the encoding information may indicate one of several intra-frame prediction modes and inter-frame prediction modes. Therefore, the processor 1710 can determine the prediction mode to be applied to the current block based on the encoding information of the current block.
[0284] Affine patterns can be applied to the current block. Affine patterns are an inter-frame prediction method that obtains the motion vectors of samples included in the current block by performing an affine transformation on the motion vectors obtained from neighboring samples of the current block. According to an embodiment, the affine-transformed motion vectors can be obtained on a sub-block basis, including samples in the current block. References will follow below. Figure 22 Describe a method for determining the motion vector of the current block based on an affine transformation.
[0285] Processor 1710 may obtain affine inter-frame mode permission information indicating whether affine inter-frame modes are permitted for higher-level units of the current block. Additionally, processor 1710 may obtain affine merge mode permission information indicating whether affine merge modes are permitted for higher-level units of the current block. Optionally, processor 1710 may obtain affine mode permission information indicating whether all prediction modes to which affine transformations are applied are permitted for higher-level units of the current block. Processor 1710 may obtain affine type permission information indicating the permitted affine types for higher-level units of the current block. A higher-level unit may be one of a video parameter set, a sequence parameter set, a picture parameter set, or a strip header.
[0286] In affine inter-frame mode, reference frame information is obtained from the bitstream, and affine parameter sets are obtained from adjacent blocks of the current block. Furthermore, the affine parameter sets are corrected based on affine parameter correction information. The current block is then predicted based on the corrected affine parameters and the reference frame.
[0287] In affine merging mode, a reference image and an affine parameter set are obtained from the adjacent blocks of the current block. Furthermore, the affine parameter set can be used together with the reference image for the prediction of the current block without correcting the affine parameter set.
[0288] When an inter-frame mode is applied to the current block and an affine inter-frame mode is permitted for use on the current block, the processor 1710 obtains affine inter-frame information indicating whether to apply the affine inter-frame mode to the current block. When the affine inter-frame information indicates that the affine inter-frame mode should be applied to the current block, the processor 1710 decodes the current block according to the affine inter-frame mode.
[0289] When a merge mode is applied to the current block and an affine merge mode is allowed for the current block, the processor 1710 obtains affine merge information indicating whether to apply the affine merge mode to the current block. When the affine merge information indicates that the affine merge mode should be applied to the current block, the processor 1710 decodes the current block according to the affine merge mode.
[0290] Processor 1710 can obtain affine type information from the bitstream. The affine type information indicates the affine type applicable to the current block. Affine types applicable to the current block may include a 6-parameter affine mode, a 4-parameter affine mode, and a 3-parameter affine mode. Processor 1710 can select one of the allowed affine types for the current block based on the affine type licensing information. (See below for further details.) Figure 22 Describe in detail the 6-parameter affine mode, the 4-parameter affine mode, and the 3-parameter affine mode.
[0291] Processor 1710 can obtain from the bit stream affine parameter information indicating the affine parameter group of the current block in the affine parameter group candidates.
[0292] Processor 1710 can obtain a candidate affine parameter set for the current block based on whether adjacent blocks of the current block, including at least the right-hand block of the current block, have been decoded. Affine parameters are determined based on the x and y components of the motion vector. The affine parameter set indicates a set of affine parameters required to predict the current block, and the affine parameter set candidates indicate the affine parameter sets that can be applied to the current block. (See below for reference.) Figures 23 to 27 Describe the method for determining candidates for affine parameter sets.
[0293] Processor 1710 can determine candidate affine parameter sets to be used in affine mode from a plurality of sample point positions of the current block. Specifically, processor 1710 can determine the upper left coordinate of the current block as a first affine parameter extraction position and the upper right coordinate of the current block as a second affine parameter extraction position. Furthermore, processor 1710 can obtain a first motion vector from at least one adjacent block adjacent to the first affine parameter extraction position and a second motion vector from at least one adjacent block adjacent to the second affine parameter extraction position.
[0294] When neither the right nor left block of the current block has been decoded, the processor 1710 may obtain a third motion vector from the first and second motion vectors. Optionally, when the current block is in 4-parameter affine mode or 3-parameter affine mode, the processor 1710 may obtain a third motion vector from the first and second motion vectors.
[0295] When the block to the left of the current block is decoded and the block to the right of the current block is not decoded, the processor 1710 can determine the lower left coordinates of the current block as the third affine parameter extraction position. Furthermore, the processor 1710 can obtain the third motion vector from one or more adjacent blocks adjacent to the third affine parameter extraction position.
[0296] When the right block of the current block is decoded and the left block of the current block is not decoded, the processor 1710 can determine the lower right coordinate of the current block as the third affine parameter extraction position. Furthermore, the processor 1710 can obtain the third motion vector from one or more adjacent blocks adjacent to the third affine parameter extraction position.
[0297] Furthermore, the processor 1710 can obtain affine parameter set candidates based on the obtained first to third motion vectors.
[0298] When both the right and left blocks of the current block are decoded, the processor 1710 can determine the lower left coordinate of the current block as the third affine parameter extraction position and the lower right coordinate of the current block as the fourth affine parameter extraction position. The processor 1710 can obtain a third motion vector from at least one adjacent block adjacent to the third affine parameter extraction position and a fourth motion vector from at least one adjacent block adjacent to the fourth affine parameter extraction position. Furthermore, the processor 1710 can obtain at least one of the following affine parameter group candidates: a first affine parameter group candidate obtained from the first motion vector to the third motion vector, a second affine parameter group candidate obtained from the first motion vector, the second motion vector, and the fourth motion vector, and a third affine parameter group candidate determined from the first and second affine parameter group candidates.
[0299] The following will refer to Figure 23 Describe in detail the method for determining candidate affine parameter sets from the positions of several sample points in the current block.
[0300] Processor 1710 can determine affine parameter group candidates from a neighboring block decoded according to an affine pattern. According to an embodiment, processor 1710 can scan neighboring blocks reconstructed in affine pattern from the neighboring blocks of the current block according to the scan order. Furthermore, processor 1710 can determine affine parameter group candidates from neighboring blocks reconstructed in affine pattern, wherein the neighboring blocks are scanned first according to the scan order. According to an embodiment, processor 1710 can determine the scan order based on whether neighboring blocks of the current block have been decoded. Optionally, processor 1710 can determine the scan order based on the encoding order of the current block.
[0301] According to an embodiment, in order to obtain affine parameter set candidates, processor 1710 can obtain reference motion vectors from the upper left, upper right, and lower left positions of adjacent blocks. Additionally, processor 1710 can obtain the horizontal variation from the difference between the reference motion vector at the upper left position and the reference motion vector at the upper right position. Similarly, processor 1710 can obtain the vertical variation from the difference between the reference motion vector at the upper left position and the reference motion vector at the lower left position. Furthermore, processor 1710 can obtain affine parameter set candidates for the current block based on the reference motion vector at the upper left position, the horizontal variation, and the vertical variation.
[0302] The following will refer to Figure 24 This describes a method for determining candidate affine parameter sets from a neighboring block decoded according to an affine pattern.
[0303] Processor 1710 can determine affine parameter group candidates from multiple neighboring blocks decoded according to an affine mode. Processor 1710 can determine a first motion vector and a first motion vector extraction position from a first neighboring block decoded in an affine mode, wherein the first neighboring block is first discovered during a scan process that scans the neighboring blocks of the current block according to a first scan order starting from the neighboring block adjacent to the upper left coordinate of the current block. Furthermore, processor 1710 can determine a second motion vector and a second motion vector extraction position from a second neighboring block decoded in an affine mode, wherein the second neighboring block is first discovered during a scan process that scans the neighboring blocks of the current block according to a second scan order starting from the neighboring block adjacent to the upper right coordinate of the current block.
[0304] When neither the right nor left block of the current block has been decoded, the processor 1710 can determine the third motion vector and the third motion vector extraction position from the first or second adjacent block.
[0305] When the left block of the current block is decoded and the right block of the current block is not decoded, the processor 1710 can determine the third motion vector and the third motion vector extraction position from the third adjacent block decoded in affine mode, wherein the third adjacent block is first discovered in the scan process of scanning the adjacent blocks of the current block according to the third scan order starting from the adjacent block adjacent to the lower left coordinate of the current block.
[0306] When the right block of the current block is decoded and the left block of the current block is not decoded, the processor 1710 can determine the third motion vector and the third motion vector extraction position from the fourth adjacent block decoded in affine mode, wherein the fourth adjacent block is first discovered in the scan process of scanning the adjacent blocks of the current block according to the fourth scan order starting from the adjacent block adjacent to the lower right coordinate of the current block.
[0307] Processor 1710 can obtain affine parameter group candidates based on the first to third motion vectors and the first to third motion vector extraction positions of the current block. The first to fourth scan order is determined based on whether adjacent blocks of the current block have been decoded.
[0308] When both the right and left blocks of the current block are decoded, the processor 1710 may obtain at least one of the following affine parameter group candidates: a first affine parameter group candidate obtained when the left block is decoded and the right block is not decoded, a second affine parameter group candidate obtained when the right block is decoded and the left block is not decoded, and a third affine parameter group candidate obtained from the first and second affine parameter group candidates.
[0309] The following will refer to Figures 25 to 27This paper describes a method for determining candidate affine parameter groups from multiple adjacent blocks decoded according to an affine pattern.
[0310] Processor 1710 can determine one or more affine parameter group candidates according to the method described above. When there are two or more affine parameter group candidates, processor 1710 can obtain the affine parameter information of the current block from the bit stream. In addition, processor 1710 can determine the affine parameter group of the current block from multiple affine parameter group candidates based on the affine parameter information.
[0311] Processor 1710 can obtain affine parameter correction information for the current block. Furthermore, processor 1710 can correct the affine parameters included in the affine parameter group based on the affine parameter correction information. When the prediction mode for the current block is affine merging mode, the operations for obtaining the affine parameter correction information and correcting the affine parameters are omitted.
[0312] Processor 1710 can reconstruct the current block based on one or more affine parameters included in the affine parameter group.
[0313] Figure 22 This describes in detail the method for deriving the motion vectors of the sample points applied to the current block 2200 in affine mode.
[0314] In affine mode, at least three affine parameters are required to derive the motion vector of the sample points of the current block 2200. Specifically, affine modes may include a 6-parameter affine mode, a 4-parameter affine mode, and a 3-parameter affine mode. The method for deriving the motion vector of the sample points of the current block 2200 according to each affine mode will be described below.
[0315] In 6-parameter affine mode, the processor 1710 can obtain three motion vectors (i.e., first motion vector 2202 to third motion vector 2206) from neighboring sample points at the upper left coordinate 2201, upper right coordinate 2203, and lower left coordinate 2205. The first motion vector 2202 can be obtained from the neighboring sample point at the upper left coordinate 2201 of the current block 2200. Furthermore, the second motion vector 2204 can be obtained from the neighboring sample point at the upper right coordinate 2203 of the current block 2200. Furthermore, the third motion vector 2206 can be obtained from the neighboring sample point at the lower left coordinate 2205 of the current block 2200. Figure 22 In this embodiment, the third motion vector 2206 is obtained based on the lower left coordinate 2205 of the current block 2200. However, according to the embodiment, the third motion vector 2206 can be obtained based on the lower right coordinate 2207 of the current block 2200. Furthermore, the processor 1710 can determine the x and y components of the first motion vector 2202, the second motion vector 2204, and the third motion vector 2206 as affine parameters.
[0316] According to an embodiment, the first motion vector 2202 can be determined as the average of the motion vectors of a plurality of adjacent blocks adjacent to the upper-left coordinate 2201 of the current block 2200. Similarly, the second motion vector 2204 can be determined as the average of the motion vectors of a plurality of adjacent blocks adjacent to the upper-right coordinate 2203 of the current block 2200. Furthermore, the third motion vector 2206 can be determined as the average of the motion vectors of a plurality of adjacent blocks adjacent to the lower-left coordinate 2205 or the lower-right coordinate 2207 of the current block 2200.
[0317] The motion vector 2210 of the sample point 2208 of the current block 2200 can be determined based on Equations 1 to 3, according to the first motion vector 2202, the second motion vector 2204, and the third motion vector 2206.
[0318] In equations 1 to 3, x represents the horizontal distance difference between the upper left coordinate 2201 of the current block 2200 and the sample point 2208 of the current block 2200, and y represents the vertical distance difference between the upper left coordinate 2201 of the current block 2200 and the sample point 2208 of the current block 2200. MV0 represents the first motion vector 2202, MV1 represents the second motion vector 2204, and MV2 represents the third motion vector 2206. MV represents the motion vector 2210 of the sample point 2208 of the current block 2200. w represents the width of the current block 2200 and h represents the height of the current block 2200. dMV x dMV represents the horizontal rate of change of the motion vector 2210. y This represents the rate of change of the vertical direction of the motion vector 2210.
[0319] [Equation 1]
[0320] dMV x =(MV1-MV0) / w
[0321] [Equation 2]
[0322] dMV y =(MV2-MV0) / h
[0323] [Equation 3]
[0324] MV = MV0 + x.dMV x +y.dMV y
[0325] Equation 1 represents the rate of change of the horizontal velocity dMV of the motion vector 2210. x The method is as follows. According to Equation 1, the value obtained by subtracting the value of the first motion vector 2202 from the second motion vector 2204 and dividing it by the width of the current block 2200 is determined as the horizontal rate of change of motion vector 2210.
[0326] Equation 2 represents the vertical rate of change dMV of the motion vector 2210. y The method is as follows. According to Equation 2, the value obtained by subtracting the value of the first motion vector 2202 from the third motion vector 2206 and dividing it by the height of the current block 2200 is determined as the vertical change rate of motion vector 2210.
[0327] Equation 3 describes the method for obtaining the motion vector 2210. According to Equation 3, the motion vector 2210 can be determined by interpolating (x, y) with (dMV) indicating the horizontal and vertical rates of change. x dMV y The value obtained by adding the inner product of the first motion vector 2202 (MV0) and the first motion vector 2202 (MV0) is given, where (x, y) is the coordinate of the sample point 2208 of the current block 2200 relative to the upper left coordinate 2201 of the current block 2200.
[0328] According to Equations 1 to 3, the motion vectors of all sample points or sub-blocks included in the current block 2200 can be determined. According to Equations 1 to 3, the motion vectors of sample points can be determined based on their different positions. Equations 1 and 2 can be applied when the vertical components of the coordinates of the extracted first motion vector 2202 and the second motion vector 2204 are the same, and the horizontal components of the coordinates of the extracted first motion vector 2202 and the third motion vector 2206 are the same. Therefore, the following will refer to... Figure 28 A general equation is described to determine the motion vector of the current block 2200.
[0329] In 6-parameter affine mode, motion vector 2210 is determined by three motion vectors, so the reference block of the current block 2200 can be obtained by scaling, rotating and shearing the current block 2200.
[0330] In 4-parameter affine mode, processor 1710 can obtain two motion vectors (i.e., a first motion vector 2202 and a second motion vector 2204) from neighboring samples of the current block 2200. Similar to 6-parameter affine mode, the first motion vector 2202 can be obtained from neighboring samples at the upper-left coordinate 2201 of the current block 2200. Likewise, the second motion vector 2204 can be obtained from neighboring samples at the upper-right coordinate 2203 of the current block 2200. Furthermore, processor 1710 can determine the x and y components of the first motion vector 2202 and the second motion vector 2204 as affine parameters.
[0331] In 4-parameter affine mode, the third motion vector 2206 is not determined from the lower left coordinate 2205 or the lower right coordinate 2207 of the current block 2200, but is determined by combining the first motion vector 2202 and the second motion vector 2204.
[0332] Equations 4 and 5 describe the method of determining the third motion vector 2206 by combining the first motion vector 2202 and the second motion vector 2204. In Equations 4 and 5, x represents the horizontal component of the motion vector and y represents the vertical component of the motion vector. MV0 represents the first motion vector 2202, MV1 represents the second motion vector 2204, and MV2 represents the third motion vector 2206. w represents the width of the current block 2200 and h represents the height of the current block 2200.
[0333] [Equation 4]
[0334] MV2[x]=(MV1[y]-MV0[y])*w / h+MV0[x]
[0335] [Equation 5]
[0336] MV2[y]=(MV0[x]-MV1[x])*w / h+MV0[y]
[0337] According to Equation 4, the horizontal coordinate value (MV2[x]) of the third motion vector 2206 is determined as the value obtained by adding the product of the value (MV1[y]-MV0[y]) and the value (w / h) to the horizontal coordinate value (MV0[x]) of the first motion vector 2202 ((MV1[y]-MV0[y])*w / h+MV0[x]), where the value (MV1[y]-MV0[y]) is the vertical coordinate value of the second motion vector 2204 minus the vertical coordinate value of the first motion vector 2202, and the value (w / h) is the width of the current block 2200 divided by the height of the current block 2200.
[0338] According to Equation 5, the vertical coordinate value (MV2[y]) of the third motion vector 2206 is determined as the value obtained by adding the product of the value (MV0[x]-MV1[x]) and the value (w / h) to the vertical coordinate value (MV0[y]) of the first motion vector 2202 ((MV0[x]-MV1[x])*w / h+MV0[y]), where the value (MV0[x]-MV1[x]) is the horizontal coordinate value of the first motion vector 2202 minus the horizontal coordinate value of the second motion vector 2204, and the value (w / h) is the width of the current block 2200 divided by the height of the current block 2200.
[0339] In 4-parameter affine mode, the x and y components of the third motion vector 2206 are derived from the first motion vector 2202 and the second motion vector 2204. Therefore, unlike 6-parameter affine mode, in 4-parameter affine mode, a reference block for the current block 2200 can be obtained by scaling and rotating the current block 2200 based on the first motion vector 2202 and the second motion vector 2204. In other words, in 4-parameter affine mode, the current block 2200 is not clipped.
[0340] In 3-parameter affine mode, processor 1710 can obtain two motion vectors (i.e., a first motion vector 2202 and a second motion vector 2204) from neighboring samples of the current block 2200. The first motion vector 2202 can be obtained from neighboring samples at the upper-left coordinate 2201 of the current block 2200. Similarly, the second motion vector 2204 can be obtained from neighboring samples at the upper-right coordinate 2203 of the current block 2200. However, unlike 4-parameter affine mode, in 3-parameter affine mode, only the x-component or y-component is obtained from the second motion vector 2204. Therefore, processor 1710 can determine the x-component and y-component of the first motion vector 2202 and the x-component or y-component of the second motion vector 2204 as affine parameters.
[0341] When the x-component of the second motion vector 2204 is available, the y-component of the second motion vector 2204 is obtained from the y-component of the first motion vector 2202. Conversely, when the y-component of the second motion vector 2204 is available, the x-component of the second motion vector 2204 is obtained from the x-component and y-component of the first motion vector 2202 and the y-component of the second motion vector 2204. Equations 6 and 7 below respectively represent the methods for determining the y-component and x-component of the second motion vector 2204.
[0342] In Equations 6 and 7, x represents the horizontal component of the motion vector, and y represents the vertical component of the motion vector. MV0 represents the first motion vector 2202, MV1 represents the second motion vector 2204, and MV2 represents the third motion vector 2206. w represents the width of the current block 2200, and h represents the height of the current block 2200.
[0343] [Equation 6]
[0344] MV1[y] = MV0[y]
[0345] [Equation 7]
[0346] MV1[x] = sqrt(w 2 -(MV1[y]-MV0[y]) 2 )+MV0[x]-w
[0347] According to Equation 6, when only the x-component of the second motion vector 2204 is available, the processor 1710 determines that the y-component of the second motion vector 2204 is the same as the y-component of the first motion vector 2202.
[0348] According to Equation 7, when only the y component of the second motion vector 2204 is available, the processor 1710 determines the x component of the second motion vector 2204 based on the x and y components (MV0[x], MV0[y]) of the first motion vector 2202 and the y component (MV1[y]) of the second motion vector 2204.
[0349] Furthermore, in the 4-parameter affine mode, the x and y components of the third motion vector 2206 can be determined according to Equations 4 and 5. In the 3-parameter affine mode, the unavailable components of the x and y components of the second motion vector 2204 are derived from the available components of the x and y components of the first motion vector 2202 and the third motion vector 2206. Therefore, in the 3-parameter affine mode, a reference block of the current block 2200 can be obtained by scaling or rotating the current block 2200 based solely on the first motion vector 2202 and the second motion vector 2204. When the x component of the second motion vector 2204 is available, the reference block of the current block 2200 can be obtained by scaling solely on the current block 2200. On the other hand, when the y component of the second motion vector 2204 is available, the reference block of the current block can be obtained by rotating solely on the current block 2200.
[0350] Figure 23 This is an embodiment used to describe the determination of affine parameter set candidates from several sample point positions of the current block.
[0351] according to Figure 23 In one embodiment, three motion vectors are obtained from the current block in 6-parameter affine mode. According to this embodiment, to obtain the three motion vectors, it is first determined whether the left and right blocks of the current block have been decoded. Figure 23 The method for determining affine parameters will be described sequentially when 1) only the left block of the current block is decoded, 2) only the right block of the current block is decoded, 3) neither the left nor the right block of the current block is decoded, and 4) both the left and right blocks of the current block are decoded.
[0352] When only the left block of the current block 2300 is decoded, a first motion vector 2302 is obtained from the adjacent sample 2304 at the upper left coordinate of the current block 2300. According to an embodiment, the first motion vector 2302 can be determined as the motion vector of a block corresponding to one of the adjacent samples 2304. Furthermore, the adjacent samples 2304 can be scanned in a specific order, and when an inter-frame predicted adjacent block is found, the scanning stops and the first motion vector 2302 is extracted from the inter-frame predicted adjacent block. Furthermore, according to an embodiment, the first motion vector 2302 can be determined as the average value of the motion vectors obtained from multiple blocks corresponding to the adjacent samples 2304.
[0353] Furthermore, a second motion vector 2306 is obtained from the adjacent sample point 2308 at the upper right coordinate of the current block 2300. According to an embodiment, the second motion vector 2306 can be determined as the motion vector of a block corresponding to one of the adjacent sample points 2308. Furthermore, the adjacent sample points 2308 can be scanned in a specific order, and when an inter-frame predicted adjacent block is found, the scanning stops and the second motion vector 2306 is extracted from the inter-frame predicted adjacent block. Furthermore, according to an embodiment, the second motion vector 2306 can be determined as the average value of motion vectors obtained from multiple blocks corresponding to the adjacent sample points 2308.
[0354] Furthermore, a third motion vector 2310 is obtained from adjacent sample points 2312 at the lower left coordinates of the current block 2300. According to an embodiment, the third motion vector 2310 can be determined as the motion vector of a block corresponding to one of the adjacent sample points 2312. Furthermore, adjacent sample points 2312 can be scanned in a specific order, and when an inter-frame predicted adjacent block is found, the scanning stops and the third motion vector 2310 is extracted from the inter-frame predicted adjacent block. Furthermore, according to an embodiment, the third motion vector 2310 can be determined as the average value of motion vectors obtained from multiple blocks corresponding to adjacent sample points 2312.
[0355] Furthermore, the horizontal rate of change can be determined as a value obtained by dividing the difference between the first motion vector 2303 and the second motion vector 2306 by the width of the current block 2300. Furthermore, the vertical rate of change can be determined as a value obtained by dividing the difference between the first motion vector 2302 and the third motion vector 2310 by the height of the current block 2300.
[0356] When only the right-hand block of the current block 2320 is decoded, the first motion vector 2322 is obtained from the adjacent sample point 2324 at the upper-left coordinate of the current block 2320, as is the case when only the left-hand block of the current block 2300 is decoded. Furthermore, the second motion vector 2326 is obtained from the adjacent sample point 2328 at the upper-right coordinate of the current block 2320.
[0357] However, since the blocks to the left of the current block 2320 have not been decoded, a third motion vector 2330 is obtained from the adjacent sample point 2332 at the lower right coordinate of the current block 2320. According to an embodiment, the third motion vector 2330 can be determined as the motion vector of a block corresponding to one of the adjacent sample points 2332. Furthermore, according to an embodiment, the third motion vector 2330 can be determined as the average value of the motion vectors obtained from multiple blocks corresponding to the adjacent sample points 2332.
[0358] The horizontal rate of change can be determined as a value obtained by dividing the difference between the first motion vector 2322 and the second motion vector 2326 by the width of the current block 2320. Furthermore, the vertical rate of change can be determined as a value obtained by dividing the difference between the second motion vector 2326 and the third motion vector 2330 by the height of the current block 2320.
[0359] When neither the left nor right block of the current block 2340 has been decoded, the first motion vector 2342 is obtained from the adjacent sample point 2344 at the upper left coordinate of the current block 2340, as is the case when only the left block of the current block 2300 has been decoded. Furthermore, the second motion vector 2346 is obtained from the adjacent sample point 2348 at the upper right coordinate of the current block 2340.
[0360] However, since neither the left nor right block of the current block 2340 has been decoded, the third motion vector is determined from the first motion vector 2342 and the second motion vector 2346. Therefore, when the 6-parameter affine mode is applied to the current block 2340 and neither the left nor right block of the current block 2340 has been decoded, the current block 2340 is essentially decoded according to the 4-parameter affine mode.
[0361] When both the left and right blocks of the current block 2360 are decoded, one of the following methods can be selected: determining the motion vector when only the left block of the current block 2300 is decoded, or determining the motion vector when only the right block of the current block 2320 is decoded. Therefore, a first motion vector 2362 and a second motion vector 2366 can be determined based on adjacent sample points 2364 at the upper left coordinate of the current block 2360 and adjacent sample points 2368 at the upper right coordinate of the current block 2360, respectively. Furthermore, a third motion vector can be determined based on either the lower left motion vector 2370 or the lower right motion vector 2374. The lower left motion vector 2370 is determined from adjacent sample point 2372 at the lower left coordinate of the current block 2360, and the lower right motion vector 2374 is determined from adjacent sample point 2376 at the lower right coordinate of the current block 2360.
[0362] The method for determining motion vectors can be determined from a higher level than the current block 2360. For example, the processor 1710 can determine a default motion vector determination method for a higher level than the current block 2360. When the default motion vector determination method is the method of determining motion vectors when only the left-hand block of the current block 2300 is decoded, a third motion vector can be determined based on the motion vector 2370 at the lower left of the current block 2300.
[0363] Optionally, the processor 1710 may obtain motion vector information for the current block 2360, wherein the motion vector information indicates one of a method for determining the motion vector when only the left block of the current block 2300 is decoded and a method for determining the motion vector when only the right block of the current block 2320 is decoded. The method for determining the motion vector for the current block 2360 can then be selected based on the motion vector information.
[0364] Figure 24 This is an embodiment used to describe determining a candidate set of affine parameters from a neighboring block decoded according to an affine pattern.
[0365] According to an embodiment, the processor 1710 can scan the adjacent samples 2410 of the current block 2400. The processor 1710 can identify whether the adjacent blocks corresponding to the adjacent samples 2410 are decoded according to an affine pattern. When an adjacent block 2420 decoded according to an affine pattern is found, the scanning of the adjacent samples 2410 is stopped, and the motion vector of the current block 2400 is derived from the reference motion vector obtained from the reference motion vector positions (i.e., the first reference motion vector positions 2422 to the third reference motion vector positions 2426) of the adjacent block 2420 decoded according to the affine pattern.
[0366] Specifically, the horizontal rate of change is determined based on the difference between the first reference motion vector at the first reference motion vector position 2422 and the second reference motion vector at the second reference motion vector position 2424, and the horizontal distance between the first reference motion vector position 2422 and the second reference motion vector position 2424. Furthermore, the vertical rate of change is determined based on the difference between the first reference motion vector and the third reference motion vector at the third reference motion vector position 2426, and the vertical distance between the first reference motion vector position 2422 and the third reference motion vector position 2426.
[0367] Then, the motion vector of the current block 2400 is derived based on the first reference motion vector, the horizontal rate of change, the vertical rate of change, and the difference between the position of the first reference motion vector 2422 and the upper left, upper right, and lower left positions of the current block 2400.
[0368] The order in which adjacent samples 2410 are scanned can be determined based on whether adjacent blocks of the current block 2420 have been decoded. When the left block of the current block 2400 is decoded, adjacent samples 2410 are scanned in the upward direction from the lower left adjacent sample to the upper left adjacent sample of the current block 2400, in the right direction from the upper left adjacent sample to the upper right adjacent sample of the current block 2400, and in the downward direction from the upper right adjacent sample to the lower right adjacent sample of the current block 2400. On the other hand, when the left block of the current block 2400 has not been decoded and the right block is decoded, adjacent samples 2410 are scanned in the upward direction from the lower right adjacent sample to the upper right adjacent sample of the current block 2400, in the left direction from the upper right adjacent sample to the upper left adjacent sample of the current block 2400, and in the downward direction from the upper left adjacent sample to the lower left adjacent sample of the current block 2400.
[0369] Furthermore, when the prediction mode of the current block 2400 is affine merging mode, the processor 1710 can obtain the reference screen index indicating the reference screen from the decoded adjacent block 2420.
[0370] Figure 25 This is an embodiment used to describe determining candidates for an affine parameter group from multiple adjacent blocks decoded according to an affine pattern when the left block of the current block 2500 is decoded and the right block is not decoded.
[0371] According to an embodiment, processor 1710 can scan adjacent samples of the current block 2500 according to three types of scan sequences. First, processor 1710 can scan upwards from the lower left adjacent sample 2502 to the upper left adjacent sample 2504 of the current block 2500. Furthermore, processor 1710 can scan to the right from the upper left adjacent sample 2504 to the upper right adjacent sample 2506 of the current block 2500. Processor 1710 can obtain a first motion vector from the lower left coordinate 2512 of the block 2510 that was decoded in affine mode and first discovered during the first scan process.
[0372] Next, the processor 1710 can scan the upper and left coordinates of the current block 2500 in a zigzag pattern, starting from the upper left adjacent sample 2504, in the order of the samples closest to the upper left of the current block 2500. Then, the processor 1710 can obtain the second motion vector from the upper left coordinate 2522 of the block 2520 that was decoded in affine mode and first discovered during the second scan process.
[0373] Third, the processor 1710 can scan in a leftward direction from the upper right adjacent sample 2506 to the upper left adjacent sample 2504 of the current block 2500. Then, the processor 1710 can scan in a downward direction from the upper left adjacent sample 2504 to the lower left adjacent sample 2502 of the current block 2500. The processor 1710 can obtain the third motion vector from the upper right coordinate 2532 of the block 2530 that was decoded in affine mode and first discovered during the third scan process.
[0374] The motion vector of the current block 2500 can be determined by inferring from the obtained first to third motion vectors.
[0375] Figure 26 This is an embodiment used to describe determining the motion vector of the current block used in the affine mode from multiple adjacent blocks decoded according to the affine mode when the right block of the current block is decoded and the left block is not decoded.
[0376] According to an embodiment, processor 1710 can scan adjacent samples of the current block 2600 according to three types of scan sequences. First, processor 1710 can scan upwards from the lower right adjacent sample 2602 to the upper right adjacent sample 2604 of the current block 2600. Furthermore, processor 1710 can scan to the left from the upper right adjacent sample 2604 to the upper left adjacent sample 2606 of the current block 2600. Processor 1710 can obtain a first motion vector from the lower right coordinate 2612 of the block 2610 that was decoded in affine mode and first discovered during the first scan process.
[0377] Next, the processor 1710 can scan the upper and right coordinates of the current block 2600 in a zigzag pattern, starting from the upper right adjacent sample 2604, in the order of the adjacent sample 2604 closest to the current block 2600. Then, the processor 1710 can obtain the second motion vector from the upper right coordinate 2622 of the block 2620 that was decoded in affine mode and first discovered during the second scan process.
[0378] Third, the processor 1710 can scan in a rightward direction from the upper-left adjacent sample 2606 to the upper-right adjacent sample 2604 of the current block 2600. Furthermore, the processor 1710 can scan in a downward direction from the upper-right adjacent sample 2604 to the lower-right adjacent sample 2602 of the current block 2600. The processor 1710 can obtain the third motion vector from the upper-left coordinate 2632 of the block 2630, which was decoded in affine mode and first discovered during the third scan process.
[0379] The motion vector of the current block 2600 can be determined by inferring from the obtained first to third motion vectors.
[0380] Figure 27 This is an embodiment used to describe determining the motion vector of the current block, which is used in the affine mode, from multiple adjacent blocks decoded according to the affine mode when the right and left blocks of the current block have not been decoded.
[0381] According to an embodiment, the processor 1710 can scan adjacent samples of the current block 2700 according to two types of scan order.
[0382] First, the processor 1710 can scan in a rightward direction from the upper left adjacent sample 2702 to the upper right adjacent sample 2704 of the current block 2700. The processor 1710 can obtain a first motion vector from the upper left coordinate 2712 of the block 2710 that is decoded in affine mode and first discovered during the first scan process.
[0383] Secondly, the processor 1710 can scan in a leftward direction from the upper right adjacent sample 2704 to the upper left adjacent sample 2702 of the current block 2700. The processor 1710 can obtain the second motion vector from the upper right coordinate 2722 of the block 2720 that was decoded in affine mode and first discovered during the second scan process.
[0384] Third, the processor 1710 can obtain the third motion vector from the lower left coordinate 2714 of the first scan processing block 2710. According to another embodiment, the processor 1710 can obtain the third motion vector from the lower right coordinate 2724 of the second scan processing block 2720.
[0385] Reference Figure 28 Detailed description based on Figures 25 to 27 The method of determining the motion vector of the current block by identifying the first to third motion vectors.
[0386] exist Figures 23 to 27 In this context, when the affine type of the current block is a 4-parameter affine mode or a 3-parameter affine mode, the generation of the third motion vector of the affine parameter group candidate can be omitted. Furthermore, when the affine type of the current block is a 3-parameter affine mode, the generation of the x-component or y-component of the second motion vector of the affine parameter group candidate can be omitted. When the affine type of the current block is a 4-parameter affine mode or a 3-parameter affine mode, the decoding of the left and right blocks of the current block is not determined, and the affine parameter group candidate of the current block can be generated solely based on the upper adjacent sample, the upper left adjacent sample, and the upper right adjacent sample of the current block.
[0387] References can be used in affine merge mode or affine inter-frame mode. Figures 23 to 27 Various affine parameter group candidates are described. Furthermore, the methods for determining affine parameter group candidates can differ in affine merging mode and affine inter-frame mode.
[0388] Figure 28 This describes a method for determining the motion vector of the current block in affine mode. Equations 8 to 10 below represent the extraction of position based on the motion vector (i.e., Figure 28 The method for determining the motion vector of the current block is from the first motion vector extraction position to the third motion vector extraction positions 2800, 2810 and 2820.
[0389] In Equations 8 and 9, w represents the horizontal distance between the first motion vector extraction position 2800 and the second motion vector extraction position 2810. Furthermore, h represents the vertical distance between the first motion vector extraction position 2800 and the third motion vector extraction position 2820. Furthermore, x represents the horizontal distance between the first motion vector extraction position 2800 and the third motion vector extraction position 2820. Furthermore, y represents the vertical distance between the first motion vector extraction position 2800 and the second motion vector extraction position 2810.
[0390] P0 represents the first motion vector, P1 represents the second motion vector, and P2 represents the third motion vector. Then, dx and dy represent the horizontal and vertical rates of change, respectively.
[0391] [Equation 8]
[0392]
[0393] [Equation 9]
[0394]
[0395] The horizontal rate of change is determined according to Equation 8, and the vertical rate of change is determined according to Equation 9. Then, the motion vector of sample point 2830 of the current block is determined according to Equation 10, based on the horizontal and vertical rates of change. In Equation 10, P... a This represents the motion vector of sample point 2830 in the current block. Furthermore, i represents the horizontal distance between the first motion vector extraction position 2800 and sample point 2830 in the current block, and j represents the vertical distance between the first motion vector extraction position 2800 and sample point 2830 in the current block.
[0396] [Equation 10]
[0397] P a =P0+idx+jdy
[0398] When three motion vectors and the extraction position of each motion vector are provided according to Equations 8 to 10, the motion vectors of the sample points included in the current block can be determined. Therefore, even when such... Figures 25 to 28 Even when the extraction positions of the motion vectors in the block are not aligned, the motion vectors of the sample points included in the current block can still be determined.
[0399] Figure 29 This illustrates a video decoding method 2900 based on an embodiment relating to dividing the current block and decoding the divided lower-level blocks.
[0400] In operation 2910, the candidate affine parameter group of the current block is obtained based on whether the adjacent blocks of the current block, including at least the right-hand block of the current block, are decoded.
[0401] According to an embodiment, to obtain affine parameter group candidates, neighboring blocks reconstructed in affine mode from the adjacent blocks of the current block can be scanned according to the scan order. Furthermore, affine parameter group candidates can be determined from the neighboring blocks reconstructed in affine mode, wherein these neighboring blocks are scanned first according to the scan order. Scan order information can be determined based on whether the neighboring blocks of the current block have been decoded.
[0402] According to an embodiment, in order to obtain affine parameter set candidates, the upper left coordinate of the current block can be determined as the first affine parameter extraction position, and the upper right coordinate of the current block can be determined as the second affine parameter extraction position. A first motion vector can be obtained from at least one adjacent block adjacent to the first affine parameter extraction position, and a second motion vector can be obtained from at least one adjacent block adjacent to the second affine parameter extraction position.
[0403] When neither the right nor left block of the current block has been decoded, a third motion vector can be obtained from the first and second motion vectors. Furthermore, when the left block of the current block has been decoded but the right block has not, a third motion vector can be obtained from the lower left coordinate of the current block. Additionally, when the right block of the current block has been decoded but the left block has not, a third motion vector can be obtained from the lower right coordinate of the current block.
[0404] Furthermore, affine parameter set candidates can be obtained based on the first to third motion vectors.
[0405] When both the right and left blocks of the current block are decoded, a third motion vector is obtained from the lower left coordinate of the current block, and a fourth motion vector is obtained from the lower right coordinate of the current block. Then, at least one of the following affine parameter group candidates can be determined: a first affine parameter group candidate obtained from the first to the third motion vectors, a second affine parameter group candidate obtained from the first, second, and fourth motion vectors, and a third affine parameter group candidate determined from the first and second affine parameter group candidates.
[0406] According to an embodiment, in order to obtain affine parameter set candidates, a first motion vector and a first motion vector extraction position can be determined from a first neighboring block decoded in affine mode, wherein the first neighboring block is first discovered during a scan process that scans the neighboring blocks of the current block according to a first scan order starting from the neighboring block adjacent to the upper left coordinate of the current block. A second motion vector and a second motion vector extraction position can be determined from a second neighboring block decoded in affine mode, wherein the second neighboring block is first discovered during a scan process that scans the neighboring blocks of the current block according to a second scan order starting from the neighboring block adjacent to the upper right coordinate of the current block.
[0407] When neither the right nor left block of the current block has been decoded, the third motion vector and its extraction position can be determined from the first or second adjacent block. When the left block of the current block has been decoded but the right block has not, the third motion vector and its extraction position can be determined from the third adjacent block decoded in affine mode, wherein the third adjacent block is first discovered during the scanning process that scans the adjacent blocks of the current block according to the third scan order starting from the adjacent block adjacent to the lower left coordinate of the current block. When the right block of the current block has been decoded but the left block has not, the third motion vector and its extraction position can be determined from the fourth adjacent block decoded in affine mode, wherein the fourth adjacent block is first discovered during the scanning process that scans the adjacent blocks of the current block according to the fourth scan order starting from the adjacent block adjacent to the lower right coordinate of the current block.
[0408] Affine parameter set candidates can be obtained based on the first motion vector to the third motion vector and the extraction position of the first motion vector to the extraction position of the third motion vector.
[0409] When both the right and left blocks of the current block are decoded, at least one of the following affine parameter group candidates can be obtained: a first affine parameter group candidate obtained when the left block is decoded and the right block is not decoded, a second affine parameter group candidate obtained when the right block is decoded and the left block is not decoded, and a third affine parameter group candidate obtained from the first and second affine parameter group candidates.
[0410] In operation 2920, based on the affine parameter information of the current block, the affine parameter group of the current block is determined from the affine parameter group candidates.
[0411] According to the embodiment, affine parameter correction information for the current block can be obtained. Furthermore, the affine parameters included in the affine parameter group can be corrected based on the affine parameter correction information.
[0412] In operation 2930, the current block is reconstructed based on one or more affine parameters included in the affine parameter group.
[0413] The video decoding method 2900 may include references Figure 17 The functions of the video decoding device 1700 are described.
[0414] Figure 30 This illustrates a video encoding device 3000 according to an embodiment relating to dividing the current block and encoding the divided lower-level blocks.
[0415] The video encoding device 3000 includes a processor 3010 and a memory 3020. Figure 30 In this embodiment, processor 3010 and memory 3020 are represented by component units located within a single device, but processor 3010 and memory 3020 do not necessarily need to be physically adjacent to each other. Therefore, according to the embodiment, processor 3010 and memory 3020 can be distributed. Figure 30 In this context, processor 3010 is represented as a single component, but according to embodiments, multiple processors may be included in the video encoding device 3000.
[0416] The processor 3010 may divide the current block into two or more lower-level blocks and determine whether to divide the current block based on the result of the division. For example, the processor 3010 may determine to divide the current block if the coding efficiency when dividing the current block meets the requirements, and the processor 3010 may determine not to divide the current block if the coding efficiency when not dividing the current block meets the requirements.
[0417] The processor 3010 can generate partitioning information indicating whether the current block should be partitioned. Then, the processor 3010 can determine the partitioning method of the current block based on coding efficiency and generate partitioning shape information indicating the partitioning method of the current block.
[0418] The processor 3010 can determine the encoding order of lower-level blocks included in the current block based on encoding efficiency according to the encoding order, and generate encoding order information indicating the encoding order of the lower-level blocks. The processor 3010 can determine the encoding order by comparing the encoding efficiency when encoding the current coding unit in the forward direction with the encoding efficiency when encoding the current coding unit in the reverse direction, thereby improving the encoding efficiency of the image.
[0419] The processor 3010 can determine the prediction mode of the next lower-level block when completing the partitioning of the current block. The processor 3010 can determine the prediction mode of the next lower-level block based on the coding efficiency of the prediction modes that can be applied to the next lower-level block. The prediction modes that can be applied to the next lower-level block include affine inter-frame mode and affine merge mode.
[0420] When the current block has not been partitioned, the processor 3010 can determine whether an affine inter-frame mode or an affine merging mode is applied to the current block. Furthermore, the processor 3010 can determine candidate affine parameter sets for the current block from its neighboring blocks based on whether neighboring blocks have been encoded. Additionally, the processor 3010 can determine the optimal affine parameter set for predicting the current block, and determine the affine inter-frame information, affine merging information, affine parameter information, and affine type information for the current block.
[0421] The processor 3010 outputs a bitstream that includes information about the encoding of the current block. Therefore, the processor 3010 can output a bitstream that includes affine inter-frame information, affine merging information, affine parameter information, and affine type information.
[0422] Figure 30 The video encoding device 3000 can perform and is controlled by Figure 17 The video decoding method and corresponding video encoding method executed by the video decoding device 1700.
[0423] Figure 31 This illustrates a video coding method 3100 based on an embodiment relating to dividing the current block and encoding the divided lower-level blocks.
[0424] In operation 3110, the affine parameter group candidates for the current block are determined based on whether the adjacent blocks of the current block, including at least the right-hand block of the current block, have been decoded.
[0425] In operation 3120, the optimal set of affine parameters for predicting the current block can be determined.
[0426] In operation 3130, the affine parameter information of the current block, which indicates the affine parameter group of the current block, can be determined.
[0427] Reference Figure 30 The functions of the described video encoding device 3000 can be included in the video encoding method 3100.
[0428] According to reference Figures 1 to 31 The described video coding technique, based on a tree-structured coding unit, encodes image data in the coding spatial domain for each coding unit in the tree structure. When decoding is performed for each maximum coding unit according to the video decoding technique based on the tree-structured coding unit, the image data in the spatial domain is reconstructed, and thus the images and video as a sequence of images can be reconstructed. The reconstructed video can be reproduced by a playback device, stored in a storage medium, or transmitted via a network.
[0429] Furthermore, the embodiments described above can be written as programs executable on a computer and can be implemented in a general-purpose digital computer that runs the program using a computer-readable recording medium.
[0430] Although this disclosure has been described in conjunction with specific embodiments thereof, other alternative forms, modifications, and variations to the application of the invention will be apparent to those skilled in the art from the foregoing description. In other words, the claims are intended to cover all such alternative forms, modifications, and variations of the invention. Therefore, everything described in this specification and drawings should be interpreted in an illustrative and non-limiting sense.
Claims
1. A video decoding method, comprising: When performing inter-frame prediction for the current block in affine inter-frame mode: Affine type information is obtained from the bitstream, which indicates whether a 4-parameter affine type or a 6-parameter affine type is used. When using a 4-parameter affine type based on the affine type information, the affine motion vectors of the sub-blocks of the current block are obtained using the motion vectors determined from the two corners of the current block. as well as When using a 6-parameter affine type based on the affine type information, the affine motion vectors of the sub-blocks of the current block are obtained using the motion vectors determined from the three corners of the current block. as well as When performing inter-frame prediction for the current block in affine merging mode: Based on whether the control point motion vectors determined from multiple corners of the current block are available, an affine merging candidate list including multiple affine merging candidates is generated. as well as Using the motion vector of the control point determined from two or three corners of the current block corresponding to one of the affine merging candidates in the affine merging candidate list, obtain the affine motion vector of the sub-block of the current block; as well as The predicted sample points of the current block are obtained by using the affine motion vectors of the sub-blocks of the current block. The affine merge candidate list includes affine merge candidates determined using the motion vectors of the blocks adjacent to the lower left corner of the current block and the motion vectors of the blocks adjacent to the lower right corner of the current block.
2. A video decoding device, comprising: The processor is configured as follows: When performing inter-frame prediction for the current block in affine inter-frame mode: Affine type information is obtained from the bitstream, which indicates whether a 4-parameter affine type or a 6-parameter affine type is used. When using a 4-parameter affine type based on the affine type information, the affine motion vectors of the sub-blocks of the current block are obtained using the motion vectors determined from the two corners of the current block. as well as When using a 6-parameter affine type based on the affine type information, the affine motion vectors of the sub-blocks of the current block are obtained using the motion vectors determined from the three corners of the current block. as well as When performing inter-frame prediction for the current block in affine merging mode: Based on whether the control point motion vectors determined from multiple corners of the current block are available, an affine merging candidate list including multiple affine merging candidates is generated. as well as Using the motion vector of the control point determined from two or three corners of the current block corresponding to one of the affine merging candidates in the affine merging candidate list, obtain the affine motion vector of the sub-block of the current block; as well as The predicted sample points of the current block are obtained by using the affine motion vectors of the sub-blocks of the current block. The affine merge candidate list includes affine merge candidates determined using the motion vectors of the blocks adjacent to the lower left corner of the current block and the motion vectors of the blocks adjacent to the lower right corner of the current block.
3. A video encoding method, comprising: When performing inter-frame prediction for the current block in affine inter-frame mode: When using the 4-parameter affine type, the affine motion vectors of the child blocks of the current block are obtained using the motion vectors determined from the two corners of the current block; When using the 6-parameter affine type, the affine motion vectors of the child blocks of the current block are obtained using the motion vectors determined from the three corners of the current block; as well as The affine type information for the current block is encoded into a bitstream, wherein the affine type information indicates whether a 4-parameter affine type or a 6-parameter affine type is used; and When performing inter-frame prediction for the current block in affine merging mode: Based on the availability of control point motion vectors determined from multiple angles of the current block, an affine merging candidate list including multiple affine merging candidates is generated; and Using the motion vectors of control points determined from two or three corners of the current block corresponding to one of the affine merging candidate lists, obtain the affine motion vectors of the sub-blocks of the current block; and The predicted sample points of the current block are obtained by using the affine motion vectors of the sub-blocks of the current block. The affine merge candidate list includes affine merge candidates determined using the motion vectors of the blocks adjacent to the lower left corner of the current block and the motion vectors of the blocks adjacent to the lower right corner of the current block.
4. A method for storing a bitstream generated by encoding video, characterized in that: The bitstream is generated by performing the following video encoding method, the video encoding method comprising: When performing inter-frame prediction for the current block in affine inter-frame mode: When using the 4-parameter affine type, the affine motion vectors of the child blocks of the current block are obtained using the motion vectors determined from the two corners of the current block; When using the 6-parameter affine type, the affine motion vectors of the child blocks of the current block are obtained using the motion vectors determined from the three corners of the current block; and Generate affine type information for the current block, wherein the affine type information indicates whether to use a 4-parameter affine type or a 6-parameter affine type; and When performing inter-frame prediction for the current block in affine merging mode: Based on the availability of control point motion vectors determined from multiple angles of the current block, an affine merging candidate list including multiple affine merging candidates is generated; and Using the motion vectors of control points determined from two or three corners of the current block corresponding to one of the affine merging candidate lists, obtain the affine motion vectors of the sub-blocks of the current block; and The predicted samples of the current block are obtained by using the affine motion vectors of the sub-blocks of the current block; and The bitstream is stored in a storage medium, and the bitstream includes the affine type information, information about the affine motion vector, and residual data between the predicted samples and the original samples of the current block. The affine merge candidate list includes affine merge candidates determined using the motion vectors of the blocks adjacent to the lower left corner of the current block and the motion vectors of the blocks adjacent to the lower right corner of the current block.
Citation Information
Patent Citations
Coding method and device, and decoding method and device
CN111183642B
