Image decoding / encoding methods and bitstream transmission methods

Abstract

A method for decoding / encoding an image and a method for transmitting a bitstream are disclosed. The image decoding method according to one embodiment of this disclosure includes: a step of decoding encoding mode information of a current encoded block; a step of dividing the current encoded block into at least one prediction block; and a step of generating prediction samples of the at least one prediction block based on the decoded encoding mode information; wherein the encoding mode information may be information indicating one of an intra-frame mode, an inter-frame mode, and a hybrid mode.

Description

[0001] This application is a divisional application of the invention patent application filed on January 16, 2018, with application number 201880007072.5 and title "Image Signal Encoding / Decoding Method and Apparatus". Technical Field

[0002] This invention relates to an image signal encoding / decoding method and apparatus, and more particularly to an image signal encoding / decoding method and apparatus applicable to a hybrid mode as an encoding mode. Background Technology

[0003] In recent years, the demand for multimedia data such as video on the Internet has been increasing dramatically. However, the current development speed of channel bandwidth is insufficient to fully meet the rapidly increasing volume of multimedia data. To address this, the Video Coding Expert Group (VCEG) of the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) and the Moving Picture Expert Group (MPEG) of the International Organization for Standardization / International Electrotechnical Commission (ISO / IEC) developed the first version of the video compression standard, High Efficiency Video Coding (HEVC), in February 2014.

[0004] High Efficiency Video Coding (HEVC) defines various techniques such as intra-frame prediction, inter-frame prediction, transform, quantization, entropy coding, and loop filtering.

[0005] The current video compression standard, High Efficiency Video Coding (HEVC), can only perform one type of prediction (intra-frame prediction or inter-frame prediction) within a coding block (or coding unit), which leads to a decrease in prediction accuracy.

[0006] Furthermore, in High Efficiency Video Coding (HEVC), the inter-frame prediction mode can only execute Advanced Motion Vector Prediction (AMVP), Merge, or Skip modes, which leads to a decrease in prediction accuracy.

[0007] Furthermore, in High Efficiency Video Coding (HEVC), the in-frame prediction mode can only execute Planar, DC, or Angular modes, which leads to a decrease in prediction accuracy. Summary of the Invention

[0008] Technical issues

[0009] The main objective of this invention is to solve the existing problems described above by providing a method for performing prediction using two or more prediction methods within a coded block.

[0010] Furthermore, the main objective is to provide a method for determining the inter-frame prediction mode based on the positional characteristics of the prediction blocks within the coded blocks when performing inter-frame prediction.

[0011] Furthermore, the main objective is to provide a method for determining the in-frame prediction mode based on the positional characteristics of the prediction blocks within the coded blocks when performing in-frame prediction.

[0012] The technical issues to be addressed in this disclosure are not limited to those mentioned above. Those skilled in the art to which this disclosure pertains will be able to further understand other technical issues not mentioned through the following description.

[0013] Technical solution

[0014] An image decoding method applicable to any form of this disclosure may include the following steps: deriving an initial motion vector of a current block from a merge candidate list comprising multiple merge candidates, the initial motion vector including an L0 initial motion vector and an L1 initial motion vector; deriving a final motion vector of the current block by modifying the initial motion vector of the current block based on a predetermined offset vector, the final motion vector including an L0 final motion vector and an L1 final motion vector, the predetermined offset vector being obtained based on the position with the smallest difference between an L0 block in an L0 reference image and an L1 block in an L1 reference image; and generating a predicted block of the current block based on the final motion vector of the current block, wherein the merge candidate list includes spatial merge candidates derived from spatially adjacent blocks and temporal merge candidates derived from temporally juxtaposed blocks of the current block, the current block being obtained by segmenting the current image based on segmentation information obtained from a bitstream, and the multiple merge candidates being distinct from each other.

[0015] An image coding method applicable to any form of this disclosure may include the following steps: deriving an initial motion vector of a current block from a merge candidate list comprising multiple merge candidates, the initial motion vector including an L0 initial motion vector and an L1 initial motion vector; deriving a final motion vector of the current block by modifying the initial motion vector of the current block based on a predetermined offset vector, the final motion vector including an L0 final motion vector and an L1 final motion vector, the predetermined offset vector being obtained based on the position with the smallest difference between an L0 block in an L0 reference image and an L1 block in an L1 reference image; and generating a predicted block of the current block based on the final motion vector of the current block, wherein the merge candidate list includes spatial merge candidates derived from spatially adjacent blocks and temporal merge candidates derived from temporally juxtaposed blocks of the current block, the current block being obtained by segmenting the current image, segmentation information of the current block being obtained from a bitstream, and the multiple merge candidates being distinct from each other.

[0016] A method for transmitting a bitstream generated by an image coding method, applicable to any form of this disclosure, wherein the image coding method may include the following steps: deriving an initial motion vector of a current block from a merge candidate list including a plurality of merge candidates, the initial motion vector including an L0 initial motion vector and an L1 initial motion vector; deriving a final motion vector of the current block by modifying the initial motion vector of the current block based on a predetermined offset vector, the final motion vector including an L0 final motion vector and an L1 final motion vector, the predetermined offset vector being obtained based on the position with the smallest difference between an L0 block in an L0 reference image and an L1 block in an L1 reference image; and generating a predicted block of the current block based on the final motion vector of the current block, wherein the merge candidate list includes spatial merge candidates derived from spatially adjacent blocks and temporal merge candidates derived from temporally juxtaposed blocks of the current block, the current block being obtained by segmenting a current image, segmentation information of the current block being obtained from the bitstream, and the plurality of merge candidates being distinct from each other.

[0017] An image decoding method applicable to any form of this disclosure may include: a step of decoding coding mode information of a current coding block; a step of dividing the current coding block into at least one prediction block; and a step of generating prediction samples of the at least one prediction block based on the decoded coding mode information; wherein the coding mode information may be information used to indicate one of intra-frame mode, inter-frame mode, and hybrid mode.

[0018] In the above image decoding method, when the encoding mode information of the current encoding block indicates a mixed mode, the current encoding block can be divided into a first prediction block and a second prediction block.

[0019] In the above image decoding method, when the coding mode information of the current coding block indicates a mixed mode, the prediction sample of the first prediction block can be generated using the prediction information of at least one of the surrounding blocks of the first prediction block, and the prediction sample of the second prediction block can be generated using the prediction information derived based on one of the following modes: intra-frame prediction mode, inter-frame prediction mode, decoder-side motion information derivation (DMID) mode, and decoder-side intra-mode derivation (DIMD) mode.

[0020] In the above image decoding method, when the encoding mode information of the current encoding block indicates a mixed mode, the prediction information of the second prediction block can be derived by decoding the prediction mode information of the second prediction block.

[0021] In the above image decoding method, the decoder-side motion information derivation (DMID) mode can derive final motion information based on the initial motion information derived from the surrounding blocks of the coded block, and use the derived final motion information as the prediction information of the prediction block within the coded block.

[0022] In the above image decoding method, the decoder-side intra-frame pattern derivation (DIMD) mode can set a template block in the reconstruction area around the coded block and use the intra-frame prediction mode that can derive the best predicted pixel value of the template block as the prediction information of the prediction block within the coded block.

[0023] In the above image decoding method, the at least one prediction block can be corner-divided within the current encoded block.

[0024] In the above image decoding method, when at least one prediction block is corner-segmented within the current coding block, diagonal filtering can be applied to the pixels on the corner boundary of the prediction block.

[0025] Furthermore, an image coding method applicable to one form of the present invention may include: the step of dividing a current coding block into at least one prediction block; and the step of generating prediction samples of the at least one prediction block based on coding mode information of the current coding block; wherein the coding mode information may be information used to indicate one of intra-frame mode, inter-frame mode, and hybrid mode.

[0026] In the above image coding method, when the coding mode information of the current coding block indicates a mixed mode, the current coding block can be divided into a first prediction block and a second prediction block.

[0027] In the above image coding method, when the coding mode information of the current coding block indicates a mixed mode, the prediction sample of the first prediction block can be generated using the prediction information of at least one of the surrounding blocks of the first prediction block, and the prediction sample of the second prediction block can be generated using the prediction information derived based on one of the following modes: intra-frame prediction mode, inter-frame prediction mode, decoder-side motion information derivation (DMID) mode, and decoder-side intra-mode derivation (DIMD) mode.

[0028] In the above image coding method, when the coding mode information of the current coding block indicates a mixed mode, the prediction mode of the second prediction block can be determined and the prediction mode information of the second prediction block can be encoded.

[0029] In the above image coding method, the at least one prediction block can be corner-divided within the current coding block.

[0030] In the above image coding method, when at least one prediction block is corner-segmented within the current coding block, diagonal filtering can be applied to the pixels on the corner boundary of the prediction block.

[0031] Furthermore, a recording medium adapted to one form of the present invention can store a bitstream generated by an image coding method, wherein the image coding method can include: the step of dividing a current coding block into at least one prediction block; and the step of generating prediction samples of the at least one prediction block based on the coding mode information of the current coding block; wherein the coding mode information can be information used to indicate one of an intra-frame mode, an inter-frame mode, and a hybrid mode.

[0032] An image decoding method applicable to any form of this disclosure may include: a step of decoding inter-frame mixing mode information of a current coding block; and a step of generating prediction samples of at least one prediction block within the current coding block based on the inter-frame mixing mode information; wherein, when the current coding block is in inter-frame mixing mode, the step of generating prediction samples of at least one prediction block within the current coding block may include: a step of dividing the current coding block into a first prediction block and a second prediction block; a step of deriving motion information of the first prediction block using a merge mode and generating prediction samples of the first prediction block using the derived motion information of the first prediction block; and a step of deriving motion information of the second prediction block using one of a merge mode, an advanced motion vector prediction (AMVP) mode, and a decoder-side motion information derivation (DMID) mode and generating prediction samples of the second prediction block using the derived motion information of the second prediction block.

[0033] In the above image decoding method, the prediction mode used to derive the motion information of the second prediction block can be determined by decoding the inter-frame prediction mode information of the second prediction block.

[0034] In the above image decoding method, the decoder-side motion information derivation (DMID) mode can derive final motion information based on the initial motion information derived from the surrounding blocks of the coded block, and use the derived final motion information as the prediction information of the prediction block within the coded block.

[0035] In the above image decoding method, the step of dividing the current encoded block into a first prediction block and a second prediction block is characterized by: being able to perform corner segmentation on the current encoded block.

[0036] In the above image decoding method, when the first prediction block and the second prediction block are corner-segmented within the current coding block, diagonal filtering can be applied to pixels adjacent to the corner boundary of the first prediction block.

[0037] An image coding method applicable to one form of the present invention may include: a step of encoding inter-frame mixing mode information of a current coding block; and a step of generating prediction samples of at least one prediction block within the current coding block based on the inter-frame mixing mode information; wherein, when the current coding block is in inter-frame mixing mode, the step of generating prediction samples of at least one prediction block within the current coding block may include: a step of dividing the current coding block into a first prediction block and a second prediction block; a step of deriving motion information of the first prediction block using a merge mode and generating prediction samples of the first prediction block using the derived motion information of the first prediction block; and a step of deriving motion information of the second prediction block using one of a merge mode, an advanced motion vector prediction (AMVP) mode, and a decoder-side motion information derivation (DMID) mode and generating prediction samples of the second prediction block using the derived motion information of the second prediction block.

[0038] In the above image coding method, the prediction mode used to derive the motion information of the second prediction block can be determined based on the inter-frame prediction mode information of the second prediction block.

[0039] In the above image coding method, the decoder-side motion information derivation (DMID) mode can derive the final motion information based on the initial motion information derived from the surrounding blocks of the coding block, and use the derived final motion information as the motion information of the prediction block within the coding block.

[0040] In the above image coding method, the step of dividing the current coding block into a first prediction block and a second prediction block can perform corner segmentation on the current coding block.

[0041] In the above image coding method, when the first prediction block and the second prediction block are corner-segmented within the current coding block, diagonal filtering can be applied to pixels adjacent to the corner boundary of the first prediction block.

[0042] A recording medium adapted to one form of the present invention is capable of storing a bitstream generated by an image coding method, wherein the image coding method includes: a step of encoding inter-frame mixing mode information of a current coding block; and a step of generating prediction samples of at least one prediction block within the current coding block based on the inter-frame mixing mode information; wherein, when the current coding block is in inter-frame mixing mode, the step of generating prediction samples of at least one prediction block within the current coding block includes: a step of dividing the current coding block into a first prediction block and a second prediction block; a step of deriving motion information of the first prediction block using a merge mode and generating prediction samples of the first prediction block using the derived motion information of the first prediction block; and a step of using a merge mode, an advanced motion vector prediction (AMVP) mode, and decoder-side motion information derivation (DMID). The steps involve deriving the motion information of the second prediction block from one of the Derivation modes and generating a prediction sample of the second prediction block using the derived motion information of the second prediction block.

[0043] An image decoding method applicable to any form of this disclosure may include: a step of decoding intra-frame mixing mode information of a current coding block; and a step of generating prediction samples of at least one prediction block within the current coding block based on the intra-frame mixing mode information; wherein, when the current coding block is an intra-frame mixing mode, the step of generating prediction samples of at least one prediction block within the current coding block may include: a step of dividing the current coding block into a first prediction block and a second prediction block; a step of generating prediction samples of the first prediction block using an intra-frame prediction mode of the first prediction block derived from the surrounding blocks of the first prediction block; and a step of determining the intra-frame prediction mode of the second prediction block by decoding the intra-frame prediction mode information or decoder-side intra-mode derivation (DIMD) mode information of the second prediction block and generating prediction samples of the second prediction block using the determined intra-frame prediction mode.

[0044] In the above image decoding method, the decoder-side intra-mode derivation (DIMD) mode can derive the final intra-mode prediction based on the intra-mode prediction derived from the surrounding blocks of the coded block, and use the derived intra-mode prediction as the prediction information of the prediction block within the coded block.

[0045] In the above image decoding method, the step of generating the prediction sample of the first prediction block can generate the prediction sample of the first prediction block by deriving the in-frame prediction mode of the surrounding blocks of the first prediction block according to a preset priority order.

[0046] In the above image decoding method, the step of generating the prediction sample of the first prediction block can generate the prediction sample of the first prediction block by deriving the in-frame prediction mode of the surrounding blocks, which is determined based on the index information of the surrounding blocks, into the in-frame prediction mode of the first prediction block.

[0047] In the above image decoding method, the step of dividing the current coded block into a first prediction block and a second prediction block can perform corner segmentation on the current coded block.

[0048] In the above image decoding method, when the first prediction block and the second prediction block are corner-segmented within the current coding block, diagonal filtering can be applied to pixels adjacent to the corner boundary of the first prediction block.

[0049] An image coding method applicable to one aspect of the present invention may include: a step of encoding intra-frame mixing mode information of a current coding block; and a step of generating prediction samples of at least one prediction block within the current coding block based on the intra-frame mixing mode information; wherein, when the current coding block is an intra-frame mixing mode, the step of generating prediction samples of at least one prediction block within the current coding block may include: a step of dividing the current coding block into a first prediction block and a second prediction block; a step of generating prediction samples of the first prediction block using an intra-frame prediction mode of the first prediction block derived from the surrounding blocks of the first prediction block; and a step of determining the intra-frame prediction mode of the second prediction block based on the intra-frame prediction mode of the second prediction block or decoder-side intra-mode derivation (DIMD) mode information and generating prediction samples of the second prediction block using the determined intra-frame prediction mode.

[0050] In the above image coding method, the step of generating the prediction sample of the first prediction block can generate the prediction sample of the first prediction block by deriving the in-frame prediction mode of the surrounding blocks of the first prediction block according to a preset priority order.

[0051] In the above image coding method, the step of generating the prediction sample of the first prediction block can generate the prediction sample of the first prediction block by deriving the intra-frame prediction mode of the surrounding blocks, which is determined based on the index information of the surrounding blocks, into the intra-frame prediction mode of the first prediction block.

[0052] In the above image coding method, the step of dividing the current coding block into a first prediction block and a second prediction block can perform corner segmentation on the current coding block.

[0053] In the above image coding method, when the first prediction block and the second prediction block are corner-segmented within the current coding block, diagonal filtering can be applied to pixels adjacent to the corner boundary of the first prediction block.

[0054] A recording medium adapted to one form of the present invention is capable of storing a bitstream generated by an image coding method, wherein the image coding method includes: a step of encoding intra-frame mixing mode information of a current coding block; and a step of generating prediction samples of at least one prediction block within the current coding block based on the intra-frame mixing mode information; wherein, when the current coding block is an intra-frame mixing mode, the step of generating prediction samples of at least one prediction block within the current coding block includes: a step of dividing the current coding block into a first prediction block and a second prediction block; a step of generating prediction samples of the first prediction block using an intra-frame prediction mode of the first prediction block derived from the surrounding blocks of the first prediction block; and a step of determining the intra-frame prediction mode of the second prediction block based on the intra-frame prediction mode of the second prediction block or decoder-side intra-mode derivation (DIMD) mode information and generating prediction samples of the second prediction block using the determined intra-frame prediction mode.

[0055] Beneficial effects

[0056] This invention can improve the accuracy of prediction by performing prediction using two or more prediction methods within the coded block.

[0057] This invention can improve compression efficiency by performing prediction using two or more prediction methods within the coded block.

[0058] This invention can improve the accuracy of prediction by determining the prediction mode between images based on the positional characteristics of the prediction blocks within the coded blocks.

[0059] This invention can improve compression efficiency by determining the prediction mode between frames based on the positional characteristics of the prediction blocks within the coded blocks.

[0060] This invention can improve the accuracy of prediction by determining the prediction mode within the image based on the positional characteristics of the prediction block within the coded block.

[0061] This invention can improve compression efficiency by determining the prediction mode within the image based on the positional characteristics of the prediction block within the coded block.

[0062] The effects achievable by this disclosure are not limited to those mentioned above. Those skilled in the art to which this disclosure pertains will be able to further understand other effects not mentioned through the following description. Attached Figure Description

[0063] Figure 1 This is a block diagram illustrating an image encoding apparatus to which one embodiment of the present invention is applied.

[0064] Figure 2 This is a schematic diagram used to illustrate an in-screen prediction mode applicable to one embodiment of the present invention.

[0065] Figure 3 This is a flowchart illustrating the encoding method of the best prediction mode for the current prediction block in the prediction mode within the image.

[0066] Figure 4 This is a schematic diagram illustrating an embodiment of a method for setting the most probable mode (MPM) candidate in an in-frame prediction mode.

[0067] Figure 5 This is a flowchart illustrating the method for setting up Advanced Motion Vector Prediction (AMVP) candidates in the inter-frame prediction mode.

[0068] Figure 6 This is a schematic diagram illustrating the locations of the surrounding blocks of the currently predicted block.

[0069] Figure 7 This is a schematic diagram illustrating the method for deriving motion information for time candidates.

[0070] Figure 8 This is a flowchart illustrating the derivation method of the merge candidate in the inter-image prediction mode.

[0071] Figure 9 This is a flowchart illustrating an example of a method for encoding predictive pattern information between images.

[0072] Figure 10This is a block diagram illustrating an image decoding apparatus to which one embodiment of the present invention is applied.

[0073] Figure 11 This is a flowchart illustrating the decoding method for the optimal in-frame prediction mode of the current prediction block.

[0074] Figure 12 This is a flowchart illustrating the decoding method for predictive mode information between images.

[0075] Figure 13 This is a schematic diagram of the first method used to segment the current coded block.

[0076] Figure 14 as well as Figure 15 This is a schematic diagram of the second method used to segment the current coded block.

[0077] Figure 16 Figure 16a , Figure 16b as well as Figure 16c ) and Figure 17 ( Figure 17a , Figure 17b as well as Figure 17c This is a schematic diagram illustrating the method for determining prediction information for prediction blocks within a mixed-mode coded block.

[0078] Figure 18 Figure 18a , Figure 18b as well as Figure 18c This is an example diagram illustrating different segmentation patterns of the peripheral template block position and size in the decoder-side motion information derivation (DMID) mode.

[0079] Figure 19 This is a schematic diagram illustrating the method for deriving motion information in the decoder-side motion information derivation (DMID) mode.

[0080] Figure 20 This is a schematic diagram illustrating the encoding method for predictive information in a hybrid mode.

[0081] Figure 21 This is a flowchart illustrating the method for determining the optimal prediction mode in an image encoding device from intra-frame prediction mode, inter-frame prediction mode, and hybrid mode.

[0082] Figure 22 This is a flowchart illustrating the encoding method for prediction information of prediction blocks within a mixed-mode encoding block in an image encoding device.

[0083] Figure 23 This is a flowchart illustrating the predictive information encoding method that utilizes the decoder-side motion information derivation (DMID) mode.

[0084] Figure 24 This is a flowchart illustrating a method for determining the prediction mode of a prediction block within a mixed-mode coding block in an image coding apparatus.

[0085] Figure 25 This is a flowchart illustrating the decoding method for prediction information in a mixed mode.

[0086] Figure 26 This is a flowchart illustrating the predictive information decoding method that utilizes the Decoder-Side Motion Information Inference (DMID) mode.

[0087] Figure 27 This is a schematic diagram used to illustrate the boundary filtering method.

[0088] Figure 28 This is a flowchart illustrating the image encoding method applicable to the first embodiment of the present invention.

[0089] Figure 29 This is a flowchart illustrating the image decoding method applicable to the first embodiment of the present invention.

[0090] Figure 30 This is a schematic diagram illustrating the derivation method of intra-frame prediction mode using decoder-side intra-mode derivation (DIMD) mode.

[0091] Figure 31 This is a sequence diagram for explaining the encoding method of the prediction information of the current encoding block to which the second embodiment of the present invention applies.

[0092] Figure 32 This is a flowchart illustrating the encoding method for inter-frame mixing mode information in an image encoding device.

[0093] Figure 33 This is a flowchart illustrating a method for determining the prediction mode of a second prediction block within a coding block of an inter-frame blending mode in an image coding apparatus.

[0094] Figure 34 This is a sequence diagram for describing a decoding method for predicting information of the current decoding block to which the second embodiment of the present invention applies.

[0095] Figure 35 This is a flowchart illustrating the image decoding method applicable to the second embodiment of the present invention.

[0096] Figure 36 This is a flowchart illustrating the image decoding method applicable to the second embodiment of the present invention.

[0097] Figure 37 This is a flowchart illustrating the image encoding method applicable to the second embodiment of the present invention.

[0098] Figure 38 This is a sequence diagram for explaining the encoding method of the prediction information of the current encoding block to which the third embodiment of the present invention applies.

[0099] Figure 39 This is a flowchart illustrating the encoding method for intra-frame mixing mode information in an image encoding device.

[0100] Figure 40 This is a flowchart illustrating the method for determining the prediction information of the first prediction block within a coding block of an intra-frame mixing mode in an image coding apparatus.

[0101] Figure 41 This is a sequence diagram for describing a method for decoding prediction information of the current coded block to which the third embodiment of the present invention applies.

[0102] Figure 42 This is a flowchart illustrating the image decoding method applicable to the third embodiment of the present invention.

[0103] Figure 43 This is a flowchart illustrating the image decoding method applicable to the third embodiment of the present invention.

[0104] Figure 44 This is a flowchart illustrating the image encoding method applicable to the third embodiment of the present invention. Detailed Implementation

[0105] This invention is capable of various modifications and has many different embodiments. Specific embodiments will be illustrated and described in detail below with reference to the accompanying drawings. However, the following description is not intended to limit the invention to specific implementations, but should be understood to include all modifications, equivalents, and substitutions within the scope of the invention's concept and technology. Similar reference numerals are used for similar constituent elements in the description of the various drawings.

[0106] In describing different constituent elements, terms such as "first" and "second" may be used, but the constituent elements are not limited by these terms. These terms are merely used to distinguish one constituent element from others. For example, without departing from the scope of the claims, a first constituent element can also be named a second constituent element, and similarly, a second constituent element can also be named a first constituent element. The term "and / or" includes a combination of multiple related descriptions or one of multiple related descriptions.

[0107] When a constituent element is described as being "connected" or "in contact" with other constituent elements, it should be understood that it can not only be directly connected or in contact with the aforementioned other constituent elements, but also that other constituent elements can exist between the two. Conversely, when a constituent element is described as being "directly connected" or "directly in contact" with other constituent elements, it should be understood that no other constituent elements exist between the two.

[0108] The terminology used in this application is for illustrative purposes only and is not intended to limit the invention. Singular statements also have plural meanings unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" are used only to indicate the presence of features, numbers, steps, actions, constituent elements, components, or combinations thereof as described in the specification, and should not be construed as excluding the possibility of one or more other features, numbers, steps, actions, constituent elements, components, or combinations thereof being present or added.

[0109] Next, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, the same reference numerals will be used for the same constituent elements in the drawings, and repeated descriptions of the same constituent elements will be omitted.

[0110] Next, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, the same reference numerals will be used for the same constituent elements in the drawings, and repeated descriptions of the same constituent elements will be omitted.

[0111] Figure 1 This is a block diagram illustrating the configuration of an image encoding apparatus 100 to which one embodiment of the present invention is applied.

[0112] The image encoding apparatus 100 is an apparatus for encoding images and may include: a block segmentation unit 101, a prediction unit 102, a transform unit 103, a quantization unit 104, an entropy encoding unit 105, an inverse quantization unit 106, an inverse transform unit 107, an addition unit 108, a loop filter unit 109, a memory unit 110, and a subtraction unit 111.

[0113] The block segmentation unit 101 can segment the largest block to be encoded (hereinafter referred to as the largest encoded block) into the smallest block to be encoded (hereinafter referred to as the smallest encoded block). This segmentation can be performed based on at least one of quadtree segmentation (hereinafter referred to as quadtree (QT) segmentation) or binary tree segmentation (hereinafter referred to as binary tree (DT) segmentation). Quadtree (QT) segmentation is a method of segmenting a parent block into four child blocks, each with a width and height equal to half the parent block's width. Binary tree (DT) segmentation is a method of segmenting a parent block into two child blocks, each with a width or height equal to half the parent block's width. Binary tree (DT) segmentation can also be referred to as binary tree segmentation.

[0114] The block segmentation unit 101 can segment the input image into at least one block. The input image can be of various shapes and sizes, such as an image, strip, parallel block, or fragment. A block can refer to a coding unit (CU), a prediction unit (PU), or a transform unit (TU).

[0115] In the embodiments where this invention is applied, the encoding unit can be used not only as an encoding execution unit but also as a decoding execution unit. Furthermore, the encoding unit can refer to an encoding block.

[0116] The prediction unit 102 may include an inter-frame prediction unit for performing inter-frame prediction and an intra-frame prediction unit for performing intra-frame prediction. The prediction unit 102 can generate prediction blocks in the current original block using the surrounding pixels of the block to be predicted (hereinafter referred to as the prediction block) or previously decoded reference images. At least one prediction block can be generated within an encoded block. When there is only one prediction block within an encoded block, the prediction block can have the same form as the encoded block. Furthermore, the generation of prediction blocks in the prediction unit 102 can have the same meaning as the prediction samples used to generate the prediction blocks.

[0117] Video signal prediction techniques broadly include intra-frame prediction and inter-frame prediction. Intra-frame prediction generates a prediction block using the surrounding pixels of the current block, while inter-frame prediction generates a prediction block by finding the most similar block to the current block from a previously encoded and decoded reference image. Inter-frame prediction is also known as inter-frame prediction, and intra-frame prediction is known as intra-frame prediction.

[0118] The prediction unit 102 can determine the optimal prediction mode for the prediction block by applying various techniques, such as rate-distortion optimization (RDO), to the residual block obtained by subtracting the prediction block from the current original block. The formula for calculating the RDO cost is shown in Formula 1 below.

[0119] [Formula 1]

[0120] (Φ,λ)=D(D)+λR(Φ)

[0121] Where D can be the distortion caused by quantization, R can be the bit rate of the compressed data stream, J can be the rate-distortion (RD) cost, φ can be the coding mode, and λ is the Lagrange multiplier, which can be a scaling factor used to match the units between the error and the bit count. To select the optimal coding mode during the encoding process, the rate-distortion (RD) cost for the appropriate mode should be less than the rate-distortion (RD) cost for other modes. The formula for calculating the rate-distortion (RD) cost can be performed while simultaneously considering both the bit rate and the error.

[0122] The residual value (residual block) between the generated prediction block and the original block can be input to the transform unit 103. Furthermore, information used during prediction, such as prediction mode information and motion vector information, can be encoded together with the residual value by the entropy coding unit 105 and then transmitted to the decoder. When using a specific coding mode, it is also possible to directly encode the original block and transmit it to the decoding unit without generating the prediction block through the prediction unit 102.

[0123] The intra-frame prediction unit can generate prediction blocks based on pixel information within the current image, i.e., reference pixel information surrounding the current block. When the encoding mode of the surrounding blocks of the current block for which intra-frame prediction needs to be performed is inter-frame prediction, reference pixels contained in the surrounding blocks for which inter-frame prediction has been applied can be replaced with reference pixels in other surrounding blocks for which intra-frame prediction has been applied. That is, if reference pixels are unavailable, they can be used after replacing the unavailable reference pixel information with at least one of the available reference pixels.

[0124] In in-frame prediction, the prediction mode can include a directional prediction mode that uses reference pixel information based on the prediction direction, and a non-directional mode that does not use directional information when performing prediction. The mode used to predict luminance information can be different from the mode used to predict chromatic difference information. When predicting chromatic difference information, the in-frame prediction mode information used in the process of predicting luminance information or the predicted luminance signal information can be used.

[0125] In addition, in-image prediction can have a total of (N+2) prediction modes, including Planar mode, DC mode, and N-angle (Angular) prediction modes.

[0126] Figure 2 The 201 in the figure represents the method for generating predicted blocks in the mean (DC) pattern.

[0127] See Figure 2 In step 201, after applying the average value of reference pixels a to s to all predicted pixels in regions R1 to R4, regions R1 generate the final predicted block by finite impulse response (FIR) filtering with two adjacent reference pixels (a, j), while regions R2 and R3 generate the final predicted block by finite impulse response (FIR) filtering with one adjacent reference pixel (b to h, k to q).

[0128] Figure 2 The 202 in the figure represents the prediction block generation method of the Planar mode.

[0129] See Figure 2 In the 202 Planar mode, the final prediction block is generated by linear interpolation between the reference pixels at the top / left position and the reference pixels copied to the bottom / right position, according to different prediction pixel positions.

[0130] Figure 2 The 203 in the figure represents the prediction direction of the N angular prediction patterns.

[0131] See Figure 2 In the 203 Angular prediction mode, the final prediction block is generated by applying the values ​​of adjacent reference pixels to the prediction block according to different prediction directions.

[0132] Figure 3 This is a flowchart illustrating the encoding method for the best prediction mode of the current prediction block.

[0133] See Figure 3 In step S301, the most probable mode (MPM) candidate can be set. The method for setting the most probable mode (MPM) candidate will be discussed in detail later. Figure 4 Please provide an explanation.

[0134] Next, in step S302, information indicating whether or not the best in-frame prediction mode using the most probable mode (MPM) is encoded can be encoded.

[0135] Next, in step S303, it is possible to determine whether the most probable mode (MPM) action information exists. When the corresponding information is true, in step S304, it is possible to encode the index information indicating which most probable mode (MPM) candidate is the same as the best in-frame prediction mode. Conversely, when the corresponding information is false, in step S305, it is possible to encode the information indicating which prediction mode among the remaining in-frame prediction modes other than the most probable mode (MPM) candidate is the best.

[0136] Figure 4 This is a schematic diagram illustrating an embodiment of a method for setting most probable pattern (MPM) candidates.

[0137] exist Figure 4 In the diagram, L represents the in-frame prediction mode information of the surrounding blocks to the left of the current prediction block, and A represents the in-frame prediction mode information of the surrounding blocks at the top.

[0138] Referring to the figure, the three most likely pattern (MPM) candidates can be determined based on the specified conditions.

[0139] Furthermore, the number of most likely pattern (MPM) candidates can be determined to be P (P>0, P is an integer), and there can be a variety of methods to determine the candidates.

[0140] Return to the Figure 1 The prediction unit 102 described herein includes an inter-frame prediction unit that can generate prediction blocks based on information from at least one of the previous or next images of the current image, and in some cases, can also generate prediction blocks based on information from a portion of the current image that has already been encoded. The inter-frame prediction unit may include a reference image interpolation unit, a motion prediction unit, and a motion compensation unit.

[0141] In addition, the inter-frame prediction features advanced motion vector prediction (AMVP), merge, and skip modes.

[0142] Next, we will combine Figures 5 to 7 This section explains the Advanced Motion Vector Prediction (AMVP) mode in inter-frame prediction.

[0143] In Advanced Motion Vector Prediction (AMVP) mode, the prediction information of the surrounding blocks of the current prediction block can be used to set the Advanced Motion Vector Prediction (AMVP) candidates.

[0144] Figure 5 This is a flowchart illustrating the method for specifying candidates for Advanced Motion Vector Prediction (AMVP). Figure 6 This is a schematic diagram illustrating the locations of the surrounding blocks of the currently predicted block.

[0145] See Figure 5 In step S501, two spatial candidates can be derived from the surrounding blocks of the current predicted block. Specifically, it is possible to select blocks containing... Figure 6 After selecting the two A1→A4 items in sequence, choose the one that is available first. Figure 6 From the three options in the order A3→A2→A5, select the one that is available first, and then select the other two space candidates.

[0146] In step S502, it is possible to follow Figure 6 After exploring two time candidates in the order B1→B2, the first available candidate is selected. The method for deriving the motion information of the time candidates will be discussed in subsequent sections. Figure 7 Please provide an explanation.

[0147] Next, in step S503, duplicate candidates can be deleted from the spatial candidates derived in step S501 and the temporal candidates derived in step S502.

[0148] Next, in step S504, when the number of currently derived candidates is less than the number of final Advanced Motion Vector Prediction (AMVP) candidates, (0,0) motion information can be added. The number of final AMVP candidates can be determined to be different values; in this specification, it is assumed to be two.

[0149] Next, in step S505, the two derived Advanced Motion Vector Prediction (AMVP) candidates are finally selected. After determining the final AMVP candidate with the lowest rate-distortion cost (RD-cost) from the two AMVP candidates derived by the method described above through the rate-distortion optimization (RDO) process, the motion inference process is performed with the position pointed to by the motion vector of the corresponding candidate as the starting point, thereby finding the optimal motion vector through motion inference.

[0150] Furthermore, the number of Advanced Motion Vector Prediction (AMVP) candidates can be determined to be Q (Q>0, where Q is an integer), and there can be a variety of methods for determining AMVP candidates.

[0151] Figure 7 This is a schematic diagram illustrating the method for deriving motion information for time candidates.

[0152] See Figure 7This method can explore the motion information of temporal candidate blocks within a collocated image, using blocks at the same position as the current block in the current image as a reference. The motion vector of the corresponding motion information points to the predicted block in reference image B. By calculating the temporal distance between reference image B and the collocated image and scaling it according to the temporal distance between reference image B and the reference image A referenced by the current image, a scaled motion vector can be derived. This scaled motion vector can be used as the motion information for the temporal candidate. In this case, reference image A and reference image B can be the same image.

[0153] Next, we will combine Figure 8 This section explains the Merge mode in the prediction between images.

[0154] In Merge mode, the Merge candidate can be set using the prediction information of the surrounding blocks of the currently predicted block.

[0155] Figure 8 This is a flowchart illustrating the method for setting merge candidates.

[0156] See Figure 8 In step S801, spatial candidates can be derived. At this time, it is possible to... Figure 6 After selecting the five A1 to A5 candidates, explore them in the order of A1→A2→A3→A4→A5 and select the four available candidates first.

[0157] Next, in step S802, time candidates can be derived, and the corresponding method is the same as that described above. Figure 5 The steps are the same as in step S502.

[0158] Next, in step S803, the currently derived duplicate candidates can be deleted and combined bidirectional candidates can be added. Combined bidirectional candidates refer to candidates that combine bidirectional motion information based on the motion information of previously determined candidates.

[0159] Next, in step S804, when the number of candidates derived so far is less than the number of candidates to be merged, (0,0) motion information can be added. The number of candidates to be merged can be determined to be different values; in this specification, it is assumed to be 5.

[0160] In step S805, the five derived candidates are selected as the final merge candidates. Through rate-distortion optimization (RDO), the merge candidate with the best motion information for the current predicted block is determined from the five merge candidates derived as described above.

[0161] Furthermore, the number of merge candidates can be determined to be R (R>0, R is an integer), and there can be a variety of methods to determine the merge candidates.

[0162] Figure 9 This is a flowchart illustrating an example of a method for encoding predictive pattern information between images.

[0163] See Figure 9 In step S901, the merge action information can be encoded.

[0164] Next, in step S902, it is possible to determine whether or not a merging action has occurred based on the merging action information.

[0165] When the Merge action is true, in step S903, the index information of the Merge candidates can be encoded and the process can be completed. Figure 9 The process is as follows. When the Merge action is false, the predicted direction can be encoded in step S904. The predicted direction can be encoded as one of three types: bidirectional, historical, or future.

[0166] Next, in step S905, it can be determined whether the predicted direction is a future direction. If it is not a future direction, in step S906, the reference image index information related to the historical direction can be encoded, and in step S907, the historical direction motion vector difference (MVD) information can be encoded. Here, motion vector difference (MVD) is an abbreviation for Motion Vector Difference, which represents the difference between the best historical motion vector of the current prediction block and the motion vector of the best advanced motion vector prediction (AMVP) candidate of the historical direction.

[0167] Next, in step S908, the historical direction motion vector prediction (MVP) information can be encoded, where motion vector prediction (MVP) is an abbreviation for Motion Vector Predictor, which refers to the best advanced motion vector prediction (AMVP) candidate among the two final advanced motion vector prediction (AMVP) candidates.

[0168] Next, in step S909, it can be determined whether the predicted direction is a historical direction. When the predicted direction is not a historical direction, in step S910, the reference image index information for the future direction can be encoded, and in step S911, the motion vector difference (MVD) information for the future direction can be encoded. In step S912, the motion vector prediction (MVP) information for the future direction can be encoded.

[0169] Next, we will explain the skip mode in the inter-frame prediction.

[0170] Skip mode is a mode that directly applies motion information from surrounding blocks to the current predicted block. Furthermore, in Skip mode, a prediction sample for the current predicted block can be generated using the motion information of surrounding blocks, and this prediction sample can be directly used as a reconstruction sample. That is, in Skip mode, the residual block generation step can be omitted.

[0171] The prediction unit 102 is able to determine which neighboring block's motion information to use as the current block's motion information when using the Skip mode.

[0172] Furthermore, the prediction unit 102 can encode the coding mode information used to indicate the prediction method of the coding block. As described above, the coding block can be divided into at least one prediction block, and a prediction method can be applied to the prediction blocks divided from a coding block. However, the prediction modes of different prediction blocks can be different.

[0173] As an example, when encoding the current coding block through inter-frame prediction, the first prediction within the current coding block can generate prediction samples based on the Advanced Motion Vector Prediction (AMVP) mode, while the second prediction block can generate prediction samples based on the Merge mode.

[0174] That is, encoding pattern information can be encoded in units of encoding blocks, and prediction pattern information can be encoded in units of prediction blocks.

[0175] Return to the Figure 1 As explained, the transformation unit 103 can generate a transformed block by transforming the residual block, which is the difference between the original block and the predicted block. The transformed block is the smallest unit used during the transformation and quantization process.

[0176] The transform unit 103 can generate a transform block with transform coefficients by transforming the residual signal into a frequency region. As a method for transforming the residual signal into a frequency region, various transform techniques can be used, such as Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), and Karhunen-Loeve Transform (KLT), thereby transforming the residual signal into a frequency region and generating transform coefficients. To use the transform techniques, matrix operations using basis vectors are performed, and depending on the prediction mode used when encoding the prediction block, various different transform techniques can be combined during matrix operations. For example, when performing in-frame prediction, a Discrete Cosine Transform can be used in the horizontal direction and a Discrete Sine Transform in the vertical direction, depending on the prediction mode.

[0177] Furthermore, the transform method can be determined based on the intra-prediction mode of the prediction block used when generating the residual block. For example, the discrete cosine transform (DCT) can be used in the horizontal direction and the discrete sine transform (DST) in the vertical direction, depending on the intra-prediction mode.

[0178] The transform block can determine the optimal transform block segmentation shape after segmenting into coding block units using either a quadtree (QT) or a binary tree (BT) method, and then transmit the transform block segmentation information to the image decoding device 1000. The transform block segmentation method described above can be referred to as a residual tree (RT) structure for coding unit (CU) units. This means that transforms can be performed either at the predictive block level or without considering the boundaries of the predictive blocks; that is, transform blocks can be determined regardless of the boundaries of the predictive blocks.

[0179] Furthermore, the transform block segmentation pattern can be determined after segmenting the predictive block units using a quadtree (QT) or binary tree (BT) method, and the transform block segmentation information can be transmitted to the image decoding device 1000 in transform block units. The transform block segmentation method described above can be referred to as a residual tree (RT) structure for predictive unit (PU) units. This means that transform blocks cannot be formed with segmentation patterns exceeding the predictive block boundaries; that is, transform blocks cannot be determined beyond the predictive block boundaries.

[0180] In the residual tree (RT) structure of the coding unit (CU) unit as described above, the transform block segmentation information can be omitted, and the transform can be performed after the entire coding block is determined as a transform block. The same method can also be applied to the residual tree (RT) structure of the prediction unit (PU) unit.

[0181] The quantization unit 104 can generate a quantized transform block by quantizing the transform block. That is, the quantization unit 104 can quantize the transform coefficients of the transform block generated from the transform unit 103 to generate a quantized transform block with quantized transform coefficients. As a quantization method, various quantization methods can be used, such as Dead Zone Uniform Threshold Quantization (DZUTQ) or Quantization Weighted Matrix, or improved quantization methods.

[0182] Furthermore, the above description illustrates and explains the case where the image encoding apparatus 100 includes both a transformation unit 103 and a quantization unit 104. However, the transformation unit 103 and the quantization unit 104 can also be selectively included. That is, the image encoding apparatus 100 can generate a transformed block by transforming the residual block but without performing a quantization process, or it can perform a quantization process without transforming the residual block into frequency coefficients, or it can even perform neither a transformation nor a quantization process.

[0183] Even if the image encoding device 100 does not perform part of the process of the transformation unit 103 and the quantization unit 104 or does not perform all of the process, the block input to the entropy encoding unit 105 is usually referred to as a "quantized transformation block".

[0184] The entropy encoding unit 105 is capable of encoding the quantized transform blocks and outputting a bitstream. That is, the entropy encoding unit 105 is capable of encoding the quantized transform blocks output from the quantization unit 104 using various entropy encoding techniques, and generating and outputting a bitstream containing additional information (e.g., information related to the prediction mode, quantization coefficients, etc.) required for decoding the corresponding blocks in the image decoding apparatus described later.

[0185] When performing entropy coding, various coding methods can be used, such as Exponential Golomb code, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).

[0186] The inverse quantization unit 106 can reconstruct the inverse quantization transform block by reversing the quantization techniques used in the quantization process on the quantized transform block.

[0187] The inverse transform unit 107 reconstructs the residual block by performing an inverse transform on the inverse quantization transform block using the same method as that used in the transform process. It can perform the inverse transform by reversing the transform technique used in the transform unit 104.

[0188] Furthermore, as described above, the inverse quantization unit 106 and the inverse transform unit 107 can perform inverse quantization and inverse transform by reversing the quantization and transform methods used in the quantization unit 104 and the transform unit 103. Alternatively, if only quantization is performed in the transform unit 103 and the quantization unit 104 without performing transform, only inverse quantization can be performed without performing inverse transform. If neither transform nor quantization is performed, the inverse quantization unit 106 and the inverse transform unit 107 can also omit inverse transform and inverse quantization, or be omitted directly from the image encoding apparatus 100.

[0189] The addition unit 108 can perform addition operations on the residual signal generated by the inverse transform unit 107 and the prediction block generated by prediction, thereby reconstructing the current block.

[0190] The loop filtering unit 109 is used to add a filtering part to the overall image area after all blocks in the current image have been reconstructed, and can include at least one of deblocking filtering, sample adaptive offset (SAO), and adaptive loop filter (ALF).

[0191] Deblocking filters eliminate block distortion in reconstructed images caused by boundaries between blocks. To determine whether deblocking is necessary, the decision can be based on the pixels contained in several columns or rows within a block. When applying a deblocking filter, a strong or weak filter can be used depending on the required deblocking intensity. Furthermore, during the application of a deblocking filter, horizontal and vertical filtering can be performed in parallel.

[0192] Sample Adaptive Offset (SAO) is an operation that minimizes the difference between a reconstructed image and the original image by performing addition or subtraction operations on the reconstructed pixels using specific values.

[0193] Adaptive Loop Filtering (ALF) is performed based on a comparison between the filtered reconstructed image and the original image. It involves dividing the pixels in the image into specific groups, determining which filter should be applied to each group, and then performing different filters on different groups. Information related to the applicability of ALF can be transmitted according to each coding unit (CU), and the shape and filter coefficients of the applicable ALF filter can differ depending on the block. Furthermore, it is possible to apply an ALF filter of the same shape (fixed shape) regardless of the characteristics of the target block.

[0194] After performing an addition operation on the residual signal generated by the inverse transformation unit 107 and the prediction block generated by prediction, the memory 110 stores the reconstructed current block after additional filtering in the loop filtering unit 109 and uses it to predict the next block or the next image, etc.

[0195] The subtraction unit 111 can generate a residual block by performing a subtraction operation between the current original block and the predicted block.

[0196] Figure 10 This is a block diagram illustrating an image decoding apparatus 1000 to which one embodiment of the present invention is applied.

[0197] See Figure 10 The image decoding device 1000 may include: a block entropy decoding unit 1001, an inverse quantization unit 1002, an inverse transform unit 1003, a prediction unit 1004, an addition unit 1005, a loop filter unit 1006, and a memory 1007.

[0198] When the image bitstream generated by the image encoding device 100 is input to the image decoding device 1000, the input bitstream can be decoded in the reverse order of the process performed by the image encoding device 100. Furthermore, the "encoding block" in the image encoding device 100 can be referred to as the "decoding block" in the image decoding device 1000.

[0199] The entropy decoding unit 1001 can obtain various information required for decoding the corresponding block and the quantized transform coefficients by parsing the bit stream transmitted from the image encoding device 100.

[0200] Furthermore, the entropy decoding unit 1001 can perform entropy decoding according to the reverse steps of entropy encoding performed in the entropy encoding unit 105 of the image encoding apparatus 100. For example, it can be compatible with various methods such as Exponential Golomb code, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC), corresponding to the methods performed in the image encoder. In the entropy decoding unit 1001, the coefficients of the transform block can be decoded using various types of flags, such as those indicating non-zero coefficients, coefficients with absolute values ​​greater than 1 or 2, and the sign of the coefficients, in partial block units within the transform block. For coefficients that cannot be represented solely by the aforementioned flags, decoding can be performed based on the sum of the coefficients represented by the flags and the signaling coefficients.

[0201] Furthermore, the entropy decoding unit 1001 is capable of decoding information related to intra-frame prediction and inter-frame prediction performed in the video coding apparatus.

[0202] The inverse quantization unit 1002 can obtain an inversely quantized block containing inversely quantized coefficients by reversing the quantization techniques used in the quantization process on the quantized coefficients decoded in the entropy decoding unit 1001. The inverse quantization unit 1002 can also... Figure 1 The inverse quantization unit 106 in the middle works in essentially the same way.

[0203] The inverse transform unit 1003 obtains a residual block with a differential signal by performing an inverse transform on the inverse quantization transform block using the same method as used in the transform process. The inverse transform unit 1003 can perform an inverse transform on the inverse quantization transform block according to the method used in the transform process. Figure 1 The inverse transformation unit 107 in the middle works in the same way.

[0204] The prediction unit 1004 generates prediction blocks using the encoding mode information decoded by the entropy decoding unit 1001, and at this time, the same prediction method as that performed in the prediction unit 102 of the image encoding apparatus 100 can be used.

[0205] Figure 11 This is a flowchart illustrating the decoding method for the optimal in-frame prediction mode of the current prediction block.

[0206] See Figure 11 In step S1101, the most probable mode (MPM) candidate can be set. The method for setting the most probable mode (MPM) candidate is the same as that described above for the prediction unit 102 of the image encoding apparatus 100. Figure 3 The method for setting the most likely pattern (MPM) candidate is the same.

[0207] Next, in step S1102, information indicating whether or not the in-frame prediction mode using the most probable mode (MPM) is encoded can be decoded.

[0208] Next, in step S1103, the presence or absence of the Most Probable Mode (MPM) action information can be determined based on the information decoded in step S1102. When the corresponding information is true, in step S1104, the index information indicating which Most Probable Mode (MPM) candidate the in-frame prediction mode is the same can be decoded, thereby determining the best in-frame prediction mode for the current prediction block. Conversely, when the corresponding information is false, in step S1105, the information indicating which of the remaining in-frame prediction modes other than the Most Probable Mode (MPM) candidate in the in-frame prediction mode is the best can be decoded, thereby determining the in-frame prediction mode for the current prediction block.

[0209] Figure 12 This is a flowchart illustrating the decoding method for predictive mode information between images.

[0210] See Figure 12 In step S1201, the merge action information can be decoded.

[0211] Next, in step S1102, it is possible to determine whether or not a merging action has occurred based on the merging action information decoded in step S1201.

[0212] When the Merge action is true, in step S1103, the index information of the Merge candidates can be decoded and the process ends. Figure 12The process is as follows. When the Merge action is false, the predicted direction can be decoded in step S1204. The predicted direction can be decoded in one of three ways: bidirectional, historical, or future.

[0213] Next, in step S1205, it can be determined whether the predicted direction is a future direction. If it is not a future direction, in step S1206, the reference image index information related to the historical direction can be decoded, and in step S1207, the historical direction motion vector difference (MVD) information can be decoded.

[0214] Next, in step S1208, the motion vector prediction (MVP) information of the historical direction can be decoded.

[0215] Next, in step S1209, it can be determined whether the predicted direction is a historical direction. When the predicted direction is not a historical direction, in step S1210, the reference image index information for the future direction can be decoded, and in step S1211, the motion vector difference (MVD) information for the future direction can be decoded. In step S1212, the motion vector prediction (MVP) information for the future direction can be decoded.

[0216] Return to the Figure 10 As explained, the addition unit 1005 can generate a reconstruction block by performing an addition operation on the prediction block generated in the prediction unit 1004 and the residual block generated by the inverse transformation unit 1003.

[0217] The loop filtering unit 1006 is used to add filtering to the entire image area after reconstructing all blocks in the current image, and can include deblocking filtering and Sample Adaptive Offset (SAO). The image encoding device 100 can receive information related to whether a deblocking filter is applied to a corresponding block or image, and information related to whether strong or weak filtering is applied when a deblocking filter is applied. The loop filtering unit 1006 can adjust according to... Figure 1 The loop filter section 109 in the middle works in essentially the same way.

[0218] After performing an addition operation on the residual signal generated by the inverse transformation unit 1003 and the prediction block generated by prediction, the memory 1007 stores the reconstructed current block after additional filtering in the loop filtering unit 1006 and uses it to predict the next block or the next image, etc.

[0219] As described above, for ease of explanation in embodiments of the present invention, the term "coding unit" is used, but this can refer to either a unit that performs encoding or a unit that performs decoding. Furthermore, a unit or cell can be a region generated by segmenting an image. Additionally, in embodiments of the present invention, a unit can refer to a block, and the current block can refer to the current encoding target block or the current decoding target block.

[0220] Next, we will combine Figures 13 to 15 The method for segmenting coded blocks is explained.

[0221] Figure 13 This is a schematic diagram of the first method used to segment the current coded block.

[0222] See Figure 13 The coded blocks can be divided into squares or rectangles. Figure 13 1301 is an example of vertical segmentation of the coded block, and 1302 is an example of horizontal segmentation of the coded block. In each example, the encoding order of the prediction blocks within the coded block can be in the order of prediction block A, prediction block B, etc. The dashed lines within the coded block are the dividing lines of the coded blocks and can represent the boundary lines between prediction blocks. Figure 13 In addition to accurately dividing the coded block into two equal parts, the coded block dividing lines in 1301 and 1302 can also be divided into two equal parts with different sizes.

[0223] Furthermore, as a first method for determining the segmentation pattern of the coded block, the rate-distortion cost (RD-Cost) is calculated for all segmentation patterns with P values ​​ranging from 1 to 2N-1, depending on whether the coded block is horizontally or vertically segmented. The segmentation pattern with the minimum rate-distortion cost (RD-Cost) is then determined as the optimal segmentation pattern for the current coded block.

[0224] Figure 14 as well as Figure 15 This is a schematic diagram of the second method used to segment the current coded block.

[0225] As a second method for determining the segmentation pattern of coded blocks, it is possible to pre-define the available coded block segmentation groups and calculate the rate-distortion cost (RD-Cost) of the pre-define segmentation pattern to determine the optimal segmentation pattern. In this case, the pre-defineable segmentation pattern information can be pre-transmitted via a higher-level header. The higher-level header refers to the transmission layer above block units, such as parameter layers, sequence parameter layers, image parameter layers, and stripe layers.

[0226] As a predefined segmentation pattern, it can include symmetrical vertical segmentation, symmetrical horizontal segmentation, asymmetrical vertical segmentation, and asymmetrical horizontal seam.

[0227] See Figure 14 It can include (1 / 4, 3 / 4) vertical division, (1 / 2, 1 / 2) vertical division, (3 / 4, 1 / 4) vertical division, (1 / 4, 3 / 4) horizontal division, (1 / 2, 1 / 2) horizontal division, and (3 / 4, 1 / 4) horizontal division. Among them, (1 / 4, 3 / 4) horizontal division, (3 / 4, 1 / 4) horizontal division, (1 / 4, 3 / 4) vertical division, and (3 / 4, 1 / 4) vertical division are asymmetrical division forms, while (1 / 2, 1 / 2) horizontal division and (1 / 2, 1 / 2) vertical division are symmetrical division forms.

[0228] Furthermore, as a predefined segmentation pattern, corner segmentation can be included.

[0229] See Figure 15 It can include a top left corner segment 1501, a top right corner segment 1502, a bottom left corner segment 1503, and a bottom right corner segment 1504. The predicted block of the segmented corner portion can be 1 / 4 the width of the encoded block.

[0230] In the top left corner segment 1501, the top left predicted block can be predicted block A (N×N), while the remaining area can be predicted block B.

[0231] Furthermore, in the upper right corner segment 1502, the upper right predicted block can be predicted block A (N×N), while the remaining area can be predicted block B.

[0232] Furthermore, in the lower left corner segment 1503, the lower left predicted block can be predicted block A (N×N), while the remaining area can be predicted block B.

[0233] Furthermore, in the lower right corner segmentation 1504, the lower right prediction block can be prediction block A (N×N), while the remaining area can be prediction block B. In each segmentation pattern, the encoding order of the encoded blocks can be prediction block A, then prediction block B. Additionally, in the corresponding segmentation method, the size of the prediction block, which is 1 / 4 of the area size, can be increased or decreased.

[0234] Furthermore, the shape of each segmentation method can be non-square, such as a rectangle or other different forms. Additionally, the rate-distortion cost (RD-Cost) of each segmentation form can be compared, and the segmentation form with the minimum RD-Cost is determined as the optimal segmentation form for the current coded block.

[0235] Next, an encoding mode (hereinafter referred to as "hybrid mode") for generating prediction samples of a prediction block using multiple prediction modes within a coding block, according to one embodiment of the present invention, will be described.

[0236] The prediction modes available in the hybrid mode include intra-frame prediction modes (Planar, DC, and Angular modes), inter-frame prediction modes (AMVP, Merge, and Skip modes), decoder-side Motion Information Derivation (DMID) mode, and decoder-side Intra Mode Derivation (DIMD) mode. In the coded blocks of the hybrid mode, prediction samples for the prediction blocks can be generated by combining multiple prediction modes as described above.

[0237] The decoder-side motion information derivation (DMID) mode refers to a prediction mode in which unencoded motion information is directly derived in the image decoding device 1000 without being encoded in the image encoding device 100, and prediction samples of prediction blocks are generated. The motion information that needs to be directly derived in the image decoding device 1000 because it is not encoded in the image encoding device 100 can be a portion of motion information containing motion vectors.

[0238] As an example, the decoder-side motion information derivation (DMID) mode is a mode in which the image decoding device 1000 derives the initial motion information of the prediction block from the surrounding blocks of the coded block and then derives the final motion information of the prediction block based on the derived initial motion information to generate a prediction sample of the prediction block.

[0239] As another example, the decoder-side motion information derivation (DMID) mode is a mode in which the image decoding device 1000 generates a prediction sample of the prediction block after deriving the final motion information of the prediction block based on the initial motion information transmitted from the image encoding device 1000.

[0240] Among them, the decoder-side intra-frame prediction mode (DIMD) can refer to a prediction mode that is not encoded in the image encoding device 1000 but is directly derived in the image decoding device 1000 to generate prediction samples of prediction blocks.

[0241] As an example, the aforementioned decoder-side intra-frame pattern derivation (DIMD) mode can set a template block in the reconstruction area around the coded block and use the intra-frame prediction mode, from which the best predicted pixel value of the set template block can be derived, as the prediction information of the prediction block within the coded block.

[0242] Furthermore, when the encoded block is in mixed mode and is divided into prediction block A and prediction block B, the prediction information of prediction block A can be determined by directly using the prediction information of the surrounding blocks of prediction block A, while the prediction information of prediction block B can be determined based on one of the following: intra-frame prediction mode (Planar mode, DC mode, and Angular mode), inter-frame prediction mode (Advanced Motion Vector Prediction (AMVP) mode, Merge mode, and Skip mode), decoder-side Motion Information Derivation (DMID) mode, and decoder-side Intra Mode Derivation (DIMD).

[0243] Figure 16 Figure 16a , Figure 16b as well as Figure 16c This is a diagram illustrating a method for determining the prediction information for block A by directly using the prediction information of surrounding blocks.

[0244] The prediction block A can explore the best prediction information of the prediction block A from the prediction information of the neighboring blocks that have been encoded and determine the best prediction information as the prediction information of the prediction block A, thereby generating the prediction block.

[0245] The predicted locations of the surrounding blocks of block A can be shown as 1601 to 1606 in Figure 16. Furthermore, when a priority order exists among the surrounding blocks, the best prediction information can be explored according to the corresponding priority order. In this example, the priority order is assumed to be the order of surrounding blocks 1, 2, 3, 4, and 5.

[0246] Figure 16a The predicted block A in 1601 can have five surrounding blocks, namely blocks 1 to 5. The rate-distortion cost (RD-Cost) can be calculated by applying the prediction information of the surrounding blocks 1 to 5 of the predicted block A to the predicted block A, and the prediction information of the surrounding block with the smallest rate-distortion cost (RD-Cost) is determined as the prediction information of the predicted block A.

[0247] right Figure 16a The description of 1602 is the same as that of 1601, except that the positions of the surrounding blocks have changed.

[0248] Figure 16b as well as Figure 16c In the diagram, 1603-1606 indicates that the number and location of surrounding blocks change depending on the location of the predicted block A. Under different circumstances, the rate-distortion cost (RD-Cost) can be calculated by applying the prediction information of all surrounding blocks to the predicted block A, and the prediction information with the minimum rate-distortion cost (RD-Cost) is determined as the best prediction information for the predicted block A. Then, the predicted block is generated using the corresponding prediction information.

[0249] Furthermore, as a method for determining the prediction information for prediction block B, the prediction information for prediction block B can also be determined directly using the prediction information of surrounding blocks. Alternatively, the prediction information for prediction block B can be determined by exploring the best prediction information in the intra-frame prediction mode (intra-frame prediction), and the prediction information for prediction block B can also be determined by exploring the best prediction information in a reference image reconstructed using motion inference (inter-frame prediction).

[0250] However, in some cases, the prediction information for block B can be determined by referring to the prediction information for block A. The following will explain some of these cases.

[0251] Figure 17 Figure 17a , Figure 17b as well as Figure 17c This is a schematic diagram illustrating the method for determining the prediction information for prediction block B.

[0252] The positions of the surrounding blocks of predicted block B can be shown as 1701-1706 in Figure 17. Furthermore, when there is a priority order among the surrounding blocks, the best prediction information can be explored according to the corresponding priority order. In this example, the priority order is assumed to be the order of surrounding blocks 1, 2, 3, 4, and 5.

[0253] In addition, such as Figure 17a As shown in 1701-1702, the number of candidate surrounding blocks can be 3 or 4 instead of 5.

[0254] At this point, the prediction information for prediction block B can be determined by referencing the prediction information for prediction block A based on the segmentation structure of the transformed block.

[0255] If the segmentation structure of the current transform block is a residual tree (RT) structure of coding unit (CU) units, then the prediction information of the prediction block B can be determined by referring to the prediction information of the prediction block A if at least one of the following conditions is met.

[0256] 1) When prediction block A is divided into sub-block units and the prediction information of each sub-block unit is different.

[0257] 2) When predicted block A is not divided into sub-block units but predicted block B is divided into sub-block units and the prediction information of each sub-block unit is different.

[0258] Furthermore, if the current transformation block has a segmentation structure of a residual tree (RT) structure of prediction unit (CU) units, then prediction block B can determine the prediction information of prediction block B by referring to the prediction information of prediction block A when at least one of the following conditions is met.

[0259] 1) When prediction block A is divided into sub-block units and the prediction information of each sub-block unit is different.

[0260] 2) When predicted block A is not divided into sub-block units but predicted block B is divided into sub-block units and the prediction information of each sub-block unit is different.

[0261] 3) When the optimal prediction mode for blocks A and B is Skip mode, the remaining cases are excluding the case where block B references the prediction information of block A.

[0262] Furthermore, the prediction information for prediction block B can be determined based on either an intra-frame prediction mode or an inter-frame prediction mode, as described above. In the case described above, such as for... Figure 1 The prediction unit 102 of the image encoding apparatus 100 is described in this paper, which can determine prediction information based on various prediction modes and generate prediction blocks using the determined prediction information.

[0263] Furthermore, the prediction information for prediction block B can be determined based on either decoder-side motion information derivation (DMID) mode or decoder-side intra-frame mode derivation (DIMD) mode. (Figure 18) Figure 19 as well as Figure 30 This is a schematic diagram illustrating the prediction information exploration method in decoder-side motion information derivation (DMID) mode and decoder-side intra-frame mode derivation (DIMD) mode.

[0264] Figure 18 Figure 18a , Figure 18b as well as Figure 18cThis example illustrates the surrounding template blocks with different segmentation patterns when exploring the prediction pattern of prediction block B generated within the current coding block using either decoder-side Motion Information Derivation (DMID) mode or decoder-side Intra-Frame Mode Derivation (DIMD) mode. The template blocks can be determined from the reconstructed regions surrounding the current coding block.

[0265] In the image decoding apparatus 1000, the decoder-side Motion Information Derivation (DMID) mode assumes the difference between the surrounding template block of the predicted block in the reference image based on the initial motion vector and the surrounding template block of the current block in the current image as an error used to calculate the rate-distortion code (RD-Cost), replacing the difference between the predicted block and the original block. Conversely, the decoder-side Intra-Frame Pattern Derivation (DIMD) mode assumes the difference between the reconstructed pixel value of the template region and the predicted pixel value of the template region derived using reference pixels surrounding the template region as an error used to calculate the rate-distortion code (RD-Cost), replacing the difference between the predicted block and the original block, thereby exploring prediction information. When motion inference is performed in the decoder-side Motion Information Derivation (DMID) mode, the initial motion vector can use the motion vector whose surrounding reconstructed motion information has been used.

[0266] exist Figure 18a In step 1801, the template block at the top can be adjacent to the prediction block B, while the template block to the left can be not adjacent to the prediction block. This is to perform encoding / quantization on a block-by-block basis. Furthermore, a template block to the left can be generated within the prediction block A region adjacent to prediction block B. This is possible when performing encoding / quantization on a block-by-block basis.

[0267] exist Figure 18a In step 1802, the template block on the left can be adjacent to the prediction block B, while the template block on the top can not be adjacent to the prediction block. Furthermore, a template block for the upper region can be generated within the prediction block A region adjacent to prediction block B.

[0268] like Figure 18b as well as Figure 18c As shown in 1803-1805, when the shape of the predicted block B is square or rectangular, the size and shape of the template blocks at the top and left sides can be different.

[0269] Figure 18c Example 1806 is a template block in a segmentation pattern where prediction block B has no adjacent coded blocks. When transform / quantization is performed on a prediction block basis, template blocks can be generated within the prediction block A region in a manner adjacent to prediction block B.

[0270] Furthermore, in Figures 1801–1806, when only one side of the template block is available, the template block for prediction block B can use only one side of the template block.

[0271] In addition to the template block shown in Figure 18, various other template blocks can be used in terms of position, size, and shape.

[0272] Figure 19 This is a schematic diagram illustrating the predictive information exploration method for decoder-side motion information derivation (DMID) modes.

[0273] exist Figure 19 1901 in the document is used to describe the method of exploring prediction information using template blocks as described above in the Decoding-Side Motion Information Inference (DMID) mode. Figure 19 1902 in the document describes a method for exploring prediction information without using template blocks when the prediction direction is bidirectional in the Derivation of Motion Information (DMID) mode on the decoder side.

[0274] See Figure 19 In the 1901 section, a template block can exist on the left / top of the current block in the current image. The template block at the top is the CurAboveTemplate block, while the template block on the left is the CurLeftTemplate block.

[0275] The CurAboveTemplate block has a horizontal length of Aw and a vertical length of Ah, while the CurLeftTemplate block has a horizontal length of Lw and a vertical length of Lh.

[0276] Furthermore, the horizontal and vertical lengths of the RefAboveTemplate and RefLeftTemplate blocks of the reference image can be the same as the horizontal and vertical lengths of the surrounding template blocks of the current block.

[0277] Motion inference in the decoder-side Motion Information Derivation (DMID) mode involves finding the template block most similar to the current block's surrounding template blocks within the reference image, and then determining the blocks adjacent to the current template block as the prediction blocks. This method is applicable to all prediction directions (historical, future, and bidirectional).

[0278] Figure 19 1902 in the example is a decoder-side motion information derivation (DMID) encoding method that does not use template blocks when the prediction direction is bidirectional prediction in the decoder-side motion information derivation (DMID) mode. It finds the position with the smallest difference between the prediction block in the historical direction and the prediction block in the future direction and then determines the motion information pointed to by the corresponding position as the prediction information of the current block.

[0279] Figure 30 This section describes a method for exploring prediction information in decoder-side intra-mode derivation (DIMD) mode. It provides examples of surrounding template regions with different segmentation patterns when exploring the prediction mode of prediction block B generated within the current coding block using DIMD mode. The template regions can be determined from the reconstructed regions surrounding the current coding block.

[0280] The template area at the top is called AboveTemplate, while the template area on the left is called LeftTemplate.

[0281] The AboveTemplate region has a horizontal length of W and a vertical length of AT, while the LeftTemplate region has a horizontal length of LT and a vertical length of H. The reference pixels for each template region can utilize R rows. Furthermore, the template region can be referred to as a template block.

[0282] It can explore intra-frame prediction modes that generate predicted pixel values ​​with the smallest error between the reconstructed pixel values ​​and the template region. It can determine the intra-frame prediction modes explored in the manner described above as the prediction information for the template region and use the corresponding prediction information as the prediction information for the current block.

[0283] Figure 20 This is a schematic diagram illustrating a method for encoding predictive information in a hybrid mode. Figure 20The explanation is given under the assumption that the current coded block is divided into prediction block A and prediction block B.

[0284] See Figure 20 In step S2001, the optimal prediction mode information for the current coding block can be encoded. As described above, the optimal coding mode for the current coding block can be determined by comparing the rate-distortion cost (RD-Cost) of the mixed mode, intra-frame prediction mode, and inter-frame prediction mode.

[0285] The specific method for determining the encoding mode by comparing rate-distortion costs (RD-Cost) will be discussed in the following sections. Figure 21 Please provide an explanation.

[0286] Figure 21 This is a flowchart illustrating the method for determining the optimal encoding mode in the image encoding device 100 from in-frame prediction mode, inter-frame prediction mode, and hybrid mode.

[0287] See Figure 21 In step S2101, prediction index information can be assigned to three different encoding modes. As an example, 0 can be assigned as the intra-frame prediction mode index, 1 as the inter-frame prediction mode index, and 2 as the mixed mode index.

[0288] First, in step S2102, the initial value of N can be set to 0 and the initial value of Best_Cost can be set to infinity. Then, the rate-distortion code (RD-Cost) of the prediction mode in the frame is calculated for index 0 and saved to N_Cost.

[0289] Next, in step S2103, it is possible to determine whether N_Cost is larger or smaller by comparing it with the current Best_Cost. When N_Cost is smaller, in step S2104, the current N value can be saved as Best_Idx and N_Cost can be saved as Best_Cost.

[0290] Next, in step S2105, it is possible to compare whether the current N is 2. If the current N is not 2, in step S2106, 1 is added to the current N and the process is repeated in step S2102. If the current N is 2, in step S2107, the current Best_Idx is determined as the best encoding mode index information for the current encoding block, and then the process ends. Figure 21 The process in.

[0291] Return to the Figure 20The explanation is as described above. Figure 21 In the process where the index information of the mixed mode is determined as the best encoding mode, in step S2001, the mixed mode information can be encoded as the best encoding mode information for the current encoding block.

[0292] In addition, it is possible to pre-transmit information on whether or not the mixed mode is active in the upper-level header, and to control whether or not the mixed mode is active in the upper-level header layer that is being transmitted.

[0293] In step S2002, it can be determined whether the optimal encoding mode of the current encoding block determined in step S2001 is a mixed mode. If the optimal encoding mode of the current encoding block is a mixed mode, the process can proceed to step S2003; if the optimal encoding mode of the current encoding block is not a mixed mode, the process can proceed to step S2006.

[0294] In step S2006, it can be determined whether the optimal coding mode for the current coding block is the intra-frame prediction mode. If the optimal coding mode for the current coding block is the intra-frame prediction mode, in step S2007, it can be determined by... (as described above) Figure 3 The same process described above is used to encode the in-frame prediction mode information. If the optimal coding mode for the current coding block is not an in-frame prediction mode, in step S2008, it is possible to encode the in-frame prediction mode information using the method described above. Figure 9 The same process described above encodes the prediction mode information between images.

[0295] Next, in step S2003, the segmentation information of the current coded block can be encoded. At this time, when the coded block is encoded according to... Figure 13 When the method illustrated is divided into two parts, the P information can be encoded after the information indicating horizontal or vertical division is encoded, and the information indicating the division as described above can also be encoded. Figure 14 The index information of the optimal segmentation pattern in the pre-defined segmentation group shown in the figure is encoded.

[0296] Furthermore, when the encoded block is in accordance with such Figure 15 When the illustrated method is segmented by a corner, the index information used to indicate which direction the segmented corner is located in can be encoded. That is, when the encoded block is segmented with a width of 1 / 4 or 3 / 4, the index information used to indicate which of the four directions the predicted block with a width of 1 / 4 is located in can be encoded.

[0297] In step S2004, the prediction information for predicted block A is encoded. Among the prediction information of surrounding blocks, the index information of the surrounding blocks with the best prediction information for predicted block A is encoded. This will be discussed in subsequent sections. Figure 22 A detailed explanation will be provided.

[0298] Figure 22 This is a flowchart illustrating the encoding method for the prediction information of prediction block A in the image encoding device 100.

[0299] See Figure 22 In step S2201, the rate-distortion cost (RD-Cost) of different prediction information can be calculated after generating a prediction block by applying the prediction information of available surrounding blocks to prediction block A.

[0300] Next, in step S2202, the prediction information with the lowest rate-distortion cost (RD-Cost) among the applicable prediction information can be determined, and in step S2203, the index information of the surrounding blocks with the corresponding prediction information can be saved.

[0301] In addition, with Figure 22 Unlike other methods, it can also avoid performing a comparison of rate-distortion cost (RD-Cost) and instead directly use the prediction information of the highest priority surrounding block among the available surrounding blocks based on the priority of surrounding blocks.

[0302] Return to the Figure 20 As explained in step S2205, the prediction information for prediction block B can be encoded.

[0303] An image encoding apparatus according to one embodiment of the present invention can determine the optimal prediction mode for prediction block B by considering only one pre-set prediction mode among five prediction modes: Merge mode, Advanced Motion Vector Prediction (AMVP) mode, Decoder-Side Motion Information Derivation (DMID) mode, In-Frame Prediction mode, and Decoder-Side In-Frame Pattern Derivation (DIMD) mode.

[0304] In the case described above, if the preset prediction mode is Merge mode or Advanced Motion Vector Prediction (AMVP) mode (inter-frame prediction mode), then it can be performed as described above. Figure 9 The detailed prediction information is encoded using the same method as described above. If the preset prediction mode is in-frame prediction mode, it can be encoded according to the method described above. Figure 3 The same encoding method is used to encode detailed prediction information.

[0305] Furthermore, if the preset prediction mode is decoder-side motion information derivation (DMID) mode, it can be performed according to... Figure 23 The method illustrated is used to encode the prediction information.

[0306] Figure 23 This is a flowchart illustrating the predictive information encoding method that utilizes the decoder-side motion information derivation (DMID) mode.

[0307] See Figure 23 In step S2301, decoder-side motion information derivation (DMID) action information can be encoded. The decoder-side motion information derivation (DMID) action information can be used to indicate whether to use decoder-side motion information derivation (DMID) to encode the prediction information.

[0308] In step S2302, it is possible to determine whether the decoder-side motion information derivation (DMID) action is true or false based on the decoder-side motion information derivation (DMID). If the decoder-side motion information derivation (DMID) action is false, the flowchart ends; if the decoder-side motion information derivation (DMID) is true, the process jumps to step S2303.

[0309] In step S2303, the predicted direction can be encoded. This is compatible with the method described above. Figure 9 The description of step S904 is the same.

[0310] In step S2304, it can be determined whether the predicted direction is a future direction. If the predicted direction is not a future direction, the process can jump to step S2305; if the predicted direction is a future direction, the process can jump to step S2308.

[0311] In step S2305, motion vectors in the historical direction can be encoded. Before the image decoding device derives motion vectors through motion inference, it can receive motion vectors with lower precision from the image encoding device and derive motion vectors with final precision (higher precision) in the image decoding device using the corresponding motion vectors as initial motion vectors.

[0312] Furthermore, it is also possible to omit the transmission of motion vectors (i.e., step S2305), and instead directly derive motion vectors with final precision using motion vectors from the reconstructed motion information of the surrounding area as the initial motion vectors for motion inference in the image decoding device. This is comparable to the method described above. Figure 9 The description of step S907 in the text is different.

[0313] In step S2306, the reference image index information for the historical direction can be encoded. This can be combined with the information described above. Figure 9 The description of step S906 is the same. However, it is also possible to determine the reference image index information of the historical direction by judgment in the image decoding device without encoding the reference image index information of the historical direction.

[0314] In step S2307, the motion vector prediction (MVP) information of the historical direction can be encoded. This enables... Figure 9 The description of step S907 is the same. However, it is also possible to determine the motion vector prediction (MVP) information of the historical direction by judgment in the image decoding device without encoding the motion vector prediction (MVP) information of the historical direction, and it is also possible to determine the optimal motion information of the historical direction by motion inference without using the motion vector prediction (MVP) information.

[0315] In step S2308, it can be determined whether the predicted direction is a historical direction. If the predicted direction is a historical direction, the flowchart ends; otherwise, in step S2309, the motion vector for the future direction can be encoded. This is consistent with the above-described... Figure 9 The description of step S911 differs from that in the previous section. Before the image decoding device derives the motion vector through motion inference, it is possible to receive a motion vector with lower precision from the image encoding device and derive a motion vector with final precision (higher precision) in the image decoding device using the corresponding motion vector as the initial motion vector.

[0316] Alternatively, instead of transmitting the motion vector (i.e., omitting step S2309), the motion vector with final precision can be directly derived in the image decoding device using the motion vector of the reconstructed motion information of the surrounding area as the initial motion vector for motion inference.

[0317] In step S2310, reference image index information for the future direction can be encoded. This is compatible with the above-described... Figure 9 The description of step S910 is the same. However, it is also possible to determine the reference image index information of the future direction in the image decoding device without encoding the reference image index information of the future direction.

[0318] In step S2311, the motion vector prediction (MVP) information for the future direction can be encoded. This enables... Figure 9 The description of step S911 is the same. However, it is also possible to determine the motion vector prediction (MVP) information of the future direction by judgment in the image decoding device without encoding the motion vector prediction (MVP) information of the future direction, and it is also possible to determine the optimal motion information of the future direction by motion inference without using the motion vector prediction (MVP) information.

[0319] Return to the Figure 20The description of step S2205 applies to an image coding apparatus according to another embodiment of the present invention, which can determine the optimal prediction mode for prediction block B by considering two or more prediction modes among five prediction modes: Merge mode, Advanced Motion Vector Prediction (AMVP) mode, Decoder-Side Motion Information Derivation (DMID) mode, In-Frame Prediction mode, and Decoder-Side In-Frame Pattern Derivation (DIMD) mode. This will be discussed in detail later. Figure 24 A detailed explanation will be provided.

[0320] Figure 24 This is a flowchart illustrating the method for determining the prediction mode of prediction block B in an image coding apparatus.

[0321] See Figure 24 In step S2401, index information can be assigned to different prediction modes. As an example, the Advanced Motion Vector Prediction (AMVP) mode can be assigned as number 0, the Merge mode as number 1, the Decoder-Side Motion Information Derivation (DMID) mode as number 2, the In-Frame Prediction mode as number 3, and the Decoder-Side In-Frame Mode Derivation (DIMD) mode as number 4.

[0322] In step S2402, the range of prediction modes considered in the image coding device can be determined by M. For example, when M is 2, the prediction mode of prediction block B can be determined using only three prediction modes: Advanced Motion Vector Prediction (AMVP), Merge, and Decoder-Side Motion Information Inference (DMID).

[0323] First, in step S2403, the initial value of N can be set to 0 and the initial value of Best_Cost can be set to infinity. Then, the rate-distortion cost (RD-Cost) of mode 0 is calculated and saved to N_Cost.

[0324] In step S2404, it can be determined whether the current N_Cost is less than the saved Best_Cost. When N_Cost is less than Best_Cost, in step S2405, the current N can be saved as Best_Idx and N_Cost can be saved as Best_Cost.

[0325] In step S2406, it can be determined whether the current N is equal to the maximum value M of the prediction mode range. When the corresponding information is different, in step S2407, the value of N can be incremented by 1 and the process can be repeated in step S2403. When the corresponding information is the same, in step S2408, the best prediction mode index of the current prediction block can be saved to Best_Idx, and then the flowchart ends.

[0326] Next, the prediction information of the prediction mode corresponding to the corresponding index information can be encoded after the Best_Idx information is encoded.

[0327] When the prediction mode is Advanced Motion Vector Prediction (AMVP) or Merge mode (inter-frame prediction mode), it can be achieved through the methods described above. Figure 9 The same process described above encodes the prediction information. When the prediction mode is in-frame prediction mode, it can be encoded using the same method as described above. Figure 3 The same process described above encodes the prediction information.

[0328] Furthermore, when the prediction mode is the decoder-side motion information derivation (DMID) mode, it can be achieved through the methods described above. Figure 23 The same process described above encodes the prediction information.

[0329] The above content introduces the encoding method of prediction information in the image coding device when the current coding block is in mixed mode.

[0330] Next, we will combine Figure 25 as well as Figure 26 This paper introduces a method for decoding prediction information in an image decoding device when the current coded block is in mixed mode. Figure 25 In this context, a decoding block can refer to an encoding block.

[0331] Figure 25 This is a flowchart illustrating the decoding method for prediction information in a mixed mode. Figure 25 The explanation is given under the assumption that the current coded block is divided into prediction block A and prediction block B.

[0332] See Figure 25 In step S2501, the encoding mode information of the current decoding block can be decoded. The encoding mode information of the current encoding block can be information indicating one of the following: intra-frame prediction mode, inter-frame prediction mode, or mixing mode.

[0333] In step S2501, it is possible to determine whether the encoding mode of the current encoding block is a mixed mode based on the decoded prediction mode information. If the encoding mode of the current encoding block is a mixed mode, the process can proceed to step S2503; if the encoding mode of the current encoding block is not a mixed mode, the process can proceed to step S2506.

[0334] In step S2506, it can be determined whether the encoding mode of the current encoding block is intra-frame prediction mode. If the encoding mode of the current encoding block is intra-frame prediction mode, in step S2507, it can be determined whether the encoding mode of the current encoding block is intra-frame prediction mode. Figure 20 In step S2007, the encoded in-frame prediction mode information is decoded. At this time, the method for decoding the in-frame prediction mode information can be compared with the method described above. Figure 11 The explanation is the same.

[0335] If the encoding mode of the current encoding block is not in-frame prediction mode, in step S2508, it is possible to... Figure 20 In step S2008, the encoded inter-frame prediction mode information is decoded. At this time, the method for decoding the inter-frame prediction mode information can be compared with the method described above. Figure 12 The explanation is the same.

[0336] Next, in step S2503, it is possible to perform the operation described above. Figure 20 In step S2003, the segmentation information of the current encoded block is decoded.

[0337] Next, in step S2504, it is possible to perform the operation described above. Figure 20 In step S2004, the prediction information of the encoded prediction block A is decoded.

[0338] Next, in step S2505, it is possible to perform the operation described above. Figure 20 In step S2005, the prediction information of the encoded prediction block B is decoded. Specifically, when the prediction mode of prediction block B is decoder-side motion information derivation (DMID) mode, it can be decoded according to... Figure 26 The method illustrated is used to decode the prediction information.

[0339] Figure 26 This is a flowchart illustrating the predictive information decoding method that utilizes the Decoder-Side Motion Information Inference (DMID) mode.

[0340] See Figure 26 In step S2601, it is possible to perform the above-described... Figure 23 In step S2301, the decoder-side motion information derivation (DMID) is encoded to decode whether the action is active or not.

[0341] In step S2602, the operation of the decoder-side motion information derivation (DMID) mode can be determined based on the decoded information regarding whether the decoder-side motion information derivation (DMID) mode is active. The flowchart ends when the decoder-side motion information derivation (DMID) mode is inactive, and proceeds to step S2603 when the decoder-side motion information derivation (DMID) mode is active.

[0342] In step S2603, it is possible to perform the above-described... Figure 23In step S2303, the encoded prediction direction is decoded. This step can be omitted when the prediction direction information is not encoded.

[0343] In step S2604, it is possible to determine whether the predicted direction is a future direction based on the decoded predicted direction information. If the predicted direction is not a future direction, the process can proceed to step S2605; if the predicted direction is a future direction, the process can proceed to step S2608.

[0344] In step S2605, it is possible to [do something] in [the context of] [the process]. Figure 23 Step S2305 decodes the motion vector of the encoded historical direction. This step can be omitted when no corresponding information is encoded.

[0345] In step S2606, it is possible to [do something] in [the context of] [the process]. Figure 23 In step S2306, the historical direction reference image index information is decoded. This step can be omitted when no corresponding information is encoded.

[0346] In step S2607, it is possible to [do something] in [the context of] [the process]. Figure 23 Step S2307 in the code involves decoding the encoded historical direction motion vector prediction (MVP) information. This step can be omitted when no such information is encoded.

[0347] In step S2608, it is possible to determine whether the predicted direction is a historical direction based on the decoded predicted direction information. If the predicted direction is a historical direction, the flowchart ends; if the predicted direction is not a historical direction, in step S2609, it is possible to... Figure 23 Step S2309 in the code decodes the motion vector of the encoded future direction. This step can be omitted when no corresponding information is encoded.

[0348] In step S2610, it is possible to [do something] in [the context of] [the process]. Figure 23 In step S2310, the reference image index information for the future direction is decoded. This step can be omitted when no such information is encoded.

[0349] In step S2611, it is possible to [do something] in [the context of] [the process]. Figure 23 Step S2311 decodes the encoded future direction motion vector prediction (MVP) information. This step can be omitted when no such information is encoded.

[0350] Figure 27 This is a schematic diagram illustrating a boundary filtering method applicable to one embodiment of the present invention.

[0351] Predicted blocks A and B generated within the current coded block may exhibit blockiness in the boundary regions. Therefore, a smoothing filter process is needed to soften these boundary regions.

[0352] See Figure 27 After determining the prediction modes of prediction blocks A and B within the current coding block and generating prediction blocks for the two blocks, filtering can be performed at the boundary between the two prediction blocks.

[0353] Figure 27 Illustrations 2701 and 2702 illustrate pixels that require filtering at the boundaries between prediction blocks when the prediction block within the coded block is bisected. Filtering is performed on pixels adjacent to the prediction block boundaries, with a filter tap of 4 and filter coefficients W1 to W4. W1 to W4 are arbitrary real numbers, and W1+W2+W3+W4 is always equal to 1. Formula 2 is the filtering formula applicable to 2701 and 2702.

[0354] [Formula 2]

[0355]

[0356] Figure 27 Figures 2703-2706 illustrate pixels that require filtering at the boundaries between prediction blocks when the prediction blocks within a coded block are corner-segmented. In the case described above, diagonal filtering can be applied to pixels e1, e2, e4, and e5 at the corner boundaries of the prediction blocks. Pixels simultaneously labeled d1 and b1 indicate that b1 and d1 are the same pixel.

[0357] Formula 3 is applicable Figure 27 The filtering formulas for 2703 to 2706 are shown in the figure.

[0358] [Formula 3]

[0359]

[0360] Referring to Formula 3, it can be confirmed that diagonal filtering was applied to pixels e1, e2, e4, and e5 at the corner boundaries.

[0361] Figure 28 This is a flowchart illustrating the image encoding method applicable to the first embodiment of the present invention.

[0362] See Figure 28 In step S2801, the image encoding device can divide the current encoding block into at least one prediction block.

[0363] Next, in step S2802, the image encoding device can generate a prediction sample of at least one prediction block based on the encoding mode information of the current encoding block.

[0364] The coding mode information can be information used to indicate one of the intra-frame mode, inter-frame mode, and hybrid mode.

[0365] Furthermore, if the coding mode information of the current coding block indicates a mixed mode, the image coding device can divide the current coding block into a first prediction block and a second prediction block.

[0366] In the scenario described above, the image coding apparatus can generate prediction samples for the first prediction block using prediction information from at least one of the surrounding blocks of the first prediction block. Furthermore, to generate prediction samples for the second prediction block, the image coding apparatus can use prediction information derived based on one of the following modes: intra-frame prediction mode, inter-frame prediction mode, decoder-side motion information derivation (DMID) mode, and decoder-side intra-mode derivation mode.

[0367] The intra-frame prediction modes can include Planar mode, DC mode, and n-angle prediction modes.

[0368] Among them, the inter-frame prediction modes can include Advanced Motion Vector Prediction (AMVP) mode, Merge mode, and Skip mode.

[0369] Furthermore, when the encoding mode information of the current encoding block indicates a mixed mode, the image encoding device can determine the prediction mode of the second prediction block and encode the prediction mode information of the second prediction block.

[0370] Furthermore, at least one predicted block can be corner-splitting within the aforementioned current encoded block.

[0371] As described above, diagonal filtering can be applied to pixels at the corner boundaries of the predicted block. This has been incorporated into the above content. Figure 27 Detailed explanations have been provided, so the relevant detailed explanations will be omitted here.

[0372] Figure 29 This is a flowchart illustrating the image decoding method applicable to the first embodiment of the present invention.

[0373] See Figure 29In step S2901, the image decoding device can decode the encoding mode information of the current encoding block.

[0374] The aforementioned encoding mode information can be used to indicate one of the intra-frame mode, inter-frame mode, and hybrid mode.

[0375] Next, in step S2902, the image decoding device is able to divide the current coded block into at least one prediction block.

[0376] If the encoding mode information of the current encoding block indicates a mixed mode, the image decoding device can divide the current encoding block into a first prediction block and a second prediction block.

[0377] Next, in step S2903, prediction samples for at least one prediction block can be generated based on the decoded encoding mode information.

[0378] Specifically, when the coding mode information of the current coding block indicates a mixed mode, the image decoding device can generate a prediction sample of the first prediction block using the prediction information of at least one of the surrounding blocks of the first prediction block, and generate a prediction sample of the second prediction block using the prediction information derived based on one of the following modes: intra-prediction mode, inter-prediction mode, decoder-side motion information derivation (DMID) mode, and decoder-side intra-mode derivation (DIMD) mode.

[0379] Under the circumstances described above, the prediction information of the second prediction block can be derived by decoding the prediction mode information of the second prediction block.

[0380] Furthermore, the aforementioned decoder-side motion information derivation (DMID) mode can refer to a mode that derives final motion information based on initial motion information derived from the surrounding blocks of the coded block and uses the derived final motion information as prediction information for prediction blocks within the coded block.

[0381] In addition, the decoder-side intra-frame pattern derivation (DIMD) mode can set template blocks in the reconstruction area around the coding block and use the intra-frame prediction information that can derive the best predicted pixel value of the template block as the prediction information of the prediction block within the coding block.

[0382] The optimal predicted pixel value for the template block refers to the predicted pixel value with the smallest difference (or the smallest error) compared to the reconstructed pixel value of the template block. Furthermore, the optimal predicted pixel value for the template block can be generated using surrounding reference pixels of the template block.

[0383] As described above, diagonal filtering can be applied to pixels at the corner boundaries of the predicted block. This has been incorporated into the above content. Figure 27 Detailed explanations have been provided, so the relevant detailed explanations will be omitted here.

[0384] In addition, through Figure 28 The bitstream generated by the image encoding method can be recorded into a storage medium.

[0385] Figure 31 This is a sequence diagram illustrating the encoding method for predicting information of the current coded block to which the second embodiment of the present invention applies. Figure 31 In this context, it is assumed that the current coding block is divided into prediction block A and prediction block B, and the coding mode of the current coding block is the inter-picture prediction mode.

[0386] See Figure 31 In step S3101, the inter-frame mixing mode information of the current coding block can be encoded. The inter-frame mixing mode information can be information used to indicate whether the coding block is encoded in inter-frame mixing mode.

[0387] By comparing the rate-distortion cost (RD-Cost) of the inter-frame blending mode and the inter-frame prediction mode, the inter-frame blending mode information of the current coding block can be encoded as true when the RD-Cost of the inter-frame blending mode is minimized. The inter-frame blending mode information can be encoded in the form of a flag.

[0388] Figure 32 This is a flowchart illustrating the encoding method of inter-frame mixing mode information in the image encoding device 100.

[0389] See Figure 32 In step S3201, it can first be determined whether the inter-screen prediction mode action is true. Specifically, if the inter-screen prediction mode action is false, this flowchart ends; if it is true, it can jump to step S1702.

[0390] In step S3202, the index information of the inter-frame prediction mode is assigned to 0 and the index information of the inter-frame mixing mode is assigned to 1.

[0391] In step S3203, the initial value of N can be set to 0 and the initial value of Best_Cost can be set to infinity. Then, the rate-distortion code (RD-Cost) of the prediction mode in the frame is calculated and saved to N_Cost.

[0392] In step S3204, it is possible to determine whether N_Cost is larger or smaller by comparing it with the current Best_Cost. When N_Cost is smaller, in step S3205, the current N value can be saved as Best_Idx and N_Cost can be saved as Best_Cost.

[0393] In step S3206, a comparison is made to determine if the current N is 1. If the current N is not 1, in step S3207, 1 is added to the current N and the process is repeated in step S3203. If the current N is 1, in step S3208, the current Best_Idx is determined as the inter-frame mixing mode information of the current coding block, and then the process ends. Figure 32 The process in.

[0394] In step S3101, the inter-frame mixing mode information generated through the corresponding process is encoded as the inter-frame mixing mode information of the current coding block. In addition, information regarding the operation or non-operation of the inter-frame mixing mode can be appended and transmitted in the upper-level header, and the operation or non-operation of the inter-frame mixing mode can be controlled within the transmitted upper-level header layer.

[0395] Return to the Figure 31 As explained in step S3102, it is possible to determine whether the current coding block is in inter-frame mixing mode based on the inter-frame mixing information of the current coding block.

[0396] If the current coding block is in inter-frame mixing mode, in step S3103, the segmentation information of the current coding block can be encoded.

[0397] At this time, when the coded block is in accordance with... Figure 13 When the method illustrated in the example diagram is divided into two parts, it is possible to encode the P information after encoding the information used to indicate horizontal or vertical division, and it is also possible to encode the information used to indicate as described above. Figure 14 The example diagram illustrates the encoding of the index information of the optimal segmentation pattern in a pre-defined segmentation group. Furthermore, when following the example diagram... Figure 15 When the example diagram illustrates the method of 1 / 4 and 3 / 4 segmentation, the index information used to indicate which of the four directions the 1 / 4 prediction block is located in can be encoded, thereby determining the segmentation pattern of the current encoded block.

[0398] In step S3104, the prediction information for prediction block A can be encoded. The prediction information for prediction block A is determined using a merge mode, and the encoding method can be found in the above description. Figure 5 Related content.

[0399] Next, in step S3104, it is also possible to not use the (Merge) mode, but instead directly use the prediction information of the highest priority surrounding block among the available surrounding blocks according to the priority order of the surrounding blocks.

[0400] In step S3105, the prediction information of prediction block B can be encoded.

[0401] An image encoding apparatus according to one embodiment of the present invention can determine the best prediction mode for prediction block B by considering only one pre-set prediction mode among three prediction modes: Merge mode, Advanced Motion Vector Prediction (AMVP) mode, and Decoder-Side Motion Information Inference (DMID) mode.

[0402] In the scenario described above, when the preset prediction mode is either Merge mode or Advanced Motion Vector Prediction (AMVP) mode, it can perform the prediction as described above. Figure 6 The same encoding method is used to encode detailed prediction information.

[0403] Furthermore, if the preset prediction mode is decoder-side motion information derivation (DMID) mode, it can be performed according to... Figure 23 The method illustrated is used to encode the prediction information.

[0404] Return to the Figure 31 The description of step S3105 applies to an image encoding apparatus according to another embodiment of the present invention, which can determine the optimal prediction mode for prediction block B by considering two or more prediction modes among three prediction modes: Merge mode, Advanced Motion Vector Prediction (AMVP) mode, and Decoder-Side Motion Information Inference (DMID) mode. This will be discussed in detail later. Figure 19 A detailed explanation will be provided.

[0405] Figure 33 This is a flowchart illustrating a method for determining the prediction mode of prediction block B in an image coding apparatus to which the second embodiment of the present invention applies.

[0406] See Figure 33In step S3301, index information can be assigned to different prediction modes. As an example, the Advanced Motion Vector Prediction (AMVP) mode can be assigned as number 0, the Merge mode as number 1, and the Decoder-Side Motion Information Derivation (DMID) mode as number 2.

[0407] In step S3302, the range of prediction modes considered in the image coding device can be determined by M. For example, when M is 2, the prediction mode of prediction block B can be determined using only three prediction modes: Advanced Motion Vector Prediction (AMVP), Merge, and Decoder-Side Motion Information Inference (DMID).

[0408] First, in step S3303, the initial value of N can be set to 0 and the initial value of Best_Cost can be set to infinity. Then, the rate-distortion cost (RD-Cost) of mode 0 is calculated and saved to N_Cost.

[0409] In step S3304, it can be determined whether the current N_Cost is less than the saved Best_Cost. When N_Cost is less than Best_Cost, in step S3305, the current N can be saved as Best_Idx and N_Cost can be saved as Best_Cost.

[0410] In step S3306, it can be determined whether the current N is equal to the maximum value M of the prediction mode range. When the corresponding information is different, in step S3307, the value of N can be incremented by 1 and the process can be repeated in step S3303. When the corresponding information is the same, in step S3308, the best prediction mode index of the current prediction block can be saved to Best_Idx, and then the flowchart ends.

[0411] Next, the prediction information of the prediction mode corresponding to the corresponding index information can be encoded after the Best_Idx information is encoded.

[0412] When the prediction mode is Advanced Motion Vector Prediction (AMVP) mode or Merge mode, it can be achieved through the methods described above. Figure 6 The same process described above encodes the prediction information. Furthermore, when the prediction mode is the decoder-side motion information derivation (DMID) mode, it is possible to encode the prediction information using the same method as described above. Figure 23 The same process described above encodes the prediction information.

[0413] The above content introduces the encoding method of prediction information in the image coding device when the current coding block is in inter-frame mixing mode.

[0414] Next, we will combine Figure 34 A method for decoding the prediction information of the current coded block in an image decoding apparatus to which the second embodiment of the present invention applies will be described. Figure 34 In this context, it is assumed that the encoding mode of the current encoding block is the inter-frame prediction mode. Here, the current decoding block can refer to the current encoding block.

[0415] See Figure 34 In step S3401, the inter-frame mixing mode information of the current decoding block can be decoded.

[0416] In step S3402, it is possible to determine whether the current decoded block is in inter-frame mixing mode based on the decoded inter-frame mixing mode information.

[0417] If the current decoding block is in inter-frame mixing mode, in step S3403, the segmentation information of the current decoding block can be decoded.

[0418] In step S3403, it is possible to perform the above-described... Figure 31 In step S3103, the segmentation information of the currently encoded block is decoded. At this point, the decoded segmentation information can be used to segment the currently decoded block. This has already been addressed in the above content. Figures 13 to 15 Detailed explanations have been provided, so the relevant detailed explanations will be omitted here.

[0419] In step S3404, it is possible to perform the above-described Figure 31 In step S3104, the prediction information of the predicted block A, which is encoded, is decoded. Specifically, in step S3104, the prediction information of the predicted block A can be derived from the prediction information of the surrounding blocks selected based on the index information (or merge index information) of the surrounding blocks. In addition, the prediction information of the surrounding block with the highest priority among the available surrounding blocks can be derived as the prediction information of the predicted block A according to the pre-set priority order of the surrounding blocks.

[0420] In step S3404, it is possible to perform the above-described Figure 31 In step S3105, the prediction information of the encoded prediction block B is decoded. Specifically, when the prediction mode of prediction block B is decoder-side motion information derivation (DMID) mode, it can be decoded according to... Figure 26 The method illustrated is used to decode the prediction information.

[0421] Figure 35 This is a flowchart illustrating the image decoding method applicable to the second embodiment of the present invention.

[0422] See Figure 35In step S3501, the image decoding device is able to decode the inter-frame mixing mode information of the coded block.

[0423] Next, in step S3502, the image decoding device can generate prediction samples for at least one prediction block within the current coded block based on inter-frame mixing mode information. For step S3502, this will be combined with... Figure 24 A detailed explanation will be provided.

[0424] Figure 36 This is a flowchart illustrating the image decoding method applicable to the second embodiment of the present invention.

[0425] See Figure 36 If the current coding block is in inter-frame mixing mode (step S3601 - Yes), in step S3602, the image decoding device can divide the current coding block into a first prediction block and a second prediction block.

[0426] Among them, the first predicted block and the second predicted block can be as follows: Figure 11 The diagram illustrates the shape of corner segmentation of the current coded block.

[0427] In the case described above, such as in Figure 27 The explanation provided indicates that diagonal filtering can be applied to pixels adjacent to the corner boundary of the first prediction block.

[0428] Next, in step S3603, the image decoding device can derive the motion information of the first prediction block using the Merge mode and generate a prediction sample of the first prediction block using the derived motion information of the first prediction block.

[0429] Next, in step S3604, the image decoding device can derive the motion information of the second prediction block using one of the Merge mode, Advanced Motion Vector Prediction (AMVP) mode, and Decoder-side Motion Information Derivation (DMID) mode, and generate a prediction sample of the second prediction block using the derived motion information of the second prediction block.

[0430] The aforementioned decoder-side motion information derivation (DMID) mode can refer to a mode that derives final motion information based on initial motion information derived from the surrounding blocks of the coded block and uses the derived final motion information as prediction information for prediction blocks within the coded block.

[0431] Furthermore, the image decoding method can determine the prediction mode used to derive the motion information of the second prediction block by decoding the inter-frame prediction mode information of the second prediction block. The inter-frame prediction mode information of the second prediction block can be information received from signaling of the image encoding device.

[0432] Figure 37 This is a flowchart illustrating the image encoding method applicable to the second embodiment of the present invention.

[0433] See Figure 37 In step S3701, the image encoding device can encode the inter-frame mixing mode information of the current encoding block.

[0434] Next, in step S3702, the image coding device is able to generate prediction samples for at least one prediction block within the current coding block based on inter-frame mixing mode information.

[0435] Furthermore, step S3702 is explained in detail below. When the current coding block is in inter-frame mixing mode, the image coding device can divide the current coding block into a first prediction block and a second prediction block, and derive the motion information of the first prediction block using the merge mode. Then, it generates a prediction sample for the first prediction block using the derived motion information of the first prediction block. Next, the image coding device can derive the motion information of the second prediction block using one of the merge mode, the advanced motion vector prediction (AMVP) mode, and the decoder-side motion information derivation (DMID) mode, and generate a prediction sample for the second prediction block using the derived motion information of the second prediction block.

[0436] In addition, through Figure 37 The bitstream generated by the image encoding method can be recorded into a storage medium.

[0437] Next, an intra-frame blending mode applicable to one embodiment of the present invention will be described.

[0438] Intra-frame blending mode is one of the intra-frame prediction modes. When the coding mode of a coded block is intra-frame blending mode within the intra-frame mode, the prediction information of the first prediction block (prediction block A) can be directly determined using the intra-frame prediction information of the surrounding blocks of the coded block, and the prediction information of the second prediction block (prediction block B) can be determined according to... Figure 3The illustrated method uses either predictive block unit determination or Decoder-side Intra Mode Derivation (DIMD) mode to predict block unit determination.

[0439] Among them, the decoder-side intra-frame prediction mode (DIMD) can refer to a prediction mode that is not encoded in the image encoding device 1000 but is directly derived in the image decoding device 1000 to generate prediction samples of prediction blocks.

[0440] As an example, the decoder-side intra-frame mode derivation (DIMD) mode is a mode in which the image decoding device 1000 derives the intra-frame prediction mode from the peripheral blocks of the coded block and, based on the derived intra-frame prediction mode, derives the final intra-frame prediction mode of the prediction block to generate prediction samples of the prediction block.

[0441] Figure 30 This section describes a method for exploring prediction information in decoder-side intra-mode derivation (DIMD) mode. It provides examples of surrounding template regions with different segmentation patterns when exploring the prediction mode of prediction block B generated within the current coding block using DIMD mode. The template regions can be determined from the reconstructed regions surrounding the current coding block.

[0442] The template area at the top is called AboveTemplate, while the template area on the left is called LeftTemplate.

[0443] The AboveTemplate region has a horizontal length of W and a vertical length of AT, while the LeftTemplate region has a horizontal length of LT and a vertical length of H. The reference pixels for each template region can utilize R rows. Furthermore, the template region can be referred to as a template block.

[0444] It can explore intra-frame prediction modes that generate predicted pixel values ​​with the smallest error between the reconstructed pixel values ​​and the template region. It can determine the intra-frame prediction modes explored in the manner described above as the prediction information for the template region and use the corresponding prediction information as the prediction information for the current block.

[0445] Furthermore, when the coded block of the in-frame mode is not in intra-frame mixing mode, it can be processed according to... Figure 3 The method illustrated is determined by the predicted block unit.

[0446] Furthermore, the method for directly using the in-frame prediction information of surrounding blocks to determine the prediction information of prediction block A is as follows. Here, the prediction information can refer to the in-frame prediction mode.

[0447] The prediction block A can explore the best prediction information of the prediction block A from the prediction information of the neighboring blocks that have been encoded and determine the best prediction information as the prediction information of the prediction block A, thereby generating the prediction block.

[0448] Furthermore, as a method for determining the prediction information of prediction block B, it is possible to use the reference pixels around the prediction block to find the final prediction mode among the planar, mean (DC), and N angle (Angular) modes and generate the prediction block using the corresponding prediction information, or to use the decoder-side intra-frame mode derivation (DIMD) mode to find the best prediction mode and generate the prediction block using the corresponding prediction information.

[0449] However, in some cases, the prediction information for block B can be determined by referring to the prediction information for block A. The following will explain some of these cases.

[0450] In the image decoding device 1000, the decoder-side motion information derivation (DMID) mode can explore prediction information by assuming the difference between the reconstructed pixel values ​​of the template region and the predicted pixel values ​​of the template region using reference pixels around the template region as the error used to calculate the rate distortion code (RD-Cost).

[0451] At this point, the prediction information for prediction block B can be determined by referencing the prediction information for prediction block A based on the segmentation structure of the transformed block.

[0452] If the current transform block's segmentation structure is a residual tree (RT) structure with coding unit (CU) units, prediction block B cannot refer to the prediction information of prediction block A to determine the prediction information of prediction block B.

[0453] Furthermore, if the current transformation block's segmentation structure is a residual tree (RT) structure of prediction unit (PU) units, prediction block B can refer to the prediction information of prediction block A to determine the prediction information of prediction block B.

[0454] Furthermore, the prediction information for block B can be determined in block units as described above. In the case described above, such as for... Figure 1The prediction unit 102 of the image encoding apparatus 100 is described in this paper, which can determine the optimal intra-frame prediction mode in units of prediction blocks and generate prediction blocks based on the determined intra-frame prediction mode or the decoder-side intra-frame mode derivation (DIMD) mode.

[0455] Figure 38 This is a schematic diagram illustrating the encoding method used to determine the prediction information of the current coded block. Figure 38 In this context, it is assumed that the current encoded block is divided into prediction block A and prediction block B, and the encoding mode of the current encoded block is in-frame mode.

[0456] See Figure 38 In step S3801, the intra-frame mixing mode information of the current coding block can be encoded. The intra-frame mixing mode information can be information used to indicate whether the coding block is encoded in intra-frame mixing mode.

[0457] By comparing the rate-distortion cost (RD-Cost) of intra-frame blending mode and intra-frame prediction mode, the intra-frame blending mode information of the current coding block can be encoded as true when the RD-Cost of intra-frame blending mode is minimized. The intra-frame blending mode information can be encoded in the form of a flag.

[0458] Figure 39 This is a flowchart illustrating the encoding method of intra-frame mixing mode information in the image encoding device 100.

[0459] See Figure 39 In step S3901, it can first be determined whether the predicted mode action in the image is true. Specifically, if the predicted mode action in the image is false, the flowchart ends; if it is true, the process jumps to step S3902.

[0460] In step S3902, the index information of the in-frame prediction mode is assigned to 0 and the index information of the in-frame blending mode is assigned to 1.

[0461] In step S3903, the initial value of N can be set to 0 and the initial value of Best_Cost can be set to infinity. Then, the rate-distortion code (RD-Cost) of the prediction mode in the frame is calculated for index 0 and saved to N_Cost.

[0462] In step S3904, it is possible to determine whether N_Cost is larger or smaller by comparing it with the current Best_Cost. When N_Cost is smaller, in step S3905, the current N value can be saved as Best_Idx and N_Cost can be saved as Best_Cost.

[0463] In step S3906, a comparison is made to determine if the current N is 1. If the current N is not 1, in step S3907, 1 can be added to the current N and the process can be repeated in step S3903. If the current N is 1, in step S3908, the current Best_Idx can be determined as the intra-frame mixing mode information of the current coding block, and then the process ends. Figure 39 The process in.

[0464] In step S3901, the intra-frame mixing mode information generated through the corresponding process is encoded as the intra-frame mixing mode information of the current coding block. In addition, the operation status information of the intra-frame mixing mode can be appended and transmitted in the upper-level header beforehand, and the operation status of the intra-frame mixing mode can be controlled in the transmitted upper-level header layer.

[0465] Return to the Figure 38 As explained in step S3802, it is possible to determine whether the current coding block is in intra-frame mixing mode based on the intra-frame mixing information of the current coding block.

[0466] If the current coding block is in intra-frame mixing mode, in step S3803, the segmentation information of the current coding block can be encoded.

[0467] At this time, when the coded block is in accordance with... Figure 13 When the method illustrated in the example diagram is divided into two parts, it is possible to encode the P information after encoding the information used to indicate horizontal or vertical division, and it is also possible to encode the information used to indicate as described above. Figure 14 The example diagram illustrates the encoding of the index information of the optimal segmentation pattern in a pre-defined segmentation group. Furthermore, when following the example diagram... Figure 15 When the example diagram illustrates the method of 1 / 4 and 3 / 4 segmentation, the index information used to indicate which of the four directions the 1 / 4 prediction block is located in can be encoded, thereby determining the segmentation pattern of the current encoded block.

[0468] In step S3804, the prediction information for predicted block A can be encoded. Among the prediction information of surrounding blocks, the index information of the surrounding blocks with the best prediction information for predicted block A can be encoded.

[0469] Figure 40 This is a flowchart illustrating the encoding method for the prediction information of the first prediction block within a coding block of an intra-frame mixing mode in an image coding apparatus.

[0470] See Figure 40In step S4001, after generating a prediction block by applying the in-frame prediction information of the available surrounding blocks to prediction block A, the rate-distortion cost (RD-Cost) of different prediction information is calculated.

[0471] In step S4002, the prediction information with the lowest rate-distortion cost (RD-Cost) among the applicable prediction information can be determined. In step S4003, the index information of the surrounding blocks with the corresponding prediction information can be saved.

[0472] In addition, it is also possible to not execute. Figure 40 Instead of going through the process described in the previous section, it directly uses the prediction information of the highest priority surrounding block among the available surrounding blocks based on the priority order of the surrounding blocks.

[0473] In step S3805, the prediction information for prediction block B is encoded. The prediction information for prediction block B can utilize the methods described above. Figure 3 Encoding is performed using in-frame predictive coding methods, or by transmitting decoder-side intra-frame mode derivation (DIMD) mode information (or flag information) that indicates the decoder-side intra-frame mode derivation (DIMD) mode.

[0474] In step S3806, since the current coding block is not in intra-frame mixing mode, the above-described method will be used. Figure 3 The in-frame predictive coding method in the image encodes prediction blocks.

[0475] Figure 41 This is a schematic diagram illustrating the decoding method used to explain the prediction information of the current decoding block. Figure 41 In this context, it is assumed that the current decoding block is divided into prediction block A and prediction block B, and the encoding mode of the current encoding block is in-frame mode. Here, the current decoding block can refer to the current encoding block.

[0476] See Figure 41 In step S4101, the intra-frame mixing mode information of the current decoding block can be decoded.

[0477] In step S4102, it is possible to determine whether the current decoded block is in intra-frame mixing mode based on the decoded intra-frame mixing mode information.

[0478] If the current decoding block is in intra-frame mixing mode, in step S4103, the segmentation information of the current encoding block can be encoded.

[0479] In step S4103, it is possible to perform the above-described... Figure 38In step S3803, the segmentation information of the currently encoded block is decoded. At this point, the decoded segmentation information can be used to segment the currently decoded block. This has been explained in the above content. Figures 13 to 15 Detailed explanations have been provided, so the relevant detailed explanations will be omitted here.

[0480] In step S4104, it is possible to perform the above-described Figure 38 In step S3804, the prediction information of the predicted block A is decoded. Specifically, in step S4104, the prediction information of the predicted block A can be derived from the prediction information of the surrounding blocks selected based on the index information of the surrounding blocks. In addition, the prediction information of the surrounding block with the highest priority among the available surrounding blocks can be derived as the prediction information of the predicted block A according to the preset priority order of the surrounding blocks.

[0481] In step S4105, it is possible to perform the above-described... Figure 38 In step S3805, the prediction information of the encoded prediction block B is decoded.

[0482] Furthermore, in step S4106, since the current decoding block is not in intra-frame mixing mode, the method described above will be utilized. Figure 11 The in-frame prediction decoding method in the image decodes in units of prediction blocks.

[0483] Figure 42 This is a flowchart illustrating the image decoding method applicable to the third embodiment of the present invention.

[0484] See Figure 42 In step S4201, the image decoding device is able to decode the intra-frame mixing mode information of the current coding block.

[0485] Next, in step S4202, the image decoding device can generate prediction samples for at least one prediction block within the current coded block based on intra-frame mixing mode information. For step S4202, this will be combined with... Figure 43 A detailed explanation will be provided.

[0486] Figure 43 This is a flowchart illustrating an image decoding method applicable to one embodiment of the present invention.

[0487] See Figure 43 If the current coding block is in intra-frame mixing mode (step S4301 - Yes), in step S4302, the image decoding device can divide the current coding block into a first prediction block and a second prediction block.

[0488] Among them, the first predicted block and the second predicted block can be as follows: Figure 15 The diagram illustrates the shape of corner segmentation of the current coded block.

[0489] In the case described above, such as in Figure 27 The explanation provided indicates that diagonal filtering can be applied to pixels adjacent to the corner boundary of the first prediction block.

[0490] Next, in step S4303, the image decoding device can generate a prediction sample for the first prediction block using the in-frame prediction mode of the first prediction block derived from the surrounding blocks of the first prediction block.

[0491] Furthermore, the image decoding device can deduce the in-frame prediction mode of the first prediction block by determining the in-frame prediction mode of the surrounding blocks of the first prediction block according to a preset priority order.

[0492] Alternatively, the image decoding device can deduce the intra-frame prediction mode of the surrounding blocks, determined based on the index information of the surrounding blocks, as the intra-frame prediction mode of the first prediction block mentioned above. The index information of the surrounding blocks can be information received from signaling of the image encoding device.

[0493] Next, in step S4304, the image decoding device can determine the intra-prediction mode of the second prediction block by decoding the intra-prediction mode information or the decoder-side intra-mode derivation (DIMD) mode information of the second prediction block, and generate prediction samples of the second prediction block using the determined intra-prediction mode.

[0494] Furthermore, if the current coded block is not in intra-frame mixing mode (step S4301 - No), in step S4305, the image decoding device can generate prediction samples for at least one prediction block within the current coded block through intra-frame prediction. That is, as in Figure 6 The explanation provided indicates that prediction samples can be generated in units of prediction blocks by deriving the prediction pattern within the image.

[0495] Figure 44 This is a flowchart illustrating an image encoding method applicable to one embodiment of the present invention.

[0496] See Figure 44 In step S4401, the image encoding device can encode the intra-frame mixing mode information of the current encoding block.

[0497] Next, in step S4402, the image coding device is able to generate prediction samples of at least one prediction block within the current coding block based on intra-frame mixing mode information.

[0498] Furthermore, step S4402 is explained in detail below. When the current coding block is in intra-frame mixing mode, the image coding device can divide the current coding block into a first prediction block and a second prediction block, and generate a prediction sample for the first prediction block using the intra-frame prediction mode of the first prediction block derived from the surrounding blocks of the first prediction block. Next, the image coding device can determine the intra-frame prediction mode of the second prediction block based on the intra-frame prediction mode information of the second prediction block or the decoder-side intra-frame mode derivation (DIMD) mode, and generate a prediction sample for the second prediction block using the determined intra-frame prediction mode.

[0499] In addition, through Figure 44 The bitstream generated by the image encoding method can be recorded into a storage medium.

[0500] The exemplary methods in this disclosure are described as a sequence of actions for clarity of explanation, but this is not intended to limit the order in which the steps are executed. The steps can be executed simultaneously or in different orders if necessary. To implement the methods in this disclosure, additional steps can be added to the example steps, or only the remaining steps (excluding some steps) can be included, or additional steps can be added after excluding some steps.

[0501] The various embodiments described herein are not a list of all possible combinations, but are merely illustrative of representative forms of the disclosure. The matters described in the various embodiments may apply independently or in combination of two or more.

[0502] Furthermore, the various embodiments described in this disclosure can be implemented using hardware, firmware, software, or a combination thereof. When implemented in hardware, they can be implemented using one or more ACICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general-purpose processors, controllers, microcontrollers, microprocessors, etc.

[0503] The scope of this disclosure includes software, device-executable instructions (e.g., operating systems, applications, firmware, programs, etc.) that enable actions in methods of various embodiments to be executed on an apparatus or computer, and a device- or computer-executable non-transitory computer-readable medium storing the aforementioned software or instructions.

[0504] Industrial availability

[0505] The present invention is an apparatus for encoding / decoding images.

Claims

1. An image decoding method, comprising the following steps: The initial motion vector of the current block is derived from a list of merge candidates that includes multiple merge candidates. The initial motion vector includes an L0 initial motion vector and an L1 initial motion vector. The final motion vector of the current block is derived by modifying the initial motion vector of the current block based on a predetermined offset vector. The final motion vector includes an L0 final motion vector and an L1 final motion vector. The predetermined offset vector is obtained based on the position where the difference between the L0 block in the L0 reference image and the L1 block in the L1 reference image is the smallest. as well as Generate a predicted block for the current block based on the final motion vector of the current block. The merge candidate list includes spatial merge candidates derived from spatially adjacent blocks and temporal merge candidates derived from temporally juxtaposed blocks of the current block. The current block is obtained by segmenting the current image based on segmentation information obtained from the bitstream. The multiple merge candidates are different from each other.

2. The image decoding method according to claim 1, wherein, The prediction blocks are generated by averaging the L0 prediction blocks indicated by the L0 final motion vector and the L1 prediction blocks indicated by the L1 final motion vector.

3. The image decoding method according to claim 1, wherein, Whether to modify the initial motion vector of the current block based on the predetermined offset vector is adaptively determined based on the flags from the bit stream signaling.

4. The image decoding method according to claim 1, wherein, The modification of the initial motion vector of the current block is performed in response to the fact that the inter-frame prediction mode of the current block is a merged mode of the decoder-side motion information inference mode.

5. The image decoding method according to claim 1, wherein, The current block is divided into sub-blocks. The final motion vector of the current block is derived using the initial motion vector of the current block, on a per-sub-block basis.

6. An image coding method, comprising the following steps: The initial motion vector of the current block is derived from a list of merge candidates that includes multiple merge candidates. The initial motion vector includes an L0 initial motion vector and an L1 initial motion vector. The final motion vector of the current block is derived by modifying the initial motion vector of the current block based on a predetermined offset vector. The final motion vector includes an L0 final motion vector and an L1 final motion vector. The predetermined offset vector is obtained based on the position where the difference between the L0 block in the L0 reference image and the L1 block in the L1 reference image is the smallest. as well as Generate a predicted block for the current block based on the final motion vector of the current block. The merge candidate list includes spatial merge candidates derived from spatially adjacent blocks and temporal merge candidates derived from temporally juxtaposed blocks of the current block. The current block is obtained by segmenting the current image. Obtain the segmentation information of the current block from the bit stream. The multiple merge candidates are different from each other.

7. The image encoding method according to claim 6, wherein, The prediction blocks are generated by averaging the L0 prediction blocks indicated by the L0 final motion vector and the L1 prediction blocks indicated by the L1 final motion vector.

8. The image encoding method according to claim 6, wherein, The flag is encoded into the bitstream to determine whether the initial motion vector of the current block is modified based on the predetermined offset vector.

9. The image encoding method according to claim 6, wherein, The modification of the initial motion vector of the current block is performed in response to the fact that the inter-frame prediction mode of the current block is a merged mode of the decoder-side motion information inference mode.

10. The image encoding method according to claim 6, wherein, The current block is divided into sub-blocks. The final motion vector of the current block is derived using the initial motion vector of the current block, on a per-sub-block basis.

11. A method for transmitting a bit stream, comprising: The bit stream is generated by executing an encoding method; as well as Transmit the bit stream, The encoding method includes the following steps: The initial motion vector of the current block is derived from a list of merge candidates that includes multiple merge candidates. The initial motion vector includes an L0 initial motion vector and an L1 initial motion vector. The final motion vector of the current block is derived by modifying the initial motion vector of the current block based on a predetermined offset vector. The final motion vector includes an L0 final motion vector and an L1 final motion vector. The predetermined offset vector is obtained based on the position where the difference between the L0 block in the L0 reference image and the L1 block in the L1 reference image is minimal. Generate a predicted block for the current block based on the final motion vector of the current block. The merge candidate list includes spatial merge candidates derived from spatially adjacent blocks and temporal merge candidates derived from temporally juxtaposed blocks of the current block. The current block is obtained by segmenting the current image. Obtain the segmentation information of the current block from the bit stream. The multiple merge candidates are different from each other.

Images