Video encoding / decoding method and apparatus

Abstract

The present application relates to a video encoding method and a video decoding method. The video decoding method according to an embodiment of the present application can include: determining an initial motion vector of a current block using a motion vector of a reconstructed region; searching for a motion vector of the current block based on the initial motion vector; and generating a prediction sample of the current block using the motion vector. The initial motion vector can include a history direction motion vector and a future direction motion vector.

Description

TECHNICAL FIELD

[0001] The present application relates to an image encoding / decoding method and apparatus, and more particularly, to a method and apparatus for predicting a current block using a plurality of prediction blocks generated based on a plurality of prediction information. BACKGROUND

[0002] In recent years, the demand for multimedia data such as video in the Internet is rapidly increasing. However, the development speed of the channel bandwidth is difficult to sufficiently satisfy the rapidly increasing amount of multimedia data. For this reason, the Video Coding Expert Group (VCEG) of the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) and the Moving Picture Expert Group (MPEG) of the International Organization for Standardization / International Electrotechnical Commission (ISO / IEC) established a video compression standard, i.e., High Efficiency Video Coding (HEVC) version 1 in February 2014.

[0003] In the High Efficiency Video Coding (HEVC), a plurality of technologies such as intra prediction (or intra-frame prediction), inter prediction (or inter-frame prediction), transform, quantization, entropy encoding, and loop filtering are defined. SUMMARY

[0004] TECHNICAL PROBLEM

[0005] The present application aims to provide a method and apparatus for predicting a current block using a plurality of prediction blocks generated based on a plurality of prediction information.

[0006] In addition, the present application aims to provide an encoding and decoding method that can more accurately derive motion information.

[0007] In addition, the present application aims to provide an efficient encoding / decoding method and apparatus using encoding information of a reconstructed region.

[0008] In addition, the present application aims to provide an efficient encoding / decoding method and apparatus of motion vector difference information.

[0009] In addition, the present application aims to provide a recording medium for storing a bitstream generated by applying the image encoding method or apparatus of the present application.

[0010] In addition, the present application aims to provide a recording medium for storing a bitstream decoded by applying the image decoding method or apparatus of the present application.

[0011] The technical problems to be solved by the present application are not limited to those mentioned above, and other technical problems not mentioned above will be understood by those skilled in the art with the general knowledge in the field of the present application.

[0012] Technical solutions

[0013] The image decoding method according to an aspect of the present application can include a step of generating a first prediction block by performing inter-picture prediction on a current block; a step of generating a second prediction block by performing intra-picture prediction on the current block; and a step of generating a final prediction block by performing weighted sum on the first prediction block and the second prediction block.

[0014] In the image decoding method according to the present application, the second prediction block can be a block predicted using only a part of available intra-picture prediction modes.

[0015] In the image decoding method according to the present application, the step of generating the first prediction block can include a step of determining an initial motion vector of the current block using motion vectors of a reconstructed region; a step of searching for a motion vector of the current block based on the initial motion vector; and a step of generating prediction samples of the current block using the motion vector, wherein the initial motion vector can include a motion vector of a history direction and a motion vector of a future direction.

[0016] In the image decoding method according to the present application, the step of searching for the motion vector can search for the motion vector using a difference between a prediction block of a history direction indicated by the motion vector of the history direction and a prediction block of a future direction indicated by the motion vector of the future direction.

[0017] In the image decoding method according to the present application, the step of searching for the motion vector can search for the motion vector based on a prediction error generated by applying a difference between the prediction block of the history direction and the prediction block of the future direction to a sum of absolute difference (SAD) calculation method.

[0018] In the image decoding method according to the present application, the step of searching for the motion vector can search for the motion vector within a search region of a predefined size.

[0019] In the image decoding method according to the present application, the motion vector of the reconstructed region can be a candidate motion information of a merge mode.

[0020] In the image decoding method according to the present application, the candidate motion information of the merge mode can include at least one of motion information of a spatial candidate block of the current block and motion information of a temporal candidate block.

[0021] In the image decoding method according to the present application, a step of decoding decoder-side motion vector derivation (DMVD) mode information can be further included. In the case where the decoder-side motion vector derivation (DMVD) mode information indicates that the decoder-side motion vector derivation (DMVD) mode is in operation, the step of searching for the motion vector of the current block can be performed based on the initial motion vector.

[0022] In the image decoding method according to the present application, the searched motion vector can be used in the derivation of a motion vector of a block decoded after the current block.

[0023] An image encoding method according to another aspect of the present application can include a step of generating a first prediction block by performing inter-picture prediction on a current block, a step of generating a second prediction block by performing intra-picture prediction on the current block, and a step of generating a final prediction block by performing a weighted sum on the first prediction block and the second prediction block.

[0024] In the image encoding method according to the present application, the second prediction block can be a block predicted using only a part of available intra-picture prediction modes.

[0025] In the image encoding method according to the present application, the step of generating the first prediction block can include a step of determining an initial motion vector of the current block using a motion vector of a reconstructed region, a step of searching for an optimal motion vector of the current block based on the initial motion vector, and a step of decoding decoder-side motion vector derivation (DMVD) mode information by determining whether or not the optimal motion vector is used. The initial motion vector can include a motion vector of a past direction and a motion vector of a future direction.

[0026] In the image encoding method according to the present application, the step of searching for the optimal motion vector can search for the optimal motion vector using a difference between a prediction block of the past direction indicated by the motion vector of the past direction and a prediction block of the future direction indicated by the motion vector of the future direction.

[0027] In the image encoding method according to the present application, the step of searching for the optimal motion vector can search for the optimal motion vector based on a prediction error generated by applying a difference between the motion vector of the past direction and the prediction block of the future direction to a sum of absolute difference (SAD) calculation method.

[0028] In the image encoding method according to the present application, the step of searching for the optimal motion vector can search for the optimal motion vector within a search region of a predefined size.

[0029] In the image encoding method according to the present application, the motion vector of the reconstructed region can be a candidate motion information of a merge mode.

[0030] In the image encoding method according to the present application, the candidate motion information of the merge mode can include at least one of a motion information of a spatial candidate block of the current block and a motion information of a temporal candidate block.

[0031] In the image encoding method according to the present application, the optimal motion vector can be used in derivation of a motion vector of a block encoded after the current block.

[0032] An image decoding method according to an embodiment of the present application can include a step of determining an initial motion vector of a current block using a motion vector of a reconstructed region, a step of searching for a motion vector of the current block based on the initial motion vector, and a step of generating a prediction sample of the current block using the motion vector, wherein the initial motion vector can include a motion vector of a past direction and a motion vector of a future direction.

[0033] In the image decoding method, the step of searching for the motion vector can search for the motion vector based on a difference between a prediction block of a past direction indicated by the motion vector of the past direction and a prediction block of a future direction indicated by the motion vector of the future direction.

[0034] In the image decoding method, the step of searching for the motion vector can search for the motion vector based on a prediction error generated by applying a difference between the motion vector of the past direction and the prediction block of the future direction to a sum of absolute difference (SAD) calculation method.

[0035] In the image decoding method, the step of searching for the motion vector can search for the motion vector within a search region of a predefined size.

[0036] In the image decoding method, the motion vector of the reconstructed region can be a candidate motion information of a merge mode.

[0037] In the image decoding method, the candidate motion information of the merge mode can include at least one of a motion information of a spatial candidate block of the current block and a motion information of a temporal candidate block.

[0038] In the image decoding method, the method can further include a step of decoding a decoder-side motion vector derivation (DMVD) mode action information, and wherein, in a case that the decoder-side motion vector derivation (DMVD) mode action information indicates that a decoder-side motion vector derivation (DMVD) mode action is performed, the step of searching for the motion vector of the current block can be performed based on the initial motion vector.

[0039] In the image decoding method, the searched motion vector can be used in a motion vector derivation of a block decoded after the current block.

[0040] An image encoding method according to an embodiment of the present application can include a step of determining an initial motion vector of a current block using a motion vector of a reconstructed region, a step of searching for a best motion vector of the current block based on the initial motion vector, and a step of decoding a decoder-side motion vector derivation (DMVD) mode action information by determining whether or not the best motion vector is used, and wherein the initial motion vector can include a motion vector of a past direction and a motion vector of a future direction.

[0041] In the image encoding method, the step of searching for the best motion vector can search for the best motion vector using a difference between a prediction block of the past direction indicated by the motion vector of the past direction and a prediction block of the future direction indicated by the motion vector of the future direction.

[0042] In the image encoding method, the step of searching for the best motion vector can search for the best motion vector based on a prediction error generated by applying the difference between the prediction block of the past direction and the prediction block of the future direction to a sum of absolute difference (SAD) calculation method.

[0043] In the image encoding method, the step of searching for the optimal motion vector can search for the optimal motion vector within a search area of a predefined size.

[0044] In the image encoding method, the motion vector of the reconstructed region can be a candidate motion information of a merge mode.

[0045] In the image encoding method, the candidate motion information of the merge mode can include at least one of motion information of a spatial candidate block of the current block and motion information of a temporal candidate block.

[0046] In the image encoding method, the optimal motion vector can be used in derivation of a motion vector of a block encoded after the current block.

[0047] In a computer-readable non-transitory recording medium storing a bitstream used in image decoding, the bitstream can include decoder-side motion vector derivation (DMVD) mode action information, and in the image decoding, in a case where the decoder-side motion vector derivation (DMVD) mode action information indicates a decoder-side motion vector derivation (DMVD) mode action, a search for a motion vector of a current block can be performed based on an initial motion vector, the initial motion vector can be determined using a motion vector of a reconstructed region, and the initial motion vector can include a motion vector of a past direction and a motion vector of a future direction.

[0048] An image decoding method according to still another embodiment of the present application can include a step of deriving an initial motion vector of a current block, a step of decoding motion vector difference information of the initial motion vector, a step of deriving a motion vector difference of the motion vector using the motion vector difference information, and a step of deriving a final motion vector of the current block by adding the initial motion vector and the motion vector difference, wherein the motion vector difference information can include information about a sign group constituted by one of an X component sign and a Y component sign of the motion vector difference.

[0049] In the image decoding method, the motion vector difference information can include at least one of X component absolute value information of the motion vector difference or Y component absolute value information of the motion vector difference.

[0050] In the image decoding method, the sign group information can indicate one of the sign group constituted by one of the X component sign and the Y component sign.

[0051] In the image decoding method, the sign group can be assigned binary information separately for each sign group.

[0052] In the image decoding method, the sign group information can be an index indicating one of the sign groups consisting of one of an X component sign and a Y component sign.

[0053] In the image decoding method, the initial motion vector can be derived using a merge candidate list of the current block.

[0054] An image encoding method to which another embodiment of the present application is applied can include the steps of determining a final motion vector of a current block, determining a motion vector difference of the final motion vector, and encoding motion vector difference information of the motion vector difference, wherein the motion vector difference and an initial motion vector of the current block are added and used to derive the final motion vector, and the motion vector difference information can include information about a sign group consisting of one of an X component sign and a Y component sign of the motion vector difference.

[0055] In the image encoding method, the motion vector difference information can include at least one of X component absolute value information of the motion vector difference or Y component absolute value information of the motion vector difference.

[0056] In the image encoding method, the sign group information can indicate one of the sign groups consisting of one of an X component sign and a Y component sign.

[0057] In the image encoding method, the sign group can be assigned binary information separately for each sign group.

[0058] In the image encoding method, the sign group information can be an index indicating one of the sign groups consisting of one of an X component sign and a Y component sign.

[0059] In the image encoding method, the initial motion vector can be derived using a merge candidate list of the current block.

[0060] A computer-readable recording medium according to still another aspect of the present application can store a bitstream generated by using the image encoding method and / or apparatus according to the present application.

[0061] A computer-readable recording medium according to still another aspect of the present application can store a bitstream generated by using the image encoding method and / or apparatus according to the present application.

[0062] A computer-readable recording medium according to still another aspect of the present application can store a bitstream generated by using the image encoding method and / or apparatus according to the present application.

[0063] Advantageous Effects

[0064] The present application can provide an image encoding / decoding method and apparatus having improved compression efficiency.

[0065] In addition, the present application can provide a method and apparatus for predicting a current block using a plurality of prediction blocks generated based on a plurality of pieces of prediction information.

[0066] In addition, the present application can improve inter-picture prediction performance by more accurately deriving motion information.

[0067] In addition, the present application can improve encoding efficiency by reducing the amount of encoding information.

[0068] In addition, the present application can design an efficient parallel processing codec structure by limiting a template region used in a current block peripheral reconstruction region used when motion information correction is performed, when designing a method for correcting motion information without transmitting the motion information using the same method in an image encoding / decoding apparatus. Thereby, the time taken to encode an image can be greatly reduced, and the complexity of an encoding apparatus can be reduced.

[0069] In addition, the present application can provide an encoding / decoding method and apparatus for efficiently allocating corresponding binary information using a template matching technique, and a recording medium storing a bitstream.

[0070] Further, the present application can provide a method, an apparatus, and a recording medium storing a bitstream, which efficiently encode / decode motion vector difference information.

[0071] Further, the present application can provide a computer-readable recording medium for storing a bitstream generated by applying the image encoding method / apparatus of the present application.

[0072] Further, the present application can provide a computer-readable recording medium for storing a bitstream decoded by applying the image decoding method and / or apparatus of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0073] Figure 1 is an explanatory diagram schematically illustrating a configuration of an image encoding apparatus 100.

[0074] Figure 2 is a schematic diagram illustrating an embodiment of a prediction section of an image encoding apparatus.

[0075] Figure 3 is a schematic diagram for explaining a method of generating a motion information candidate group in a SKIP (skip) mode or a MERGE (merge) mode.

[0076] Figure 4 is a schematic diagram for explaining a position of a spatial candidate block and a position of a temporal candidate block.

[0077] Figure 5 is a schematic diagram for explaining a method of deciding motion information of a temporal candidate.

[0078] Figure 6 is a schematic diagram for explaining a method of generating an intra prediction block using a DC (direct current) mode.

[0079] Figure 7 is a schematic diagram for explaining a method of generating an intra prediction block using a Planar mode.

[0080] Figure 8 is a schematic diagram illustrating N directional prediction modes.

[0081] Figure 9 is a schematic diagram for explaining a method of configuring a most probable mode (MPM) candidate mode list.

[0082] Figure 10 is a schematic diagram for explaining an embodiment of encoding encoding information in an image encoding apparatus.

[0083] Figure 11is a schematic diagram for explaining the configuration of an image decoding apparatus to which the present application is applied.

[0084] Figure 12 is a schematic diagram for explaining a prediction section of an image decoding apparatus.

[0085] Figure 13 is a schematic diagram for explaining an embodiment of decoding encoding information in an image decoding apparatus.

[0086] Figure 14 is a schematic diagram for explaining a prediction section of an image encoding apparatus.

[0087] Figure 15 is a schematic diagram for explaining a prediction section of an image decoding apparatus.

[0088] Figure 16 is a method for explaining an encoding method of encoding information to which an embodiment of the present application is applied.

[0089] Figure 17 is a method for explaining a decoding method of decoding information to which an embodiment of the present application is applied.

[0090] Figure 18 is an explanatory diagram illustrating a result list of an initial motion information search section in a decoder side motion vector derivation (DMVD) mode in a prediction section in an image encoding / decoding apparatus.

[0091] Figure 19 is a schematic diagram for explaining an action of a decoder side motion vector derivation (DMVD) motion extrapolation section in a prediction section in an image encoding / decoding apparatus.

[0092] Figure 20 is a schematic diagram for explaining a template matching mode and a bi-directional matching mode.

[0093] Figure 21 is an explanatory diagram of a method of setting a template block.

[0094] Figure 22 is a schematic diagram for explaining a template matching technique using a template region in a reconstructed region around a current block in an intra prediction mode.

[0095] Figure 23 is a schematic diagram for explaining a linear mode (LM) applied to a color difference block.

[0096] Figure 24 is a schematic diagram for explaining a method of applying a single weighting value to a prediction block.

[0097] Figure 25 FIG. 1 is a schematic diagram illustrating a method for applying multiple weighting values to a prediction block.

[0098] Figure 26 FIG. 2 is a schematic diagram illustrating another method for applying multiple weighting values to a prediction block.

[0099] Figure 27 FIG. 3 is a schematic diagram illustrating a method for applying weighting values to only a partial region of a prediction block.

[0100] Figure 28 FIG. 4 is a schematic diagram illustrating another method for applying weighting values to only a partial region of a prediction block.

[0101] Figure 29 FIG. 5 is a flowchart illustrating an image encoding apparatus.

[0102] Figure 30 FIG. 6 is a schematic diagram illustrating a prediction section of an image encoding apparatus in detail.

[0103] Figure 31 FIG. 7 is a flowchart illustrating a derivation method of candidate motion information of SKIP (SKIP), MERGE (MERGE) mode.

[0104] Figure 32 FIG. 8 is a flowchart illustrating a derivation method of candidate motion information of AMVP (Advanced Motion Vector Prediction) mode.

[0105] Figure 33 FIG. 9 is a schematic diagram illustrating a position of a reconstructed block used for deriving a spatial / temporal candidate in the derivation method of candidate motion information of SKIP (SKIP), MERGE (MERGE), AMVP (Advanced Motion Vector Prediction) mode.

[0106] Figure 34 FIG. 10 is a schematic diagram illustrating a derivation method of a temporal candidate in the derivation method of candidate motion information of SKIP (SKIP), MERGE (MERGE), AMVP (Advanced Motion Vector Prediction) mode.

[0107] Figure 35 FIG. 11 is a schematic diagram illustrating a derivation method of a combined bi-directional candidate mode in the derivation method of candidate motion information of SKIP (SKIP), MERGE (MERGE) mode.

[0108] Figure 36 FIG. 12 is a flowchart illustrating an encoding method of prediction information.

[0109] Figure 37 FIG. 13 is a flowchart illustrating an image decoding apparatus in outline.

[0110] Figure 38 FIG. 1 is a schematic diagram illustrating a prediction section of an image encoding apparatus to which an embodiment of the present application is applied.

[0111] Figure 39 FIG. 2 is a flowchart illustrating a decoding method of prediction information.

[0112] Figure 40 FIG. 3 is a schematic diagram illustrating a prediction section of an image encoding apparatus to which an embodiment of the present application is applied.

[0113] Figure 41 FIG. 4 is a schematic diagram illustrating a prediction section of an image decoding apparatus to which an embodiment of the present application is applied.

[0114] Figure 42 FIG. 5 is a table illustrating a decoder-side motion vector derivation (DMVD) initial motion information search section in a prediction section in an image encoding / decoding apparatus to which an embodiment of the present application is applied.

[0115] Figure 43 FIG. 6 is a schematic diagram illustrating a decoder-side motion vector derivation (DMVD) motion extrapolation section in a prediction section in an image encoding / decoding apparatus to which an embodiment of the present application is applied.

[0116] Figure 44 FIG. 7 is a schematic diagram illustrating a template matching mode of a decoder-side motion vector derivation (DMVD) mode to which an embodiment of the present application is applied.

[0117] Figure 45 FIG. 8 is a schematic diagram illustrating a bi-directional matching mode of a decoder-side motion vector derivation (DMVD) mode to which an embodiment of the present application is applied.

[0118] Figure 46 FIG. 9 is a schematic diagram illustrating a motion extrapolation method of a decoder-side motion vector derivation (DMVD) mode to which an embodiment of the present application is applied.

[0119] Figure 47 FIG. 10 is a flowchart illustrating an encoding procedure of prediction information to which an embodiment of the present application is applied.

[0120] Figure 48 FIG. 11 is a flowchart illustrating a decoding procedure of prediction information to which an embodiment of the present application is applied.

[0121] Figure 49 FIG. 12 is a schematic diagram illustrating a method of using optimal motion information of a template matching mode as candidate motion information of a skip (SKIP), merge (MERGE) mode to which an embodiment of the present application is applied.

[0122] Figure 50 is a schematic diagram for explaining a method for using the best motion information of a template matching mode as a candidate motion information of an advanced motion vector prediction (AMVP) mode, to which one embodiment of the present application is applied.

[0123] Figure 51 is a schematic diagram for explaining a motion information decision method using a plurality of templates when deriving the best motion information of a template matching mode, to which one embodiment of the present application is applied.

[0124] Figure 52 is a schematic diagram for explaining a method for correcting motion information of a candidate motion list of a SKIP (skip) mode and a MERGE (merge) mode using a template matching mode, to which one embodiment of the present application is applied.

[0125] Figure 53 is a schematic diagram for explaining a method for changing a reference picture when correcting a candidate motion information of a SKIP (skip) mode and a MERGE (merge) mode, to which one embodiment of the present application is applied.

[0126] Figure 54 is a schematic diagram for explaining a method for rearranging motion information of a candidate motion list of a SKIP (skip) mode and a MERGE (merge) mode using a template matching mode, to which one embodiment of the present application is applied.

[0127] Figure 55 is a schematic diagram for explaining a method for dividing a template of a template matching mode into sub-templates and applying different weighting values to the respective sub-templates, to which one embodiment of the present application is applied.

[0128] Figure 56 is a flowchart for explaining an image encoding method, to which one embodiment of the present application is applied.

[0129] Figure 57 is a flowchart for explaining an image encoding method, to which one embodiment of the present application is applied.

[0130] Figure 58 is a flowchart for illustrating an image encoding apparatus.

[0131] Figure 59 is a schematic diagram for explaining a prediction unit of an image encoding apparatus.

[0132] Figure 60 is a flowchart for illustrating an encoding method of prediction information.

[0133] Figure 61 is a flowchart for illustrating an image decoding apparatus.

[0134] Figure 62 is a schematic diagram for explaining a prediction section of an image encoding apparatus to which an embodiment of the present application is applied.

[0135] Figure 63 is a flowchart for illustrating a decoding method of prediction information.

[0136] Figure 64 is a schematic diagram for explaining a prediction section of an image encoding apparatus to which an embodiment of the present application is applied.

[0137] Figure 65 is a schematic diagram for explaining a prediction section of an image decoding apparatus to which an embodiment of the present application is applied.

[0138] Figure 66 is a flowchart for illustrating an encoding method of coding information to which an embodiment of the present application is applied.

[0139] Figure 67 is a flowchart for illustrating a decoding method of coding information to which an embodiment of the present application is applied.

[0140] Figure 68 is a flowchart for illustrating an encoding / decoding method of motion vector difference information to which an embodiment of the present application is applied.

[0141] Figure 69 is a table for illustrating a method of using template matching cost decision binary information when encoding motion vector difference information to which an embodiment of the present application is applied.

[0142] Figure 70 is a schematic diagram for explaining a template matching technique when deciding binary information of motion vector difference information to which an embodiment of the present application is applied.

[0143] Figure 71 is a schematic diagram for explaining a method of assigning binary information to each of candidate in a most probable mode (MPM) candidate list using a template matching technique applied to a decoder-side intra mode derivation (DIMD) mode and a method of assigning binary information to a chroma candidate mode to which an embodiment of the present application is applied.

[0144] Figure 72 is a schematic diagram for explaining a template matching technique of a decoder-side intra mode derivation (DIMD) mode to which an embodiment of the present application is applied.

[0145] Figure 73is a schematic diagram for explaining a method for deciding whether or not to use a template in a reconstructed area in a periphery of a prediction block according to whether or not to use a template in a reconstructed area in a periphery of a current block when performing a local brightness compensation technique, which is applied to an embodiment of the present application.

[0146] Figure 74 is a transform unit of an image encoding apparatus to which an embodiment of the present application is applied.

[0147] Figure 75 is an inverse transform unit of an image encoding / decoding apparatus to which an embodiment of the present application is applied.

[0148] Figure 76 is a schematic diagram for explaining a method for using a template matching technique in order to decide binary information of a transform type in a transform unit / inverse transform unit of an image encoding / decoding apparatus, which is applied to an embodiment of the present application.

[0149] Figure 77 is a flowchart for explaining an image decoding method to which an embodiment of the present application is applied.

[0150] Figure 78 is a flowchart for explaining an image encoding method to which an embodiment of the present application is applied. DETAILED DESCRIPTION

[0151] Next, embodiments to which the present application is applied will be explained in detail with reference to the accompanying drawings of the present specification so that a person having an ordinary knowledge in the technical field to which the present application pertains can easily carry out the present application. However, the present application can be realized in many different forms and is not limited to the embodiments explained in the present specification. Also, in the accompanying drawings, parts irrelevant to the explanation of the present application are omitted in order to clearly explain the present application, and similar parts are assigned with similar reference numerals throughout the specification.

[0152] Throughout the specification, when it is written that a certain part is "connected" to other parts, it includes not only a case where it is directly connected but also a case where other elements are interposed therebetween and electrically connected.

[0153] Also, throughout the specification, when it is written that a certain part "includes" a certain constituent element, unless it is explicitly written to the contrary, it does not mean that other constituent elements are excluded but means that other constituent elements can be further included.

[0154] Also, terms such as 1st, 2nd, etc. can be used when different constituent elements are explained, but the above-mentioned constituent elements are not limited by the above-mentioned terms. The above-mentioned terms are only used to distinguish one constituent element from other constituent elements.

[0155] Moreover, in the embodiments related to the apparatus and method described in the present specification, a part of the configuration of the apparatus or a part of the steps of the method can be omitted. Furthermore, the order of a part of the configuration of the apparatus or a part of the steps of the method can be changed. Furthermore, other configuration or other steps can be inserted into a part of the configuration of the apparatus or a part of the steps of the method.

[0156] Furthermore, a part of the configuration or a part of the steps of the first embodiment of the present application can be added to or replace a part of the configuration or a part of the steps of the second embodiment of the present application.

[0157] Moreover, the configuration included in the embodiments of the present application is illustrated separately only to indicate different specific functions, and does not mean that each configuration is constituted by a separate hardware or a software configuration unit. That is, although each configuration is described for convenience of explanation, at least two of the configurations can be combined into one configuration, or one configuration can be divided into a plurality of configurations and perform the corresponding functions. The combined embodiments and the divided embodiments of each configuration described above are also included in the scope of the claims of the present application without departing from the essence of the present application.

[0158] First, the terms used in the present application will be briefly described as follows.

[0159] The decoding apparatus (Video Decoding Apparatus) described in the following content can be included in an apparatus in a service terminal such as a civilian security camera, a civilian security system, a military security camera, a military security system, a personal computer (PC), a notebook computer, a portable multimedia player (PMP), a wireless communication terminal, a smart phone, a television (TV) application service, and a service server, and can be various apparatuses equipped with a communication modem for performing communication with a user terminal, a wired / wireless communication network, and the like, a memory for storing various programs and data for performing image decoding or performing inter or intra prediction between or within decoded pictures, a microprocessor for performing calculation and control by executing the programs, and the like.

[0160] Further, the video encoded into a bitstream by the encoder can be transmitted to a video decoding apparatus in real time or non-real time through wired / wireless communication networks such as the Internet, a short range wireless communication network, a wireless LAN network, a wireless broadband access service (WiBro) network, a mobile communication network, or the like, or through various communication interfaces such as a cable, a universal serial bus (USB), and the like, and played back by being reconstructed into a video through decoding. Alternatively, the bitstream generated by the encoder can be stored in a storage. The storage can include a volatile memory and a non-volatile memory. In the present specification, the storage can be represented as a recording medium in which the bitstream is stored.

[0161] Generally, a video can be composed of a series of pictures, and each picture can be divided into a coding unit such as a block. Further, it will be understood by those skilled in the art to which the present embodiment pertains that the term "picture" described in the following content can be replaced with other terms having the same meaning such as image, frame, and the like. Further, it will be understood by those skilled in the art to which the present embodiment pertains that the term "coding unit" described in the following content can be replaced with other terms having the same meaning such as unit block, block, and the like.

[0162] Next, embodiments to which the present application is applied will be described in more detail with reference to the accompanying drawings. In describing the present application, repeated descriptions of the same constituent elements will be omitted.

[0163] Figure 1 is an exemplary diagram schematically illustrating a configuration of a video encoding apparatus 100.

[0164] The video encoding apparatus 100 is an apparatus for encoding a video, and can generally include a picture division unit 101, prediction units 102 and 103, a transform unit 105, a quantization unit 106, an entropy encoding unit 107, an inverse quantization unit 108, an inverse transform unit 109, an addition unit 110, a loop filter unit 111, a storage unit 112, and a subtraction unit 104.

[0165] The image division section 101 can divide the input image into the minimum size of the encoding target block (hereinafter referred to as "minimum encoding block") from the maximum size of the encoding target block (hereinafter referred to as "maximum encoding block") in stages. As a method of dividing the block, various methods can be used. The quad-tree division (hereinafter referred to as "QT division") is a division method in which the current encoding block is accurately divided into four. The binary-tree division (hereinafter referred to as "BT division") is a division method in which the encoding block is accurately divided into two in the horizontal direction or the vertical direction. In addition to these, various division methods can be used. Furthermore, a method in which various division methods are used at the same time can be used.

[0166] The prediction sections 102 and 103 can generate a prediction block using the surrounding pixels of the current prediction target block within the current original block or the pixels within the reference image that has already been encoded / decoded. For one encoding block, one or more prediction blocks can be generated. In the case where the prediction block of the encoding block is one, the prediction block can have the same shape as the encoding block.

[0167] The prediction technique of the video signal can be roughly composed of intra prediction and inter prediction. The intra prediction is a method of generating a prediction block using the surrounding pixels of the current block. The inter prediction is a method of searching for a block most similar to the current block from the reference image that has already been encoded / decoded and generating a prediction block.

[0168] After the prediction block is generated, a residual block can be generated by subtracting the prediction block from the original block. For the generated residual block, the best prediction mode in which the prediction block is generated using various techniques such as rate-distortion optimization (RDO) can be determined. For example, the calculation formula of the rate-distortion optimization (RDO) cost can be as shown in Mathematical Formula 1.

[0169]

Mathematical Formula 1

[0170]

[0171] In the above mathematical expression 1, D, R, J respectively represent degradation due to quantization, code rate of the compressed stream, and rate-distortion (RD) cost. In addition, Φ is an encoding mode, and λ is a Lagrangian multiplier. λ is used as a scaling correction coefficient for matching units between error amount and bit amount. In order to be able to select an optimal encoding mode in an encoding process, the J, i.e., rate-distortion (RD) cost, when a corresponding mode is applied should be smaller than in the case where other modes are applied. As shown in the above mathematical expression 1, the rate-distortion (RD) cost should be calculated while considering both bit rate (R) and error (D).

[0172] Figure 2 is a schematic diagram illustrating an embodiment of a prediction section of an image encoding apparatus.

[0173] The intra prediction section 201 can perform intra prediction using original information and reconstructed information. For example, selection of at least one of a plurality of reference pixel rows, generation of reference pixels, filtering of the reference pixels, generation of a prediction block using the reference pixels, and / or filtering of the generated prediction block can be performed. The intra prediction mode search section 202 can determine an optimal intra prediction mode, for example, based on rate-distortion (RD) costs calculated for each of the prediction modes, and then generate or select a prediction block based on the optimal intra prediction mode and finally output the prediction block.

[0174] The inter prediction section 203 can perform inter prediction using original information and reconstructed information. The inter prediction section 203 can calculate rate-distortion (RD) costs for a plurality of inter prediction modes including a SKIP mode, a MERGE mode, and an AMVP mode, respectively. The MERGE candidate search section 204 can construct a candidate motion information set for the SKIP mode and the MERGE mode. The merge candidate decision section 205 can determine optimal motion information from the candidate motion information set based on the rate-distortion (RD) costs. The AMVP candidate search section 206 can construct a candidate motion information set for the AMVP mode. The motion estimation section 207 can perform motion estimation using the candidate motion information set and determine optimal motion information. The motion compensation section 208 can generate a prediction block by performing motion compensation using the optimal motion information. Alternatively, the motion compensation section 208 can finally select / output the generated prediction block associated with the optimal motion information.

[0175] The above-described inter-picture prediction modes can include three modes (a SKIP mode, a MERGE mode, and an AMVP mode). Each of the prediction modes can generate a prediction block of a current block using motion information (prediction direction information, reference picture information, and a motion vector). In addition, an additional prediction mode using motion information can exist.

[0176] In the case of the SKIP mode, prediction information (e.g., optimal prediction information) of the current block can be decided using motion information of a reconstructed area. A motion information candidate group can be constituted within the reconstructed area, and a candidate having a minimum rate-distortion (RD) cost in the corresponding candidate group can be used as the prediction information of the current block. The method of constituting the motion information candidate group of the SKIP mode is the same as the method of constituting the motion information candidate group of the MERGE mode, which will be described later.

[0177] In the case of the MERGE mode, the prediction information (e.g., optimal prediction information) of the current block is decided using motion information of a reconstructed area, like the SKIP mode. However, the difference between the two is that the SKIP mode searches for motion information having a prediction error of 0 from a motion information candidate group, whereas the MERGE mode searches for motion information having a prediction error of 0 from a motion information candidate group. Like the SKIP mode, a motion information candidate group can be constituted within the reconstructed area, and a prediction block can be generated by using a candidate having a minimum rate-distortion (RD) cost in the corresponding candidate group as the prediction information of the current block.

[0178] Figure 3 is a diagram for explaining a method of constituting a motion information candidate group in the SKIP mode or the MERGE mode.

[0179] Figure 4 is a diagram for explaining the positions of a spatial candidate block and the positions of a temporal candidate block.

[0180] Figure 5 is a diagram for explaining a method of deciding motion information of a temporal candidate.

[0181] The maximum number of the motion information candidate group can be decided as the same value in the video encoding apparatus and the video decoding apparatus, or the corresponding number information can be transmitted in a higher-level header (a header transmitted in a higher level than a block, such as a video parameter set, a sequence parameter set, a picture parameter set, a parallel block header, a slice header, etc.) of the video encoding apparatus.

[0182] In steps S301 and S302, only when spatial candidate blocks and temporal candidate blocks are encoded in an inter-frame prediction mode can the motion information derived from the corresponding motion information be included in the motion information candidate group.

[0183] In step S301, spatial candidates can be selected from the vicinity of the current block within the same image. For example, four out of five spatial candidate blocks at specific locations can be selected as spatial candidates. The locations of the spatial candidate blocks can be as follows: Figure 4 The positions A1 to A5 are shown. However, the number and position of spatial candidate blocks are not limited to this and can be any block within the reconstructed area. Spatial candidates can be considered in the order of A1, A2, A3, A4, A5, and the motion information of the most available spatial candidate blocks can be used to determine the spatial candidates. However, the order in which multiple candidates are considered is not limited to the order described above. In the case of duplicate motion information among multiple spatial candidates, only the motion information of the candidate with the higher priority can be considered.

[0184] In step S302, a time candidate can be selected from an image encoded / decoded prior to the current image. For example, one of two time candidate blocks within a collocated image can be selected as a time candidate. The location of the time candidate block can be as follows: Figure 4 The positions of B1 and B2 are shown. The position of each candidate is determined based on a block within the image that is at the same position as the current block in the current image. The juxtaposed images can be set under the same conditions in both the image encoding and decoding apparatus. For example, a reference image corresponding to a specific reference image index can be selected as the juxtaposed image. Alternatively, signaling can be applied to the index information used to indicate the juxtaposed image. Temporal candidates can be considered in the order of blocks B1 and B2, and motion information of the preferred candidate blocks can be used to determine the temporal candidates. However, the number, position, and consideration order of temporal candidate blocks are not limited to the embodiments described above.

[0185] like Figure 4 as well as Figure 5 As shown, for example, the motion information of a candidate block (B1 or B2) within the juxtaposed image can refer to a predicted block located within the reference image B. The reference images for each candidate block may be different, but for ease of explanation in this specification, they are all represented as reference image B. The corresponding motion vector can be determined as a temporal candidate motion vector by scaling it using the ratio of the distance (TD) between the juxtaposed image and reference image B to the distance (TB) between the current image and reference image A. For example, the scaling described above can be performed using the following mathematical formula 2.

[0186] [Equation 2]

[0187]

[0188] where MV represents a motion vector of the motion information of the temporal candidate block, MV scale represents a scaled motion vector, TB represents a temporal distance between the collocated picture and the reference picture B, and TD represents a temporal distance between the current picture and the reference picture A. At this time, the reference picture A and the reference picture B can be the same reference picture. The scaled motion vector can be determined as the motion vector of the temporal candidate. Alternatively, the reference picture information of the temporal candidate motion information can be determined as the reference picture of the current picture and the temporal candidate motion information can be derived.

[0189] Step S303 can be performed only when the number of the motion information candidates derived in steps S301 and S302 is less than the maximum number. In step S303, duplicated candidates can be removed. Further, new bi-directional motion information candidates can be added using the combination of the motion information candidates derived in steps S301 and S302. The bi-directional motion information candidate refers to a new candidate derived by sequentially selecting one of the previously derived history or future direction motion information and combining them.

[0190] The priority order of the bi-directional motion information candidate combination is given in Table 1 below. However, Table 1 is only an example and there can be combinations different from those of Table 1.

[0191] [Table 1]

[0192] Sequence of construction 0 1 2 3 4 5 6 7 8 9 10 11 Candidate index (history) 0 1 0 2 1 2 0 3 1 3 2 3 Candidate index (future) 1 0 2 0 2 1 3 0 3 1 3 2

[0193] In a case where the maximum number of the motion information candidates cannot be filled even if the bi-directional motion information candidates are used, step S304 is performed. In step S304, the motion vector of the motion information candidate can be fixed to a zero motion vector and the maximum number of the motion information candidates can be filled by changing the reference picture of different prediction directions. The advanced motion vector prediction (AMVP) mode determines the motion information (e.g., the best motion information) of the current block by performing motion estimation on the reference pictures of different prediction directions, respectively. Here, the prediction direction can be either a uni-direction using only one of the history / future or a bi-direction using both the history and the future. Using the motion information determined by the motion estimation, a prediction block can be generated by performing motion compensation. Here, a motion information candidate set for the motion estimation is derived for the reference pictures of different prediction directions, respectively. The corresponding motion information candidate set is used as a starting position for the motion estimation.

[0194] The method for deriving a motion information candidate set for motion prediction of an advanced motion vector prediction (AMVP) mode is similar to the method described in Figure 3 The method described above is partially similar. The maximum number of motion information candidates of the advanced motion vector prediction (AMVP) mode can be determined as the same value in the image encoding device and the image decoding device, or the corresponding number information can be transmitted in the upper level of the block. In the advanced motion vector prediction (AMVP) mode, only when the spatial candidate block and the temporal candidate block are encoded in the inter prediction mode, the motion information derived by using the corresponding motion information can be included in the motion information candidate set.

[0195] In the case of the advanced motion vector prediction (AMVP) mode, the number of spatial candidate blocks can be 2, and the priority order for selecting the spatial candidate can also be different from the case of the merge (MERGE) mode. However, the remaining description can be the same as the description of step S301. In addition, the temporal candidate of the advanced motion vector prediction (AMVP) mode can be derived in the same way as the method described in step S306.

[0196] In the case of the advanced motion vector prediction (AMVP) mode, when the currently derived selected candidate contains duplicate motion information, it can be deleted. In addition, when the number of derived candidates is less than the maximum number of candidates, zero motion information candidates can be added in the same way as step S308.

[0197] One of the motion information candidates derived in the above-described manner can be selected. For example, the motion information candidate with the minimum rate-distortion (RD) cost value can be selected as the optimal motion information candidate, and the corresponding motion information is used as the basis to obtain the optimal motion information of the advanced motion vector prediction (AMVP) mode through the motion prediction process.

[0198] Figure 6 is a schematic diagram for describing a method for generating an intra prediction block suitable for a mean (DC) mode.

[0199] The intra prediction mode can include a planar mode, a DC mode, and N directional modes (angular mode or directional mode), etc. for a total of (N+2) prediction modes. In the case of the DC mode, the R1 to R4 regions constituting the current block can be predicted by different methods, respectively. For example, an average of the reference pixels a to s can be calculated and applied to all the predicted pixels of the R1 to R4 regions. Next, in the R1 region, a final prediction block can be generated by using a finite impulse response (FIR) filtering of 2 adjacent reference pixels a and j. In addition, in the R2 and R3 regions, a final prediction block can be generated by using a finite impulse response (FIR) filtering of 1 adjacent reference pixel (1 of b to h, k to q).

[0200] Figure 7 is a diagram for explaining a method of generating an intra prediction block to which a planar mode is applied.

[0201] The planar mode generates a final prediction block by using a linear interpolation method of the reference pixels of the upper and left positions and the copied reference pixels corresponding to the lower and right positions, respectively, at different predicted pixel positions.

[0202] Figure 8 is a diagram illustrating N directional prediction modes.

[0203] The directional prediction mode generates a final prediction block by applying the values of the adjacent reference pixels of different prediction directions to the prediction block.

[0204] In order to efficiently encode / decode the intra prediction mode, a most probable mode (MPM) can be used. At this time, in order to constitute a most probable mode (MPM) candidate mode list of the current block, a candidate mode can be derived from a spatial candidate block around the current block. At this time, the available spatial candidate block can be the same as the spatial candidate block in Figure 4 However, it is not limited thereto, and the number and / or position of the spatial candidate block can be changed.

[0205] Figure 9 is a diagram for explaining a method of constituting a most probable mode (MPM) candidate mode list.

[0206] In step S901, the reconstructed intra prediction modes of the spatial candidate blocks are first derived as the most probable mode (MPM) candidate modes. The intra prediction modes of the blocks can be added as the most probable mode (MPM) candidate modes in the order of A1, A2, A3, A4, A5 blocks. Among them, the candidate blocks without reconstructed intra prediction modes can be excluded. In addition, as described above, the priority order of the spatial candidate blocks can be changed.

[0207] In step S902, the candidate intra prediction modes derived in step S901 are added with the neighboring directional modes of the directional modes in the order from high to low priority. For example, in the case that the candidate modes in step S901 are the 4th directional mode and the 11th directional mode, the 3rd and 5th modes neighboring the 4th directional mode and the 10th and 12th modes neighboring the 11th directional mode can be added as the most probable mode (MPM) candidate modes.

[0208] In step S903, the non-directional modes can be added as the most probable mode (MPM) candidate modes.

[0209] In step S904, it is determined whether the number of the current most probable mode (MPM) candidate modes is greater than or equal to the number of the maximum most probable mode (MPM) candidate modes. When it is determined in step S904 that the number of the current most probable mode (MPM) candidate modes is less than the number of the maximum most probable mode (MPM) candidate modes, the most probable mode (MPM) candidate mode list can be constituted by adding the default candidate modes until the number of the maximum most probable mode (MPM) candidate modes. At this time, the default candidate modes can include any intra prediction modes such as the vertical mode, the horizontal mode, the mean (DC) mode, the planar mode, etc. The priority order among the default candidate modes is not limited to the order as described above, but can be changed. When it is determined that the number of the current most probable mode (MPM) candidate modes is greater than or equal to the number of the maximum most probable mode (MPM) candidate modes, the most probable mode (MPM) candidate mode list can be constituted with the most probable mode (MPM) candidate modes corresponding to the number of the maximum most probable mode (MPM) candidate modes in the order from high to low priority.

[0210] The transform unit 105 can generate a transform block by performing a transform on a residual block, which is a difference between the original block and the prediction block. The transform block is the smallest unit used in the transform and quantization processes. The transform unit 105 can generate a transform block having transform coefficients by transforming a residual signal from a spatial region to a frequency region. As a method of transforming a residual signal to a frequency region, various transform techniques such as a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), etc. can be used. By performing a transform, a residual signal can be transformed to a frequency region and transform coefficients can be generated thereby. In order to facilitate the use of transform techniques, a matrix operation using a basis vector will be performed, and various different transform techniques can be mixedly used in performing the matrix operation according to a prediction mode used when encoding the prediction block. For example, when performing intra prediction, a discrete cosine transform can be used in a horizontal direction and a discrete sine transform can be used in a vertical direction according to a prediction mode. Alternatively, different transform techniques can be used in a horizontal direction and a vertical direction according to an aspect ratio, a size (one side length or both side lengths in a horizontal or vertical direction), a shape, a prediction mode (inter prediction or intra prediction), an inter prediction mode, etc. of a current block.

[0211] The quantization unit 106 can generate a quantized transform block by quantizing a transform block. That is, the quantization unit can generate a quantized transform block (Quantized Transform Coefficient) having quantized transform coefficients by quantizing transform coefficients of a transform block generated from the transform unit 105. As a quantization method, a dead zone uniform threshold quantization (DZUTQ) or a quantization weighted matrix, etc. can be used. However, it is not limited to the above examples, and various quantization methods improved from the above quantization methods can be used, for example.

[0212] In the above, the case where the image encoding apparatus includes the transform unit and the quantization unit is illustrated and described. However, the image encoding apparatus can selectively include the transform unit and / or the quantization unit. That is, the image encoding apparatus can generate a transform block by performing transform on a residual block but not perform a quantization process. Or, the image encoding apparatus can perform only quantization without performing transform on a residual block. Or, the image encoding apparatus can not perform transform and quantization. Even in the case where only one of the transform unit or the quantization unit is performed or all processes are not performed in the image encoding apparatus, the block input to the entropy encoding unit 107 is generally referred to as a "quantized transform block".

[0213] The entropy encoding unit 107 can encode the quantized transform block and output a bitstream. That is, the entropy encoding unit can encode the coefficients of the quantized transform block output from the quantization unit using various encoding techniques such as entropy encoding, and generate and output a bitstream including additional information (e.g., information related to a prediction mode (information related to a prediction mode refers to information belonging to intra prediction or inter prediction), quantization coefficients, etc.) required to decode the corresponding block in the image decoding apparatus described later.

[0214] The inverse quantization unit 108 can reconstruct an inverse quantized transform block by inversely performing the quantization technique used in the quantization process on the quantized transform block.

[0215] The inverse transform unit 109 reconstructs a residual block by performing inverse transform on the inverse quantized transform block using the same method as that used in the transform process, and can perform inverse transform by inversely performing the transform technique used in the transform unit.

[0216] The inverse quantization unit and the inverse transform unit can perform inverse quantization and inverse transform by inversely using the quantization method and the transform method used in the quantization unit and the transform unit. Or, in the case where only quantization is performed without transform in the encoding step, only inverse quantization can be performed without inverse transform in the reconstruction step. Also, in the case where neither transform nor inverse transform is performed in the encoding step, neither inverse transform nor inverse quantization can be performed in the decoding step. Or, information related to the corresponding block can be omitted without being included in the image encoding apparatus.

[0217] The addition unit 110 can reconstruct a current block by performing addition on the residual signal generated in the inverse transform unit and the prediction block generated by prediction.

[0218] The filtering unit 111 is a process of additionally performing filtering on the entire image after reconstructing all blocks within the current image, and can include deblocking filtering and sample adaptive offset (SAO), etc. Deblocking filtering refers to filtering for reducing blocking distortion generated in a process of encoding an image in a block unit. Sample adaptive offset (SAO) refers to filtering that minimizes a difference between a reconstructed image and an original image by performing addition or subtraction on a reconstructed pixel using a specific value.

[0219] The memory 112 stores a reconstructed current block in which filtering is additionally performed on a residual signal generated in the inverse transform unit and a prediction block generated by prediction, and can be used to predict a next block or a next image, etc.

[0220] The subtraction unit 104 generates a residual block by performing subtraction on a current original block and a prediction block.

[0221] Figure 10 FIG. 1 is a schematic diagram illustrating an embodiment of encoding information in an image encoding apparatus.

[0222] In step S1001, action information of a skip (SKIP) mode is encoded. In the skip mode, a merge (MERGE) mode is used when a current block is predicted, and a predicted block of the current block is used as a reconstructed block in a decoding device. In step S1002 of inter-picture prediction, it is determined whether or not the skip (SKIP) mode is in action. When it is determined in step S1002 that the skip (SKIP) mode is in action (Yes), in step S1007, merge (MERGE) candidate index information for the skip (SKIP) mode is encoded, and then the present flowchart ends. When it is determined in step S1002 that the skip (SKIP) mode is not in action (No), in step S1003, a prediction mode is encoded. In step S1004, it is determined whether the prediction mode is an inter-picture prediction mode or an intra-picture prediction mode. When it is determined in S1004 that the prediction mode is the inter-picture prediction mode (Yes), in step S1005, action information of a merge (MERGE) mode is encoded. In step S1006, it is determined whether or not the merge (MERGE) mode is in action. When it is determined in step S1006 that the merge (MERGE) mode is in action (Yes), the process proceeds to step S1007, and merge (MERGE) candidate index information for the merge (MERGE) mode is encoded, and then the present flowchart ends. When it is determined in step S1006 that the merge (MERGE) mode is not in action (No), in step S1008, a prediction direction is encoded. The prediction direction can be one of a past direction, a future direction, and a bi-direction. In step S1009, it is determined whether or not the prediction direction is the future direction. When it is determined in step S1009 that the prediction direction is not the future direction (Yes), in step S1010, reference picture index information of the past direction is encoded. In step S1011, motion vector difference (MVD) information of the past direction is encoded. In step S1012, motion vector predictor (MVP) information of the past direction is encoded. When it is determined in step S1009 that the prediction direction is the future direction or the bi-direction (No), after step S1012 is completed, in step S1013, it is determined whether or not the prediction direction is the past direction. When it is determined in step S1013 that the prediction direction is not the past direction (Yes), in step S1014, reference picture index information of the future direction is encoded. In step S1015, motion vector difference (MVD) information of the future direction is encoded. In step S1016, motion vector predictor (MVP) information of the future direction is encoded, and then the present flowchart ends.

[0223] When it is determined in S1004 that the prediction mode is the intra prediction mode (No), the action information for the most probable mode (MPM) is encoded in S1017. The most probable mode (MPM) refers to a method of transmitting index information for the most probable mode (MPM) candidate intra prediction mode when the most probable mode (MPM) candidate intra prediction mode includes the optimal intra prediction mode information for the current block decided in the image encoding apparatus, after the most probable mode (MPM) candidate intra prediction mode is constituted from the intra prediction mode using the reconstructed blocks in the periphery of the current block. It is determined in S1018 whether the most probable mode (MPM) acts or not. The most probable mode (MPM) is a mode in which the index information is encoded when the same mode as the prediction mode of the current block is included in the prediction mode of the reconstructed block in the periphery of the current block. When it is determined in S1018 that the most probable mode (MPM) acts (Yes), the index information for the most probable mode (MPM) candidate intra prediction mode is encoded in S1019. When it is determined in S1018 that the most probable mode (MPM) does not act (No), the optimal intra prediction mode information for the luminance (Luma) in the remaining intra prediction modes other than the most probable mode (MPM) candidate intra prediction mode is encoded in S1020.

[0224] After S1019 and S1020 are ended, the optimal intra prediction mode information for the chroma (Chroma) is encoded, and then the present flowchart is ended. The module for performing S1008 to S1016 can be referred to as a motion information encoding section. Further, the module for performing S1017 to S1020 can be referred to as a luminance intra prediction mode encoding section.

[0225] Figure 11 is a schematic diagram for explaining the constitution of the image decoding apparatus to which the present application is applied.

[0226] The image decoding apparatus is an apparatus for decoding an image, and can include an entropy decoding section, an inverse quantization section, an inverse transform section, a prediction section, an addition operation section, a loop filter section, and a memory section. The encoding block in the image encoding apparatus can be referred to as a decoding block in the image decoding apparatus.

[0227] The entropy decoding section 1101 can reconstruct various information required when decoding the corresponding block and the quantized transform coefficient by analyzing the bit stream transmitted from the image encoding apparatus.

[0228] The inverse quantization section 1102 can reconstruct the inverse quantization block having inverse quantization coefficients by inversely performing the quantization technique used in the quantization process on the quantization coefficients decoded in the entropy decoding section.

[0229] The inverse transform section 1103 can reconstruct the residual block having the difference signal by performing inverse transform on the inverse quantization transform block using the same method as that used in the transform process, and the inverse transform can be performed by inversely performing the transform technique used in the transform section.

[0230] The prediction sections 1107, 1108 generate the prediction block using the prediction mode information decoded in the entropy decoding section, and the same method as that performed in the prediction section of the image encoding apparatus can be used at this time.

[0231] The addition operation section 1104 can reconstruct the current block by performing addition operation on the residual signal reconstructed in the inverse transform section and the prediction block generated by prediction.

[0232] The filter section 1105 is used to perform filtering on the entire area of the image after reconstructing all the blocks within the current image, and the filtering can include deblocking filtering, sample adaptive offset (SAO), and the like. The detailed contents are the same as those described for the filter section 111 of the image encoding apparatus described above.

[0233] The memory 1106 stores the current block reconstructed by performing addition operation on the residual signal generated in the inverse transform section and the prediction block generated by prediction, and the filtering additionally performed in the loop filter section, and can be used to predict the next block or the next image, and the like.

[0234] Figure 12 is a schematic diagram for describing the prediction section of the image decoding apparatus.

[0235] In the intra prediction section 1201, when the prediction mode is intra prediction, the intra prediction mode decision section 1202 can decide the intra prediction mode information and generate the prediction block by performing intra prediction.

[0236] In the inter-picture prediction section 1203, when the prediction mode is inter prediction, the prediction mode of the current block can be reconstructed in SKIP, MERGE, and AMVP modes. In the case of the SKIP mode or the MERGE mode, a candidate motion information set for the SKIP mode and the MERGE mode can be constructed in the merge candidate search section 1204. The merge candidate decision section 1205 can decide the motion information of the current block from the corresponding candidate motion information set. In the case of the AMVP, the AMVP candidate search section 1206 can construct a candidate motion information set for the AMVP mode. The motion information decision section 1207 can decide the motion information of the current block from the corresponding candidate motion information set using the transmitted motion vector prediction (MVP) information. Next, the prediction block can be generated by performing motion compensation in the motion compensation section 1208 using the motion information of the current block decided in each mode.

[0237] Figure 13 is a schematic diagram for describing an embodiment of decoding encoded information in an image decoding device.

[0238] In step S1301, the action information of the SKIP mode is decoded. In step S1302, it is determined whether the SKIP mode is in action or not. When the SKIP mode is in action in step S1302 (Yes), the MERGE candidate index information for the SKIP mode is decoded in step S1307, and the flowchart ends. When the SKIP mode is not in action in step S1302 (No), the prediction mode is decoded in step S1303. In step S1304, it is determined whether the prediction mode is an inter-picture prediction mode or an intra-picture prediction mode. When the prediction mode is an inter-picture prediction mode in step S1304 (Yes), the action information of the MERGE mode is decoded in step S1305. In step S1306, it is determined whether the MERGE mode is in action or not. When the MERGE mode is in action in step S1306 (Yes), the flowchart proceeds to step S1307 and the MERGE candidate index information for the MERGE mode is decoded, and then the flowchart ends. When the MERGE mode is not in action in step S1306 (No), the prediction direction is decoded in step S1308. The prediction direction can be one of a past direction, a future direction, and a bi-direction. In step S1309, it is determined whether the prediction direction is the future direction or not. When the prediction direction is not the future direction in step S1309 (Yes), the reference picture index information of the past direction is decoded in step S1310. In step S1311, the MVD information of the past direction is decoded. In step S1312, the MVP information of the past direction is decoded. When the prediction direction is the future direction or the bi-direction in step S1309 (No), after step S1312 is completed, it is determined whether the prediction direction is the past direction or not in step S1313. When the prediction direction is not the past direction in step S1313 (Yes), the reference picture index information of the future direction is decoded in step S1314. In step S1315, the MVD information of the future direction is decoded. In step S1316, the MVP information of the future direction is decoded, and then the flowchart ends. When the prediction mode is an intra-picture prediction mode in step S1304 (No), the action information of the MPM is decoded in step S1317.In step S1318, it is determined whether or not the most probable mode (MPM) is in operation. When the most probable mode (MPM) is in operation in step S1318 (Yes), in step S1319, index information of the most probable mode (MPM) candidate intra prediction mode is decoded. When the most probable mode (MPM) is not in operation in step S1318 (No), in step S1320, the best intra prediction mode information of the luminance (Luma) in the remaining intra prediction modes other than the most probable mode (MPM) candidate intra prediction mode is decoded. After the end of step S1319 and step S1320, the best intra prediction mode information of the color difference (Chroma) is decoded, and then the present flowchart is ended. The module for performing steps S1308 to S1316 can be referred to as a motion information decoding section. Further, the module for performing steps S1317 to S1320 can be referred to as a luminance intra prediction mode decoding section.

[0239] Next, an embodiment in which a weighted sum prediction block is generated by weighting and summing prediction blocks generated using two or more pieces of prediction information will be described. Figure 14 to Figure 17 is a drawing commonly applicable to the embodiments described below.

[0240] Figure 14 is a schematic diagram for describing a prediction section of an image encoding apparatus.

[0241] The intra prediction section 1401 can perform intra prediction using original information and reconstructed information. Whether or not the intra prediction mode is decided using the decoder-side intra mode derivation (DIMD) mode is determined by the rate-distortion (RD) cost. In the case where the decoder-side intra mode derivation (DIMD) mode is not used, the luminance intra prediction mode exploration section 1402 can decide the intra prediction mode most suitable for the luminance block from the intra prediction modes using the rate-distortion (RD) cost. In the case where the decoder-side intra mode derivation (DIMD) mode is used, the luminance intra prediction mode exploration section 1403 using the decoder-side intra mode derivation (DIMD) can set a template region in the current block peripheral reconstructed area, and derive the luminance intra prediction mode most suitable for the template region using the reconstructed pixels in the template region peripheral, thereby deciding the intra prediction mode most suitable for the template region as the best luminance intra prediction mode of the current block. Next, the color difference intra prediction mode exploration section 1404 can explore the intra prediction mode most suitable for the color difference block using the rate-distortion (RD) cost.

[0242] The inter prediction unit 1405 can perform inter prediction using original information and reconstructed information. The rate-distortion (RD) cost values of the skip (SKIP) mode, the merge (MERGE) mode, the advanced motion vector prediction (AMVP) mode, and the decoder-side motion vector derivation (DMVD) mode can be calculated and the best prediction mode can be determined. The descriptions of the drawing numbers 1406 to 1409 are the same as those of the drawing numbers 204 to 207 in FIG. 20. Figure 2 The decoder-side motion vector derivation (DMVD) initial motion exploration unit 1410 can derive initial motion information applicable to the decoder-side motion vector derivation (DMVD) mode using inter prediction modes of reconstructed blocks in the current block neighborhood and the reference picture. The decoder-side motion vector derivation (DMVD) motion speculation unit 1411 can determine the best motion information by performing decoder-side motion vector derivation (DMVD) motion speculation using the derived initial motion information. The motion compensation unit 1412 can generate a prediction block by performing motion compensation using the best motion information determined in each mode. Here, the motion information can include at least one of a motion vector, a reference picture index, and prediction direction information.

[0243] The prediction information exploration unit 1413 for generating a weighted sum prediction block can be used to explore additional prediction information (hereinafter referred to as "Nth auxiliary prediction information", N is an integer of 1 or more) for a weighted sum of the best prediction information (hereinafter referred to as "main prediction information") determined using the intra prediction mode and the inter prediction mode. The additional prediction information can be one or more, and can correspond to any prediction information in the intra prediction mode and the inter prediction mode. For example, when the main prediction information is a planar (Planar) mode in the intra prediction mode, the 1st auxiliary prediction information can be the first motion information in the merge (MERGE) candidate list, and the 2nd auxiliary prediction information can be a vertical mode in the intra prediction mode. In addition, the weighting value information of different prediction blocks for each prediction information can also be determined by the rate-distortion (RD) cost value and included in the additional prediction information for transmission.

[0244] Figure 15 is a schematic diagram for describing a prediction unit of an image decoding apparatus.

[0245] The intra prediction section 1501 can determine whether or not the intra prediction mode needs to be decided using the decoder side intra mode derivation (DIMD) mode using the reconstructed information. In a case where the decoder side intra mode derivation (DIMD) mode is not used, the luma intra prediction mode decision section 1502 can decide the intra prediction mode of the luma block using the reconstructed information. In a case where the decoder side intra mode derivation (DIMD) mode is used, the luma intra prediction mode decision section using the decoder side intra mode derivation (DIMD) 1503 can set a template region in the current block surrounding reconstructed area, and derive the luma intra prediction mode most suitable for the template region using the reconstructed pixels surrounding the template region, thereby deciding the intra prediction mode most suitable for the template region as the luma intra prediction mode of the current block. Next, the chroma intra prediction mode decision section 1504 can decide the intra prediction mode of the chroma block.

[0246] The inter prediction section 1505 can decide one of the SKIP mode, the MERGE mode, the AMVP mode, and the DMVD in a case where the inter prediction is performed using the reconstructed information. The description of the drawing numbers 1506 to 1509 is the same as that of the drawing numbers 1204 to 1207 in the Figure 12 The description of the drawing numbers 1510 and 1511 is the same as that of the drawing numbers 1410 and 1411 in the Figure 14 The motion compensation section 1512 can generate the prediction block by performing the motion compensation using the motion information decided in each mode. The prediction information decision section 1513 for deciding whether or not the weighted and prediction is applied and generating the weighted and prediction block can decide N number of secondary prediction information (i.e., Nth secondary prediction information) for weighted and with the primary prediction information (hereinafter referred to as "main prediction information") decided using the intra prediction mode and the inter prediction mode.

[0247] Figure 16 is a method for explaining an encoding method of the encoding information to which one embodiment of the present application is applied.

[0248] In step S1601, the action information of the SKIP mode is encoded. In step S1602, it is determined whether the SKIP mode is in action. When the SKIP mode is determined to be in action, in step S1607, the action information of the DMVD mode is encoded. In step S1608, it is determined whether the DMVD mode is in action, and when the DMVD mode is determined to be in action, in step S1609, the DMVD mode information is encoded. When the DMVD mode is determined not to be in action, in step S1610, the MERGE candidate index information for the SKIP mode is encoded. When the SKIP mode is determined not to be in action, in step S1603, the prediction mode is encoded. In step S1604, it is determined whether the prediction mode is inter prediction. When the prediction mode is determined to be inter prediction, in step S1605, the action information of the MERGE mode is encoded. In step S1606, it is determined whether the MERGE mode is in action. When the MERGE mode is determined to be in action, steps S1607 to S1610 can be executed, which are the same as the above description. The MERGE candidate index information in step S1610 is candidate index information for the MERGE mode instead of the SKIP mode. When the MERGE mode is determined not to be in action, in step S1620, the motion information encoding unit can encode the motion information. Step S1620 can correspond to the process of steps S1008 to S1010 in Figure 10 When the prediction mode is intra prediction mode instead of inter prediction, in step S1611, the action information of the DIMD mode is encoded. In step S1612, it is determined whether the DIMD mode is in action. When the DIMD mode is determined not to be in action, in step S1630, the intra prediction mode information can be encoded by the luma intra prediction mode encoding unit. Step S1630 can correspond to the process of steps S1017 to S1020 in Figure 10 Step S1613 can encode the chroma intra prediction mode information. By the encoding method as described above, the main prediction information can be encoded.

[0249] In step S1614, the weighted-sum prediction action information for deciding whether to weight and sum with the primary prediction information is encoded. Either the weighted-sum prediction is applied unconditionally (the weighted-sum prediction action is performed unconditionally) without encoding the corresponding information, or it is not applied (the weighted-sum prediction action is not performed unconditionally). In step S1615, it is decided whether to perform the weighted-sum prediction action. When it is decided not to perform the weighted-sum prediction action, the present flowchart is ended, and when it is decided to perform the weighted-sum prediction action, in step S1616, the weighted-sum prediction information is encoded. The weighted-sum prediction information can include the weighting value information assigned to each prediction block generated using the primary prediction information and the secondary prediction information, the weighted-sum method, and the like.

[0250] Figure 17 is a method for explaining a decoding method of encoded information to which an embodiment of the present application is applied.

[0251] In step S1701, the action information of the SKIP mode is decoded. In step S1702, it is decided whether to perform the action of the SKIP mode. When it is decided to perform the SKIP mode action, in step S1707, the action information of the DMVD mode is decoded. In step S1708, it is decided whether to perform the action of the DMVD mode, and when it is decided to perform the DMVD mode action, in step S1709, the DMVD mode information is decoded. When it is decided not to perform the DMVD mode action, in step S1710, the MERGE candidate index information for the SKIP mode is decoded. When it is decided not to perform the SKIP mode, in step S1703, the prediction mode is decoded. In step S1704, it is decided whether the prediction mode is inter prediction. When it is decided that the prediction mode is inter prediction, in step S1705, the action information of the MERGE mode is decoded. In step S1706, it is decided whether to perform the action of the MERGE mode. When it is decided to perform the MERGE mode action, steps S1707 to S1710 can be executed, which are the same as explained above. The MERGE candidate index information in step S1710 is candidate index information for the MERGE mode, not for the SKIP mode. When it is decided not to perform the MERGE mode action, in step S1720, the motion information decoding section can decode the motion information. Step S1720 can correspond to Figure 13corresponding to the process of steps S1308 to S1310 in FIG. 13. When the prediction mode is the intra-picture prediction mode instead of the inter-picture prediction mode, the action information of the decoder-side intra-mode derivation (DIMD) mode is decoded in step S1711. In step S1712, it is determined whether or not the action of the decoder-side intra-mode derivation (DIMD) mode is performed. When it is determined that the action of the decoder-side intra-mode derivation (DIMD) mode is not performed, the luma intra-picture prediction mode information is decoded in step S1730. Step S1730 can correspond to the process of steps S1317 to S1320 in FIG. 13. Figure 13 corresponding to the process of steps S1308 to S1310 in FIG. 13. When the prediction mode is the intra-picture prediction mode instead of the inter-picture prediction mode, the action information of the decoder-side intra-mode derivation (DIMD) mode is decoded in step S1711. In step S1712, it is determined whether or not the action of the decoder-side intra-mode derivation (DIMD) mode is performed. When it is determined that the action of the decoder-side intra-mode derivation (DIMD) mode is not performed, the luma intra-picture prediction mode information is decoded in step S1730. Step S1730 can correspond to the process of steps S1317 to S1320 in FIG. 13.

[0252] (Embodiment 1)

[0253] Embodiment 1 relates to a method of generating a weighted sum prediction block using prediction information of an inter-picture prediction mode (SKIP mode, MERGE mode, AMVP mode, DMVD mode) and prediction information of an additional inter-picture prediction mode.

[0254] Before the method of generating a weighted sum prediction block is described in detail, the prediction modes used to generate the prediction information required in this embodiment will first be described in detail. The SKIP mode, MERGE mode, and AMVP mode are the same as described above. The DMVD mode refers to a method of generating a prediction block by generating the same motion information in the case where the motion information is not encoded / decoded in the image encoding / decoding apparatus.

[0255] Figure 18 is an explanatory diagram illustrating a result list of the initial motion information search section in the DMVD mode in the prediction section in the image encoding / decoding apparatus.

[0256] Other additional motion information not listed in this list can also be included in the initial motion information list.

[0257] Figure 19 is a schematic diagram for explaining the action of the DMVD motion search section in the prediction section in the image encoding / decoding apparatus.

[0258] The optimal motion information can be determined after performing a decoder side motion vector derivation (DMVD) mode using initial motion information determined by the initial motion information exploration section using the decoder side motion vector derivation (DMVD). The performing of the decoder side motion vector derivation (DMVD) mode can mean that the initial motion information is corrected.

[0259] The decoder side motion vector derivation (DMVD) mode includes a mode using a template (hereinafter referred to as a "template matching mode") and a mode not using the template (hereinafter referred to as a "bi-directional matching mode").

[0260] In the case of using the bi-directional matching mode 1901, the uni-directional motion vectors of the respective initial motion information are scaled in a linear manner into reference pictures of opposite prediction directions. The scaling of the motion vectors is performed in a manner proportional to the distance between the current picture and the reference pictures of the respective directions. After determining the motion vectors 1902 in both directions by the above-described manner, the motion vector of each direction minimizing the difference between the prediction blocks of the past and future directions is determined as the optimal motion information 1903.

[0261] Figure 20 is a diagram for explaining the template matching mode and the bi-directional matching mode.

[0262] In the bi-directional matching mode 2002, the prediction block of the current block can be generated using the tie of the two prediction blocks of the past and future directions after generating the motion vectors of the past and future directions of the current block in a linear manner.

[0263] In the case of using the template matching mode 1904, the number of template blocks can be determined in the reconstructed area. In the case of using a single template block 1905 (hereinafter referred to as a "single template matching mode"), the left and upper reconstructed areas adjacent to the periphery of the current block can be determined as the template blocks, and the optimal motion information can be determined by predicting 1907 the motion using the respective template blocks.

[0264] After exploring the prediction block most similar to the template block in the single template matching mode 2001, the block adjacent to the corresponding template block can be determined as the prediction block of the current block. The cost of motion prediction is the sum of the prediction error and the hypothetical bit amount of motion information. The prediction error can be obtained by various calculation methods such as the sum of absolute difference (SAD), the sum of absolute hadamard transform difference (SATD), the sum of square difference (SSD), etc. Mathematical formula 3, mathematical formula 4, and mathematical formula 5 are mathematical formulas describing the calculation methods of the sum of absolute difference (SAD), the sum of absolute hadamard transform difference (SATD), and the sum of square difference (SSD), respectively.

[0265] [mathematical formula 3]

[0266] Diff(i, j) = Template(i, j) - PredBlk(i, j)

[0267] [mathematical formula 4]

[0268] DiffT(i, j) = HT(Template(i, j) - PredBlk(i, j))

[0269] [mathematical formula 5]

[0270] Diff(i, j) = Template(i, j) - PredBlk(i, j)

[0271] In the above mathematical expression, i, j represent the position of a pixel, Template(i, j) represents the pixel of a template block, and PredBlk(i, j) represents the pixel of a prediction block. The HT() function in the mathematical expression 4 represents a function value obtained by performing Hadamard transform on the difference block of the template block and the prediction block. The hypothetical bit amount of the motion information is not the information actually transmitted, but a value calculated for the hypothetical bit amount of the motion information expected in the same manner in the image encoding apparatus and the image decoding apparatus. For example, the hypothetical bit amount can be determined by calculating the difference vector size of the motion vector of the initial motion information and the motion vector in the motion information currently performing motion estimation. In addition, the hypothetical bit amount of the motion information can be calculated using the bit amount of the reference image information. In the case of using a plurality of template blocks 1906 (hereinafter referred to as "multi-template matching mode"), a plurality of template blocks can be determined in the reconstructed area.

[0272] Figure 21 is an example diagram of a method of setting a template block.

[0273] The template blocks can be set in the manner shown in the example diagram of the drawing reference numbers 2101 to 2103 in Figure 21 In the template block setting method A 2101, the template blocks can be determined in the lower left end (template A), the upper left end (template B), the left side of the upper end (template C), and the right side of the upper end (template D) of the current block, respectively. Each template block can be determined in a plurality of different sizes and shapes. In the template block setting method B 2102, the template blocks can be determined in the lower left end (template A), the upper left end (template B), the left side of the upper end (template C), and the right side of the upper end (template D) of the current block in the same manner as the template block setting method A, but the difference is that all of the reconstructed areas of the left side and the upper end adjacent to the current block are used. In the template block setting method C 2103, the template blocks can be generated while considering both the template block setting method A and the template block setting method B. In addition, the reconstructed areas of the left side and the upper end adjacent to the current block can be determined to be used for a single template block. As described above with reference to Figure 21As described above, the template block can be generated from the reconstructed area around the current block by various methods. However, it can also be used by transmitting information indicating the shape and size of the template block from the image encoding apparatus. After performing the motion estimation 1907 for searching for a prediction block most similar to each of the decided template blocks in the reference image, the motion information most suitable for each of the template blocks is estimated, and the best motion information is decided by searching for the motion information most suitable for the current block among the corresponding motion information. The index information of the above-mentioned best motion information can also be transmitted at the same time (i.e., equivalent to transmitting the index information of the template block).

[0274] As described above, the main prediction information can be generated using four prediction modes in the inter-picture prediction mode. In addition, an additional inter-picture prediction mode for generating the main prediction information can also be used. The additional auxiliary prediction information can be derived by various methods.

[0275] In the case where the main prediction information is motion information generated by the SKIP mode or the MERGE mode, as the method 1-a, it can be weighted and summed with other candidate motion information in the candidate motion information list of the SKIP or MERGE mode. At this time, the other candidate motion information corresponding to the auxiliary prediction information can transmit the index information of the candidate motion information, or can not transmit it, but can decide the pre-set candidate motion information as the auxiliary prediction information in a manner such as using the motion information of the next order of the candidate motion information corresponding to the prediction information by convention. As the method 1-b, it can be weighted and summed with the decided motion information by using the reconstructed motion information around the current block as the initial motion information and performing additional motion estimation. At this time, the motion information corresponding to the auxiliary prediction information can express the motion using other parameters of the non-prediction direction, the reference image, the two-dimensional (2D) motion vector. For example, it can be expressed using a three-dimensional (3D) (inter-image distance, X coordinate, Y coordinate) motion vector. In addition, only a part of the motion information can be transmitted. For example, the prediction direction can not be transmitted, but can be unconditionally decided as the direction opposite to the prediction direction of the motion information of the SKIP or MERGE mode. As the method 1-c, it can be weighted and summed with the motion information generated by the decoder-side motion vector derivation (DMVD) mode. At this time, the motion information corresponding to the auxiliary prediction information can be derived using the template matching mode or the bi-directional matching mode of the decoder-side motion vector derivation (DMVD) mode, and can also decide which of the motion information decided according to each mode is suitable for the weighted and summed best motion information and transmit the corresponding mode information.

[0276] In the case where the primary prediction information is motion information generated through an advanced motion vector prediction (AMVP) mode, as method 2-a, one of the candidate motion information of the SKIP and MERGE modes can be selected for weighted sum. At this time, either the index information of the candidate motion information corresponding to the secondary prediction information can be transmitted, or the pre-set candidate motion information can be used unconditionally. For example, the weighted sum can be performed unconditionally with the candidate motion information having the highest priority. As method 2-b, the motion information determined through the additional motion estimation can be determined as the secondary prediction information in the same manner as method 1-b. At this time, the motion estimation can be performed using other initial motion information except for the initial motion information used when the motion information of the primary prediction information is derived. Other than this, it is the same as method 1-b. Method 2-c is the same as method 1-c.

[0277] In the case where the primary prediction information is motion information generated through a decoder-side motion vector derivation (DMVD) mode, method 3-a is the same as method 1-a. Method 3-b is the same as method 1-b. As method 3-c, in the case where the decoder-side motion vector derivation (DMVD) mode determined as the primary prediction information is a bi-directional matching mode, the best motion information of the uni-template block matching mode is determined as the secondary prediction information. In the case where the primary prediction information is the uni-template block matching mode, the best motion information of the bi-directional matching mode can be determined as the secondary prediction information in reverse, or the motion information of the sub-template block most suitable for the current block among the multi-template matching mode can be determined as the secondary prediction information. In the case where the primary prediction information is the multi-template block matching mode, the motion information of the sub-template block other than the sub-template block corresponding to the primary prediction information can be determined as the secondary prediction information.

[0278] In the above-described methods, the primary prediction information and the secondary prediction information should be different from each other. In addition, more than one secondary prediction information can be used instead of one secondary prediction information. As the weighting values respectively applied to each prediction block generated using the primary prediction information and the secondary prediction information, the weighting value information can be determined by transmitting any set information of the pre-set weighting value set of the primary prediction information and the secondary prediction information. The weighting value can be a complex number, and the sum of the weighting values allocated to each prediction block is 1.

[0279] Figure 24 to Figure 28is a diagram for explaining a method of performing a weighted sum between prediction blocks using a primary prediction information and a secondary prediction information in a case where the secondary prediction information is 1. In the corresponding diagram, a prediction block A is prediction information generated through the primary prediction information, and a prediction block B is prediction information generated through the secondary prediction information. Further, it is assumed that the sizes of the prediction blocks A and B are 4x4, and a weighted sum is performed using an applicable weighting value on prediction pixels at the same position in the prediction block A and the prediction block B.

[0280] Figure 24 is a diagram for explaining a method of applying a single weighting value to a prediction block.

[0281] In Figure 24 , a weighted sum can be performed and a final prediction block can be generated by applying a weighting value Al to all pixels of the prediction block A and applying a weighting value W2 to all pixels of the prediction block B.

[0282] Figure 25 is a diagram for explaining a method of applying a plurality of weighting values to a prediction block.

[0283] In Figure 25 , the prediction block A can be divided into N arbitrary regions along a vertical direction and different weighting values can be applied to each of the arbitrary regions, respectively. In the present example, the prediction block A is divided into 4 arbitrary regions and different weighting values Wl, W2, W3, W4 are applied to each of the regions of the prediction block A from the left side in order, respectively, and different weighting values W5, W6, W7, W8 are applied to each of the regions of the prediction block B from the left side, respectively. Among them, the sum of Wl and W5, the sum of W2 and W6, the sum of W3 and W7, and the sum of W4 and W5 are 1, respectively. After the weighting values are decided in the manner as described above, a weighted sum is performed and a final prediction block is generated.

[0284] Figure 26 is a diagram for explaining another method of applying a plurality of weighting values to a prediction block.

[0285] Figure 26 The method in Figure 25 is the same as the method in

[0286] Figure 27 and Figure 28 is an example diagram of a method of applying a weighted sum to a part of a region, instead of applying a weighted sum to the entire region of a prediction block.

[0287] Figure 27 is a diagram for explaining a method of applying a weighting value to only a part of a region of a prediction block.

[0288] In Figure 27In this case, the weighting value can be applied only to the upper left region of the prediction blocks A and B. Referring to Figure 27 The four pixels of the upper left can be determined as the upper left region, and the final prediction block can be generated by performing a weighted sum only on the prediction pixels corresponding to the respective regions in the prediction blocks A and B.

[0289] Figure 28 is a diagram for explaining another method of applying a weighting value to only a portion of a region of a prediction block.

[0290] Figure 28 The method in Figure 27 is the same as the method in

[0291] Further, in Figure 27 , Figure 28 The weighting values applied to the prediction pixels of the prediction blocks A and B in

[0292] In the weighted sum method, a predetermined weighted sum method and weighting value information can be used according to the type of the main prediction information and the auxiliary prediction information of the weighted sum. Further, in the case of performing a weighted sum on three or more prediction blocks, it is not necessary to perform a weighted sum on each prediction block at once, but it is also possible to perform a weighted sum in the manner of first performing a weighted sum on the main prediction information and the first auxiliary prediction information, and then sequentially performing a weighted sum on the result and the second auxiliary prediction information, the third auxiliary prediction information (two prediction blocks at a time).

[0293] (Embodiment 2)

[0294] In this embodiment, a method of generating a weighted sum prediction block using prediction information of an intra-picture prediction mode and prediction information of an additional intra-picture prediction mode will be described in detail.

[0295] Before a detailed description of the method of generating a weighted sum prediction block, a prediction mode for generating prediction information required in this embodiment will be first described in detail. The decoder-side intra mode derivation (DIMD) mode refers to a method of generating a prediction block by generating the same intra-picture prediction mode in the case where the intra-picture prediction information is not encoded / decoded in the image encoding apparatus and the image decoding apparatus.

[0296] Figure 22 is a diagram for explaining a template matching technique using a template region in a reconstructed region around a current block in an intra-picture prediction mode.

[0297] In Figure 22From the reconstructed region around the current block, a template region is determined. The template region can be determined in different sizes. Next, a reference region of the template region is determined from the periphery of the template region. The template region is intra-predicted using the reconstructed pixels of the reference region, and the difference between the predicted value and the reconstructed value of the template region, i.e., the prediction error amount, is determined as the template matching cost value. As the calculation method of the prediction error amount, various different methods such as the absolute error sum (SAD), the absolute change error sum (SATD), and the square error sum (SSD) can be used.

[0298] Figure 23 is a schematic diagram for explaining a linear mode (LM) suitable for a chrominance block.

[0299] The prediction block using the linear mode (LM) in the chrominance block can be generated with reference to mathematical expression 6.

[0300]

Mathematical Expression 6

[0301] pred C (i, j) = a • rec L '(i, j) + β

[0302] wherein predC(i, j) represents the predicted pixel at different position (i, j) of the chrominance block, rec'L(i, j) represents the down-sampled reconstructed pixel obtained using the optimal intra-prediction mode of the luminance block. a, β are linear regression parameters for minimizing the difference between the reconstructed pixels of the template region around the luminance block and the reconstructed pixels of the template region around the chrominance block using the linear regression method. The linear regression parameters a, β can be derived with reference to mathematical expression 7.

[0303]

Mathematical Expression 7

[0304]

[0305]

[0306] wherein L(n) represents the down-sampled reconstructed pixels of the template region around the luminance block, and C(n) represents the reconstructed pixels of the template region around the chrominance block. The mode of the prediction block of the chrominance block generated by the above method is the linear mode (LM). The template regions around the luminance block and the chrominance block can be determined in the manner as shown in FIG. 2301.

[0307] As described above, in the intra prediction mode, a total of four prediction modes including the mean (DC), the planar (Planar), the directional mode (hereinafter referred to as "Normal Intra mode"), and the decoder-side intra mode derivation (DIMD) mode can be used to generate the primary prediction information of the luma block, and in the chroma block, the linear mode (LM) can also be used to generate the primary prediction information. In addition, an additional inter picture prediction mode for generating the primary prediction information can also be used. The additional secondary prediction information can be derived by a plurality of different methods.

[0308] In the case where the primary prediction information of the luma block is the intra prediction mode generated by the Normal Intra mode, as the method 4-a, an intra prediction mode other than the intra prediction mode corresponding to the primary prediction information can be determined as the secondary prediction information. At this time, the intra prediction mode as the secondary prediction information can select one of the most probable mode (MPM) candidate list, and in the case described above, the candidate index information can be transmitted, or the candidate intra prediction mode with the highest priority can be fixedly used as the secondary prediction information. As the method 4-b, an intra prediction mode determined by the decoder-side intra mode derivation (DIMD) mode other than the intra prediction mode corresponding to the primary prediction information can be used as the secondary prediction information.

[0309] In the case where the primary prediction information of the luma block is the intra prediction mode generated by the decoder-side intra mode derivation (DIMD) mode, as the method 5-a, one of the intra prediction modes included in the most probable mode (MPM) candidate list other than the intra prediction information corresponding to the primary prediction information can be determined as the secondary prediction information. At this time, the most probable mode (MPM) candidate mode information can be transmitted, or the candidate intra prediction mode with the highest priority can be fixedly used as the secondary prediction information. As the method 5-b, the other intra prediction mode derived by the decoder-side intra mode derivation (DIMD) can be used as the secondary prediction information. At this time, the intra prediction mode as the secondary prediction information can be derived using a part of the template when the decoder-side intra mode derivation (DIMD) mode is performed, or can be derived using all the templates.

[0310] In the case where the primary prediction information of the chroma block is the intra prediction mode generated by the linear mode (LM), as the method 6-a, the intra prediction mode other than the intra prediction mode corresponding to the primary prediction information can be determined as the secondary prediction information. Figure 23In the linear mode (LM), one set of linear regression parameters can be derived from the template region within the reconstructed region of the luma block and the chroma block and the predicted block of the chroma block can be generated using mathematical expression 6, or more than two sets of linear regression parameters can be derived and the predicted blocks can be generated using the linear regression parameters of each set and mathematical expression 6, respectively, and then the generated predicted blocks can be weighted and summed. FIG. 2302 is an example of deriving two sets of linear regression parameters, i.e., a and β, in the linear mode (LM). The reconstructed pixels of the luma block peripheral template region are divided based on an arbitrary critical point, and the linear regression parameters are derived using the luma reconstructed pixels of the A region divided based on the critical value and the pixels of the chroma block peripheral template region at the same positions as the reconstructed pixels of the luma block peripheral template region included in the A region, respectively. The linear regression parameters of the B region are also derived using the same method. The predicted blocks are generated using the two sets of derived linear regression parameters and mathematical expression 6, respectively, and the final predicted block is generated by performing weighted sum on the generated predicted blocks.

[0311] In the above-described method, the primary prediction information and the secondary prediction information should be different from each other. In addition, more than one secondary prediction information can be used instead of one secondary prediction information. As the weighting values respectively applied to the predicted blocks generated using the primary prediction information and the secondary prediction information, the weighting value information can be determined by transmitting any set of information from among the sets of weighting values of the primary prediction information and the secondary prediction information that are set in advance. The weighting values can be complex numbers, and the sum of the weighting values allocated to the predicted blocks is 1.

[0312] The above-described method can be applied to the case where the predicted blocks are generated using the primary prediction information and the secondary prediction information. Figure 24 to Figure 28 The above-described method can be applied to the case where the predicted blocks are generated using the primary prediction information and the secondary prediction information.

[0313] In the weighted sum method, the weighted sum method and the weighting value information set in advance can be used according to the types of the primary prediction information and the secondary prediction information of the weighted sum. In addition, in the case where more than three predicted blocks are weighted and summed, the predicted blocks can not be weighted and summed all at once, but can be weighted and summed in a manner such as first weighting and summing the primary prediction information and the first secondary prediction information, and then sequentially weighting and summing the result and the second secondary prediction information, the third secondary prediction information (only two predicted blocks at a time).

[0314] (Embodiment 3)

[0315] In the present embodiment, the method of generating a weighted sum prediction block using the prediction information of the inter picture prediction mode and the prediction information of the additional intra picture prediction mode or the prediction information of the intra picture prediction mode and the additional inter picture prediction information will be described in detail.

[0316] In the case where the main prediction information is an intra picture prediction mode generated by a normal inter (Normal Intra) mode or a decoder side intra mode derivation (DIMD) mode, the motion information of the corresponding mode can be decided as the auxiliary prediction information after the best prediction mode most suitable as the auxiliary prediction information is decided from among SKIP, MERGE, AMVP, DMVD modes. The motion information of the auxiliary prediction information can be all transmitted in the same manner as the main prediction mode, or only a part of the motion information can be transmitted, or the motion information can be decided as the auxiliary prediction information in the image encoding apparatus and the image decoding apparatus without being transmitted by using the prediction information set in advance.

[0317] In the case where the main prediction information is motion information generated by SKIP, MERGE, AMVP, DMVD modes, the intra picture prediction mode decided by the normal inter (Normal Intra) mode or the decoder side intra mode derivation (DIMD) mode can be decided as the auxiliary prediction information. For example, in the case of the MERGE mode, the main prediction information can be a merge index. The intra picture prediction mode information as the auxiliary prediction information can be decided in only a part of the intra picture prediction modes by reducing the number of the intra picture prediction modes. The part of the intra picture prediction modes can mean a part of all available intra picture prediction modes. For example, the part of the intra picture prediction modes can mean a non-directional mode. Or, only one fixed intra picture prediction mode can be used as a default mode. For example, the default mode can be a Planar mode.

[0318] More than one auxiliary prediction information can be used instead of one auxiliary prediction information. As the weighting values respectively applied to each prediction block generated using the main prediction information and the auxiliary prediction information, the weighting value information can be decided by transmitting any set of information of the weighting value sets of the main prediction information and the auxiliary prediction information set in advance. The weighting values can be complex numbers, and the sum of the weighting values allocated to each prediction block is 1.

[0319] Alternatively, the weighting values can be determined based on the prediction modes of the surrounding blocks of the current block. For example, the surrounding blocks can be an upper surrounding block and a left surrounding block. For example, the upper surrounding block can be the rightmost block among the blocks existing in the upper side of the current block. For example, when the upper left coordinates of the current block of WxH size are (0, 0), the upper surrounding block can be the block including the pixel of (-1, H-1) coordinates. For example, the left surrounding block can be the lowermost block among the blocks existing in the left side of the current block. For example, when the upper left coordinates of the current block of WxH size are (0, 0), the left surrounding block can be the block including the pixel of (W-1, -1) coordinates. For example, the weighting values can be determined based on the number of intra-prediction blocks and inter-prediction blocks in the surrounding blocks. For example, in the case where the intra-prediction blocks are more than the inter-prediction blocks in the surrounding blocks, the intra-prediction blocks of the current block can be assigned with larger weighting values than the inter-prediction blocks. For example, in the case where both the upper surrounding block and the left surrounding block are intra-prediction, the intra-prediction blocks and the inter-prediction blocks of the current block can be assigned with weighting values of 0.75:0.25, respectively. In contrast, in the case where both the upper surrounding block and the left surrounding block are inter-prediction, the intra-prediction blocks and the inter-prediction blocks of the current block can be assigned with weighting values of 0.25:0.75, respectively. In the case where the upper surrounding block is inter-prediction and the left surrounding block is intra-prediction, the intra-prediction blocks and the inter-prediction blocks of the current block can be assigned with weighting values of 0.5:0.5, respectively.

[0320] The Figure 24 to Figure 28 The determined main prediction information, secondary prediction information, weighting and method, and weighting value information, etc. can be transmitted as the weighting and prediction information, but some information can be previously set in the image encoding apparatus and the image decoding apparatus and then used.

[0321] In the weighting and method, the previously set weighting and method and weighting value information can be used according to the types of the main prediction information and the secondary prediction information of the weighting and. Further, in the case where three or more prediction blocks are subjected to the weighting and, the weighting and of each prediction block can not be performed at once, but a method of, for example, first performing the weighting and of the main prediction information and the first secondary prediction information, and then sequentially performing the weighting and of the result and the second secondary prediction information, the third secondary prediction information (two prediction blocks at a time), etc. can be adopted.

[0322] In Figure 25 and Figure 26In the case where one of the main prediction mode or the sub prediction mode is the directional intra prediction mode, the weighting value information can be set in advance according to the direction of the directional mode. For example, in the case of the weighted sum of the motion information with the horizontal direction mode, a weighted sum method as shown in FIG. 8 can be used, and in the prediction block corresponding to the directional mode, higher weighting values are assigned in the order of W1, W2, W3, W4, and in the prediction block generated by the motion information, higher weighting values are assigned in the order of W4, W3, W2, W1. In the case of the vertical direction mode, a weighted sum method as shown in FIG. 9 can be used, and the other descriptions are the same as the above-described example. Figure 25 Figure 26

[0323] Figure 29 is a block diagram that outlines the configuration of an image encoding apparatus.

[0324] Referring to FIG. 29, Figure 29 the image encoding apparatus is an apparatus for encoding an image, and can include a block division unit 2901, a prediction unit 2902, a transform unit 2903, a quantization unit 2904, an entropy encoding unit 2905, an inverse quantization unit 2906, an inverse transform unit 2907, an addition unit 2908, a loop filter unit 2909, a memory unit 2910, and a subtraction unit 2911.

[0325] Figure 29 The respective configuration units illustrated in FIG. 29 are separately illustrated in order to represent different features and functions in the image encoding apparatus, and do not represent that the respective configuration units are configured by mutually separated hardware or one software unit. That is, although the respective configuration units are listed and described for convenience of explanation, at least two of the respective configuration units can be combined into one configuration unit, or one configuration unit can be divided into a plurality of configuration units to perform corresponding functions, and the embodiment in which the respective configuration units are integrated and the embodiment in which the respective configuration units are separated are included in the scope of the claims of the present application without departing from the essence of the present application.

[0326] Further, a part of the configuration elements can not be an essential configuration element for performing the essence of the present application, but can be an optional configuration element for improving performance. The present application can include only the essential configuration units for implementing the essence of the present application, excluding the configuration elements for improving performance, and a structure including the essential configuration elements, excluding the optional configuration elements for improving performance, is also included in the scope of the claims of the present application.

[0327] ​​The block division unit 2901 can divide the input image into at least one block. At this time, the input image can have various forms and sizes such as an image, a slice, a parallel block, a tile, or a slice. The block can refer to a coding unit (CU), a prediction unit (PU), or a transform unit (TU). The above division can be performed based on at least one of a quad tree, a binary tree, and a ternary tree division.

[0328] The quad tree refers to a division method of dividing a current coding block into exactly four. The binary tree refers to a division method of dividing a coding block into exactly two along a horizontal direction or a vertical direction. The ternary tree refers to a division method of dividing the above block into three lower-level blocks. For example, the three lower-level blocks can be obtained by dividing the width or height of the above upper-level block in a ratio of 1:2:1. By performing division based on the binary tree as described above, the block can have not only a square form but also a non-square form. In addition, a method of simultaneously using a plurality of division methods for division can be used.

[0329] The block division unit 2901 can divide a block of a maximum size, which needs to be encoded (hereinafter referred to as a maximum coding block), into a block of a minimum size, which needs to be encoded (hereinafter referred to as a minimum coding block).

[0330] The prediction unit 2902 can generate a prediction block using a pixel of a surrounding of a block, which currently needs to be predicted in a current original block (hereinafter referred to as a prediction block), or a pixel in a reference image, which has been previously encoded / decoded.

[0331] As the prediction block, one or more prediction blocks can be generated in the coding block. In the case where the prediction block in the coding block is one, the prediction block can be in the same form as the coding block.

[0332] The prediction technology of a video signal includes intra prediction and inter prediction.

[0333] The intra prediction is a method of generating a prediction block using a surrounding pixel of a current block.

[0334] The inter prediction is a method of searching for a block most similar to a current block from a reference image, which has been previously encoded / decoded, and generating a prediction block.

[0335] Further, the prediction unit 2902 can determine the optimum prediction mode of the prediction block by using various techniques such as rate-distortion optimization (RDO) in a residual block obtained by subtracting the prediction block from the original block after generating the prediction block. The calculation formula of the rate-distortion optimization (RDO) cost is shown in mathematical expression 8.

[0336] [mathematical expression 8]

[0337]

[0338] where D, R, J represent degradation due to quantization, code rate of the compressed stream, and rate-distortion (RD) cost, respectively, Φ represents the encoding mode, λ represents a Lagrangian multiplier, and can be used as a scaling correction coefficient for matching the unit of the error amount and the bit amount. In order to be able to select the optimum encoding mode in the encoding process, the rate-distortion (RD) cost value when the corresponding mode is applied should be smaller than the case where another mode is applied, and in the formula for calculating the rate-distortion (RD) cost value, the calculation can be performed while considering the bit rate and the error at the same time.

[0339] Figure 30 is a flowchart illustrating the flow in the prediction unit of the image encoding apparatus.

[0340] In the case where the intra prediction is performed using the original information and the reconstructed information 3001, the optimum intra prediction mode can be determined by using the rate-distortion (RD) cost value for each prediction mode 3002 and generating the prediction block.

[0341] In the case where the inter prediction is performed using the original information and the reconstructed information 3003, the rate-distortion (RD) cost value can be calculated for the SKIP mode, the MERGE mode, and the AMVP mode.

[0342] In the MERGE candidate search unit 3004, the candidate motion information set for the SKIP mode and the MERGE mode can be constructed. Further, in the corresponding candidate motion information set, the optimum motion information can be determined by using the rate-distortion (RD) cost value 3005.

[0343] In the AMVP candidate search unit 3006, the candidate motion information set for the AMVP mode can be constructed. Further, the optimum motion information can be determined by performing the motion estimation 3007 using the corresponding candidate motion information set.

[0344] The prediction block can be generated by performing motion compensation 3008 using the best motion information decided in each mode.

[0345] The above-described inter-picture prediction can be constituted by three modes (SKIP mode, MERGE mode, and AMVP mode). Motion information (prediction direction information, reference picture information, and motion vector) can be encoded according to each prediction mode.

[0346] The SKIP mode can decide the best prediction information using motion information of a reconstructed area. The SKIP mode can constitute a motion information candidate group in the reconstructed area, and generate a prediction block by using a candidate having the minimum rate-distortion (RD) cost value in the corresponding candidate group as prediction information. Here, since the method of constituting the motion information candidate group is the same as the method of constituting the motion information candidate group of the MERGE mode described below, the description thereof will be omitted in the present specification.

[0347] The MERGE mode is the same as the SKIP mode in terms of deciding the best prediction information using motion information of a reconstructed area. However, the difference between the two modes is that the SKIP mode searches for motion information having a prediction error of 0 from a motion information candidate group, whereas the MERGE mode searches for motion information having a prediction error of 0 from a motion information candidate group. As in the SKIP mode, a motion information candidate group can be constituted in a reconstructed area, and a prediction block can be generated by using a candidate having the minimum rate-distortion (RD) cost value in the corresponding candidate group as prediction information.

[0348] Figure 31 The method of generating a motion information candidate group of the SKIP mode and the MERGE mode is illustrated. The maximum number of the motion information candidate group can be decided to be the same value in the image encoding apparatus and the image decoding apparatus, or the corresponding number information can be transmitted in advance in a higher header (a header transmitted in a higher level than a block, such as a video parameter level, a sequence parameter level, a picture parameter level, etc.) of the image encoding apparatus.

[0349] Referring to Figure 31 In step S3101, four candidates can be selected from five spatially neighboring blocks of the current block in the same picture. Figure 33is a diagram illustrating the positions of spatial candidates. In addition, the positions of the spatial candidates can be changed to any block within the reconstruction region. The spatial candidates can be considered in the order of Al, A2, A3, A4, and A5, and the motion information of the spatial candidate block available in priority can be determined as the spatial candidate. In the case where there is repeated motion information, only the motion information of the candidate having a higher priority order can be considered.

[0350] In step S3102, one candidate can be selected from the two temporal candidate blocks. Figure 33 The positions of the temporal candidates are illustrated. The position of each candidate can be determined based on the block in the collocated image having the same position as the current block position of the current image. The collocated image can be set in the same condition in the image encoding apparatus and the image decoding apparatus within the reconstruction image. The temporal candidates can be considered in the order of Bl and B2 blocks, and the motion information of the candidate block available in priority can be determined as the temporal candidate.

[0351] For the method of determining the motion information of the temporal candidate, refer to Figure 34 .

[0352] Refer to Figure 34 The motion information of the candidate block (Bl, B2) in the collocated image refers to the prediction block located in the reference image B. (The reference image of each candidate block can be different from each other. In this specification, it is expressed as the reference image B for convenience.) With respect to the corresponding motion vector, the motion vector calculated by scaling the distance between the collocated image and the reference image B and the distance between the current image and the reference image A in the corresponding proportion and scaling the motion vector of the candidate block can be determined as the motion vector of the temporal candidate motion information. Mathematical expression 9 is the scaling mathematical expression.

[0353]

Mathematical Expression 9

[0354]

[0355] where MV represents the motion vector of the motion information of the temporal candidate block, MVscale represents the scaled motion vector, TB represents the temporal distance between the collocated image and the reference image B, and TD represents the temporal distance between the current image and the reference image A. In addition, the reference image A and the reference image B can be the same reference image. The motion vector scaled by the above-described method can be determined as the motion vector of the temporal candidate, and the reference image information of the temporal candidate motion information can be determined as the reference image of the current image, thereby deriving the motion information of the temporal candidate.

[0356] Furthermore, in step S3101 and step S3102, only in the case that the spatial candidate block and the temporal candidate block are coded in the inter picture prediction mode, the corresponding motion information is included into the motion information candidate group.

[0357] Step S3103 is executed only in the case that the maximum number of motion information candidate groups cannot be derived in step S3101 and step S3102, and is a step of adding a new bi-directional motion information candidate group using the combination of the motion information candidates derived in the previous steps. The bi-directional motion information candidate refers to sequentially selecting one of the previously derived history or future direction motion information and combining them into a new candidate.

[0358] Figure 35 The priority order of the bi-directional motion information candidate combination is given in the table of Figure 25 In addition to the combinations in the table of

[0359] The advanced motion vector prediction (AMVP) mode can determine the best motion information by performing motion estimation on the reference pictures of different prediction directions respectively. The prediction direction can be either a single direction using only one of the history / future directions, or a bi-directional direction using both the history and future directions. The prediction block can be generated by performing motion compensation using the best motion information determined by means of motion estimation. The motion information candidate group for performing motion estimation can be derived for the reference pictures of different prediction directions respectively. The corresponding motion information candidate group can be used as the starting position of motion estimation.

[0360] Figure 32 is a flow chart illustrating the derivation method of the motion information candidate group for motion estimation of the advanced motion vector prediction (AMVP) mode. The maximum number of the motion information candidate group can be determined as the same value in the image encoding device and the image decoding device, or the corresponding number information can be transmitted in advance in the upper header of the image encoding device.

[0361] Referring to Figure 32 , step S3201 is the same as step S3101, and step S3202 is the same as step S3102. Figure 31The description of step S3101 is different, the number of spatial candidates derived as (2) can be different, and the priority order for selecting the spatial candidates can also be different. In addition, since the description of S3201 is the same as that of step S3101, the repeated description will be omitted.

[0362] Since step S3202 is the same as the description of step S3102, the repeated description will be omitted.

[0363] In step S3203, when the currently derived post-selected contains repeated motion information, it can be deleted.

[0364] Since step S3204 is the same as the description of step S3104, the repeated description will be omitted.

[0365] Among the motion information candidates derived in the manner described above, the motion information candidate with the minimum rate-distortion (RD) cost value can be selected as the best motion information candidate, and the corresponding motion information is used as the basis to obtain the best motion information in the advanced motion vector prediction (AMVP) mode through the motion prediction process.

[0366] In addition, in the description of step S3201 and step S3202, only in the case of encoding the spatial candidate block and the temporal candidate block in the inter-picture prediction mode, the motion information derived using the corresponding motion information can be included in the motion information candidate group.

[0367] The transform unit 2903 can generate a transform block by transforming a residual block that is the difference between the original block and the prediction block. The transform block is the smallest unit used in the transformation and quantization processes. The transform unit 2903 can generate a transform block having transform coefficients by transforming a residual signal into a frequency region. Among them, as a method of transforming a residual signal into a frequency region, various transform techniques such as a transform based on a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), etc. can be used, whereby a residual signal can be transformed into a frequency region and transform coefficients can be generated. In order to facilitate the use of transform techniques, a matrix operation using a basis vector will be performed, and depending on the prediction mode used when encoding the prediction block, various different transform techniques can be used in combination when performing the matrix operation. For example, when performing intra prediction, depending on the prediction mode, a discrete cosine transform can be used in the horizontal direction and a discrete sine transform can be used in the vertical direction.

[0368] The quantization section 2904 can generate a quantized transform block by quantizing the transform block. That is, the quantization section 2904 can generate a quantized transform block (Quantized Transform Coefficient) having quantized transform coefficients by quantizing the transform coefficients of the transform block generated from the transform section 2903. As a quantization method, a method such as dead zone uniform threshold quantization (DZUTQ) or a quantization weighted matrix can be used, and various different quantization methods such as a modified quantization can also be used.

[0369] Furthermore, in the above, the case where the transform section 2903 and the quantization section 2904 are included in the image encoding apparatus is illustrated and described, but the transform section and the quantization section can also be selectively included in the image encoding apparatus. That is, the image encoding apparatus can generate a transform block by transforming a residual block without performing a quantization process, can perform only a quantization process without transforming a residual block into frequency coefficients, or can not perform either a transform or a quantization process. Even in the case where only a part of the transform section or the quantization section is performed or all processes are not performed in the image encoding apparatus, the block input to the entropy encoding section is generally referred to as a "quantized transform block".

[0370] The entropy encoding section 2905 can encode the quantized transform block and output a bitstream. That is, the entropy encoding section 2905 can encode the coefficients of the quantized transform block output from the quantization section 2904 using various encoding techniques such as entropy encoding, and generate and output a bitstream including additional information (for example, information related to a prediction mode, quantized coefficients, and the like) required when the corresponding block is decoded in the image decoding apparatus described later.

[0371] That is, the entropy encoding section 2905 can encode the coefficients of the quantized transform block output from the quantization section 2904 using various encoding techniques such as entropy encoding, and generate and output a bitstream including additional information (for example, information related to a prediction mode (the information related to a prediction mode can include motion information or in-picture prediction mode information decided in the prediction section, and the like), quantized coefficients, and the like) required when the corresponding block is decoded in the image decoding apparatus described later.

[0372] The inverse quantization section 2906 can reconstruct an inverse quantized transform block by inversely performing the quantization technique used in the quantization process on the quantized transform block.

[0373] The inverse transform unit 2907 reconstructs the residual block by performing inverse transform on the inverse quantization transform block using the same method as that used in the transform process. The inverse transform can be performed by inversely performing the transform method used in the transform unit.

[0374] Further, the inverse quantization unit and the inverse transform unit described above can perform inverse quantization and inverse transform by inversely using the quantization method and the transform method used in the quantization unit and the transform unit. Alternatively, in the case where only quantization is performed without transform in the transform unit and the quantization unit, only inverse quantization can be performed without inverse transform. Also, in the case where neither transform nor quantization is performed, the inverse quantization unit and the inverse transform unit can not perform inverse transform and inverse quantization, or can be omitted directly from the image encoding apparatus.

[0375] The addition operation unit 2908 reconstructs the current block by performing addition operation on the residual signal generated in the inverse transform unit and the prediction block generated by prediction.

[0376] The filter unit 2909 is a process of performing filtering on the entire image after reconstructing all the blocks in the current image, and can include deblocking filtering and sample adaptive offset (SAO), etc. Deblocking filtering refers to an operation for reducing blocking distortion generated in the process of encoding the image in block units, and sample adaptive offset (SAO) refers to an operation of minimizing the difference between the reconstructed image and the original image by performing addition operation or subtraction operation on the reconstructed pixels using a specific value.

[0377] The memory 2910 stores the reconstructed current block on which filtering is additionally performed in the loop filter unit after addition operation on the residual signal generated in the inverse transform unit and the prediction block generated by prediction, and can be used for prediction of the next block or the next image, etc.

[0378] The subtraction operation unit 2911 generates a residual block by performing subtraction operation on the current original block and the prediction block.

[0379] Figure 36 is a flowchart illustrating an encoding flow of the encoding information in the image encoding apparatus.

[0380] Referring to Figure 36 In step S3601, the action information of the skip (SKIP) mode is encoded.

[0381] Next, in step S3602, it is determined whether or not the SKIP mode is in operation. When it is determined in step S3602 that the SKIP mode is in operation, in step S3607, the MERGE candidate index information for the SKIP mode is encoded, and then the present flowchart ends.

[0382] In contrast, when it is determined in step S3602 that the SKIP mode is not in operation, in step S3603, the prediction mode is encoded.

[0383] Next, in step S3604, it is determined whether the prediction mode is an inter prediction mode or an intra prediction mode. When it is determined in S3604 that the prediction mode is an intra prediction mode, in step S3608, the intra prediction mode information is encoded, and then the present flowchart ends.

[0384] In contrast, when it is determined in S3604 that the prediction mode is an inter prediction mode, in step S3605, the operation information of the MERGE mode is encoded.

[0385] Next, in step S3606, it is determined whether or not the MERGE mode is in operation. When it is determined in step S3606 that the MERGE mode is in operation, the processing proceeds to step S3607, and the MERGE candidate index information for the MERGE mode is encoded, and then the present flowchart ends.

[0386] In contrast, when it is determined in step S3606 that the MERGE mode is not in operation, in step S3609, the prediction direction is encoded. The prediction direction can be one of a past direction, a future direction, and a bi-direction.

[0387] Next, in step S3610, it is determined whether or not the prediction direction is the future direction. When it is determined in step S3610 that the prediction direction is not the future direction, in step S3611, the reference picture index information of the past direction is encoded. Next, in step S3612, the MVD (Motion Vector Difference) information of the past direction is encoded. Next, in step S3613, the MVP (Motion Vector Predictor) information of the past direction is encoded.

[0388] In contrast, when it is determined in step S3610 that the prediction direction is the future direction or the bidirectional direction, after the completion of step S3613, it is determined in step S1313 whether the prediction direction is the past direction. Next, when it is determined in step S3614 that the prediction direction is not the past direction, the reference picture index information of the future direction is encoded in step S3615. Next, the motion vector difference (MVD) information of the future direction is encoded in step S3616. The motion vector prediction (MVP) information of the future direction can be encoded in step S3617, and then the flowchart ends.

[0389] Figure 37 is a block diagram schematically illustrating the configuration of the image decoding apparatus.

[0390] Referring to Figure 37 The image decoding apparatus is an apparatus for decoding an image, and can include an entropy decoding section 3701, an inverse quantization section 3702, an inverse transform section 3703, a prediction section 3704, an addition operation section 3705, a loop filter section 3706, and a memory section 3707.

[0391] Figure 37 The respective configuration sections illustrated in FIG. 1 are separately illustrated in order to represent different features and functions in the image decoding apparatus, and do not represent that the respective configuration sections are configured by mutually separated hardware or one software unit. That is, although the respective configuration sections are described in order to facilitate the explanation, at least two of the respective configuration sections can be combined into one configuration section, or one configuration section can be divided into a plurality of configuration sections to perform the corresponding functions, and the embodiment in which the respective configuration sections are integrated and the embodiment in which the respective configuration sections are separated are included in the scope of the claims of the present application without departing from the essence of the present application.

[0392] Further, some of the configuration elements can not be essential configuration elements for performing the essential functions in the present application, but can be optional configuration elements for improving performance. The present application can include only the configuration sections essential for implementing the essence of the present application except for the configuration elements for improving performance, and a structure including the essential configuration elements except for the optional configuration elements for improving performance is also included in the scope of the claims of the present application.

[0393] The encoding block in the image encoding apparatus can be referred to as a decoding block or can continue to be referred to as an encoding block in the image decoding apparatus.

[0394] The entropy decoding section 3701 can read various information required when decoding the corresponding block and the quantized transform coefficient by analyzing the bitstream transmitted from the image encoding apparatus.

[0395] The inverse quantization section 3702 can reconstruct an inverse quantization block having inverse quantization coefficients by inversely performing the quantization technique used in the quantization process on the quantization coefficients decoded in the entropy decoding section.

[0396] The inverse transform section 3703 can reconstruct a residual block having difference signals by performing inverse transform on the inverse quantization transform block using the same method as that used in the transform process, and can perform the inverse transform by inversely performing the transform technique used in the transform section.

[0397] The prediction section 3704 generates a prediction block using the prediction mode information decoded in the entropy decoding section, and can use the same method as that performed in the prediction section of the image encoding apparatus at this time.

[0398] The addition operation section 3705 can reconstruct a current block by performing addition operation on the residual signal reconstructed in the inverse transform section and the prediction block generated by prediction.

[0399] The filter section 3706 is a process of additionally performing filtering on the entire area of an image after reconstructing all blocks within the current image, and can include deblocking filtering, sample adaptive offset (SAO), and the like, and the detailed contents are the same as those described for the loop filter section of the image encoding apparatus.

[0400] The memory 3707 stores the reconstructed current block in which the residual signal generated in the inverse transform section and the prediction block generated by prediction are subjected to addition operation and then filtering is additionally performed in the loop filter section, and can be used to predict the next block or the next image, and the like.

[0401] Figure 38 is a flowchart illustrating the flow in the prediction section of the image decoding apparatus.

[0402] Referring to Figure 38 When the prediction mode is intra prediction, the best intra prediction mode information 3801 can be decided, and a prediction block can be generated by performing intra prediction 3802.

[0403] When the prediction mode is inter prediction, the best prediction mode 3803 can be decided from among SKIP, MERGE, and AMVP modes. Next, in the case of decoding by the SKIP mode or the MERGE mode, candidate motion information sets for the SKIP mode and the MERGE mode can be constituted in a merge (MERGE) candidate search section 3804. In the corresponding candidate motion information sets, the best motion information 3805 can be decided.

[0404] In the case of decoding by advanced motion vector prediction (AMVP), a candidate motion information set for the advanced motion vector prediction (AMVP) mode can be constituted in the advanced motion vector prediction (AMVP) candidate search section 3806. In the respective candidate motion information sets, the best motion information 3807 can be decided using the transmitted motion vector prediction (MVP) information. Next, a prediction block can be generated by performing motion compensation 3808 using the best motion information decided in each mode.

[0405] Figure 39 is a flowchart illustrating a decoding procedure of coding information in an image decoding apparatus.

[0406] Referring to Figure 39 In step S3901, the motion information of the skip (SKIP) mode is decoded.

[0407] Next, in step S3902, it is determined whether or not the skip (SKIP) mode is in operation. When it is determined in step S3902 that the skip (SKIP) mode is in operation, in step S3907, the merge (MERGE) candidate index information for the skip (SKIP) mode is decoded, and then the present flowchart is ended.

[0408] On the contrary, when it is determined in step S3902 that the skip (SKIP) mode is not in operation, in step S3903, the prediction mode is decoded.

[0409] Next, in step S3904, it is determined whether the prediction mode is an inter prediction mode or an intra prediction mode. When it is determined in S3904 that the prediction mode is an intra prediction mode, in step S3908, the intra prediction mode information is decoded, and then the present flowchart is ended.

[0410] On the contrary, when it is determined in S3904 that the prediction mode is an inter prediction mode, in step S3905, the motion information of the merge (MERGE) mode is decoded.

[0411] Next, in step S3906, it is determined whether or not the merge (MERGE) mode is in operation. When it is determined in step S3906 that the merge (MERGE) mode is in operation, the processing can be branched to step S3907, and the merge (MERGE) candidate index information for the merge (MERGE) mode is decoded, and then the present flowchart is ended.

[0412] On the contrary, when it is determined in step S3906 that the MERGE mode does not operate, the prediction direction is decoded in step S3909. Here, the prediction direction can be one of the past direction, the future direction, and the bi-direction.

[0413] Next, in step S3910, it is determined whether the prediction direction is the future direction. When it is determined in step S3910 that the prediction direction is not the future direction, the reference picture index information of the past direction is decoded in step S3911. Next, in step S3912, the motion vector difference (MVD) information of the past direction is decoded. Next, in step S3913, the motion vector predictor (MVP) information of the past direction is decoded.

[0414] Further, when it is determined in step S3910 that the prediction direction is the future direction or the bi-direction, after step S3913 is completed, in step S3914, it is determined whether the prediction direction is the past direction. Next, when it is determined in step S3914 that the prediction direction is not the past direction, in step S3915, the reference picture index information of the future direction is decoded. Next, in step S3916, the motion vector difference (MVD) information of the future direction is decoded. Next, in step S3917, the motion vector predictor (MVP) information of the future direction can be decoded, and then the flowchart ends.

[0415] Next, various embodiments to which the present application is applied will be described in more detail with reference to the accompanying drawings.

[0416] The image encoding / decoding method to which an embodiment of the present application is applied can be included as one of the inter-picture prediction modes, including a decoder-side motion vector derivation (DMVD) mode.

[0417] The decoder-side motion vector derivation (DMVD) mode can refer to a mode in which a part or all of the motion information is directly derived in the image decoding apparatus without being transmitted from the image encoding apparatus.

[0418] That is, the decoder-side motion vector derivation (DMVD) mode can refer to a mode in which the motion information is derived by the image decoding apparatus itself. Alternatively, the decoder-side motion vector derivation (DMVD) mode can refer to a mode in which the initial motion information derived based on the information transmitted from the image encoding apparatus is corrected by the image decoding apparatus itself.

[0419] Figure 40 is a schematic diagram illustrating a prediction section in a video encoding apparatus to which an embodiment of the present application is applied.

[0420] Referring to Figure 40 , the same processes as steps 3001 to 3007 in FIG. 3 can be performed in steps 4001 to 4007. That is, a prediction block can be generated using the optimal intra prediction mode decided through steps 4001 to 4007, or motion compensation 4010 can be performed using the optimal motion information decided through steps 4003 to 4007 and a prediction block can be generated. Figure 30

[0421] Figure 40 The video encoding apparatus in FIG. 4 can perform a motion information decision method using a decoder-side motion vector derivation (DMVD) mode in a case where inter prediction is additionally performed. In order to perform motion estimation in the same manner in the video encoding apparatus, initial motion information 4008 can be decided using motion information of a reconstructed region. Next, optimal motion information can be decided by performing motion estimation 4009 using the decided initial motion information, and a prediction block can be generated by performing motion compensation 4010 using the decided optimal motion information. Here, the motion information can include at least one of a motion vector, a reference picture index, and prediction direction information.

[0422] Figure 41 is a schematic diagram illustrating a prediction section in a video decoding apparatus to which an embodiment of the present application is applied.

[0423] Referring to Figure 41 , the same processes as steps 3801 to 3807 in FIG. 3 can be performed in steps 4101 to 4107. That is, a prediction block can be generated using the intra prediction mode decided through steps 4101 to 4102, or motion compensation 4110 can be performed using the motion information decided through steps 4103 to 4007 and a prediction block can be generated. Figure 38

[0424] Figure 41 ​​The video decoding device in the image decoding device can perform a motion information decision method using a decoder-side motion vector derivation (DMVD) mode in the case of additionally performing inter-picture prediction. In order to perform motion prediction in the same manner in the image encoding / decoding device, initial motion information 4108 can be decided using motion information of a reconstructed region. Next, motion information can be decided by performing motion prediction 4109 using the decided initial motion information, and a predicted block can be generated by performing motion compensation 4110 using the decided motion information. That is, in the case of the decoder-side motion vector derivation (DMVD) mode, the video decoding device can correct the initial motion information by itself without signaling information and derive motion information for motion compensation.

[0425] Further, Figure 41 The decoder-side motion vector derivation (DMVD) initial motion information exploration unit 4108 in the image decoding device can decide motion information of a merge (MERGE) candidate derived through steps 4104-4105 as decoder-side motion vector derivation (DMVD) initial motion information, or can decide motion information derived through steps 4106-4107 as decoder-side motion vector derivation (DMVD) initial motion information.

[0426] Figure 42 FIG. 4 is a schematic diagram illustrating the decoder-side motion vector derivation (DMVD) initial motion information exploration unit 4008, 4108 in the image encoding / decoding device.

[0427] Referring to Figure 42 the table, Figure 42 The index in the table can refer to a priority order of the initial motion information. As the initial motion information, at least one of candidate motion information of a skip (SKIP) mode, candidate motion information of an advanced motion vector prediction (AMVP) mode, candidate motion information of a merge (MERGE) mode, motion information possessed by a sub-block in a reconstructed region located at a top, a left side, a top-left corner, a top-right corner, a bottom-left corner of a current block, or zero motion information can be used. Further, various different motion information candidates derived through other reconstructed information can be used.

[0428] As described above, the initial motion information can be motion information derived using an inter-picture prediction mode such as a skip (SKIP) mode, an advanced motion vector prediction (AMVP) mode, or a merge (MERGE) mode, as described in Figure 31 to Figure 35

[0429] Figure 43 FIG. 4 is a schematic diagram illustrating the decoder-side motion vector derivation (DMVD) initial motion information exploration unit 4008, 4108 in the image encoding / decoding device.

[0430] ​The video encoding device can determine the optimal motion information after performing a decoder-side motion vector derivation (DMVD) mode using initial motion information determined by the decoder-side motion vector derivation (DMVD) initial motion information search unit.

[0431] The video decoding device can determine the motion information needed for motion compensation after performing a decoder-side motion vector derivation (DMVD) mode using initial motion information determined by the decoder-side motion vector derivation (DMVD) initial motion information search unit. Here, performing a decoder-side motion vector derivation (DMVD) mode can mean correcting the initial motion information.

[0432] The decoder-side motion vector derivation (DMVD) mode can include a mode using a template (hereinafter referred to as a "template matching mode") and a mode not using a template (hereinafter referred to as a "bi-directional matching mode").

[0433] In the case of using the bi-directional matching mode 4301, the uni-directional motion vectors of the respective initial motion information can be scaled in a linear manner to reference pictures in the opposite prediction direction. Here, the scaling of the motion vectors can be performed in a manner proportional to the distance between the current picture and the reference pictures in the respective direction. After determining the motion vectors 4302 in both directions by the above-described manner, the motion vector of the respective direction that minimizes the difference between the prediction blocks indicated by the motion vectors of the respective direction can be determined as the optimal motion information 4304.

[0434] In addition, in the case of the initial motion information being bi-directional motion information having both the motion information of the history direction and the motion information of the future direction, the motion information that minimizes the difference between the prediction blocks indicated by the motion information of the respective direction can be determined as the optimal motion information 4304.

[0435] As an example, the motion vector that minimizes the difference between the prediction block indicated by the initial motion vector of the history direction and the prediction block indicated by the initial motion vector of the future direction can be determined as the optimal motion vector. Here, the determined optimal motion vector can be used as the motion information of the current block.

[0436] Figure 45 is a schematic diagram illustrating a method of searching for a motion vector based on the prediction blocks indicated by the motion vectors of the history and future directions of the current block in the bi-directional matching mode. The position that minimizes the difference between the prediction blocks of the respective direction in 4301 can be searched for by searching for the position that minimizes the difference between the prediction blocks of the respective direction in 4301. Figure 45 After the position that minimizes the difference between the prediction blocks of the respective direction in 4301 is searched for, the motion vectors of the respective direction can be corrected to point to the searched position. At this time, the corrected motion vector can be used as the optimal motion vector or the motion vector used for performing motion compensation (or inter prediction).

[0437] In case of using the template matching mode 4305, a template region 4306 can be decided from the reconstructed region. The size and shape of the template block can be decided by various methods such as deciding the left side and the top end of the reconstructed region adjacent to the periphery of the current block as the template block. By using the motion estimation 4307 of the corresponding template block, the optimal motion information can be decided. Figure 44 is a diagram illustrating a method of deciding the prediction block of the current block by using the block adjacent to the corresponding template block after exploring the prediction block most similar to the template block in the single template matching mode.

[0438] In the process of exploring the optimal motion information in the bi-directional matching mode and the template matching mode, the exploration pattern can be decided in various ways. Among them, the process of exploring the final motion information can be the motion estimation process.

[0439] Figure 46 is a diagram illustrating the process of performing exploration with a diamond-shaped exploration pattern.

[0440] Referring to Figure 46 P1 can be decided based on the initial motion information, and the cost value of the motion estimation of the pixel S1 inside the diamond shape including P1 can be calculated. P1 can be used as the starting point of the exploration.

[0441] After calculating it, the position P2 with the minimum cost value can be used as the reference to calculate the cost value of the motion estimation of the pixel S2 inside the diamond shape again, and the position P3 with the minimum cost value can be decided. The above-mentioned process is performed repeatedly in the exploration region, and the exploration process is continued until the cost value of the central pixel inside the diamond shape reaches the minimum, so that the motion information indicating the optimal position is decided as the motion information of the template matching mode or the bi-directional matching mode. In addition, Figure 46 In the process of exploring the position with the minimum cost value in the exploration pattern, the predefined number of times can be repeated.

[0442] As Figure 44 and Figure 45 illustrated, in the process of exploring (or motion estimation) the optimal motion information in the bi-directional matching mode and the template matching mode, the exploration region (or the motion estimation region) can be limited 4303 in the reference image.

[0443] Especially, in the process of performing the template matching mode, the template settable region (hereinafter referred to as "template estimation region") of the current block in the current image can also be limited 4303.

[0444] If the search region is not limited in the video decoding device and the search is repeatedly performed until the best motion position is found, the calculation complexity of the video decoding device will inevitably become very high. Moreover, if the template search region is not limited, the parallel processing design of the current picture will become very difficult, and thus the processing speed needs to be improved by limiting it.

[0445] The limitation condition can be set such that the prediction block of the history / future direction in the bi-directional matching mode does not exceed the motion search region, and the template block of the prediction block in the reference picture in the template matching mode does not exceed the motion search region. In addition, the limitation condition can be set such that the template block of the current block in the current picture in the template matching mode does not exceed the template search region.

[0446] In addition, the search region can be set as a window of an arbitrary size (I x J). The size of the window can be changed according to the distance between the current picture and the reference picture, or the size of the window can be transmitted in a higher level header (e.g., a video parameter set level, a sequence parameter set level, a picture parameter set level, a slice level, etc.).

[0447] In addition, the same size of the search region can be used in the video encoding device and the video decoding device.

[0448] In addition, the search region cannot be determined to have a size that exceeds the partition boundary determined in a higher level header such as a slice and a parallel block.

[0449] However, in the bi-directional matching mode, because it is premised that the motion vectors in different directions have linear directionality, the search regions in different directions can be different from each other.

[0450] In addition, the search region can be determined to include the largest coding block of the current block.

[0451] In addition, a window of an arbitrary size (K x L) can be set in the same manner as the search region, and the corresponding window can be determined as the template search region. The size of the window can be transmitted in a higher level header, or the same size of the window can be used in the video encoding / decoding device. In addition, it cannot be determined to have a size that exceeds the partition boundary determined in a higher level header such as a slice and a parallel block. A template region that is not included in the template search region will not be used as a template for the template matching mode.

[0452] The cost value of the motion information search process is the sum of the prediction error amount and the hypothetical bit amount of the motion information. The prediction error can be obtained by various calculation methods such as the sum of absolute difference (SAD), the sum of absolute hadamard transform difference (SATD), the sum of square difference (SSD), and the like. Mathematical expression 10, mathematical expression 11, and mathematical expression 12 are mathematical expressions describing the calculation methods of the sum of absolute difference (SAD), the sum of absolute hadamard transform difference (SATD), and the sum of square difference (SSD), respectively.

[0453] [mathematical expression 10]

[0454] Diff(i, j) = Template(i, j) - PredBlk(i, j)

[0455] [mathematical expression 11]

[0456] DiffT(i, j) = HT(Template(i, j) - PredBlk(i, j))

[0457] [mathematical expression 12]

[0458] Diff(i, j) = Template(i, j) - PredBlk(i, j)

[0459] where i, j represent the positions of pixels, Template(i, j) represents the pixels of the template block, and PredBlk(i, j) represents the pixels of the prediction block. The HT() function in mathematical expression 4 is a function value obtained by performing a Hadamard transform on the difference block of the template block and the prediction block. The hypothetical bit amount of the motion information is not the information actually transmitted, but a value calculated for the hypothetical bit amount of the motion information expected in the same manner in the image encoding apparatus and the image decoding apparatus. For example, the hypothetical bit amount can be determined by calculating the difference vector size of the motion vector of the initial motion information and the motion vector in the motion information currently performing motion estimation. In addition, the hypothetical bit amount of the motion information can be calculated using the bit amount of the reference image information.

[0460] Figure 47 is a flowchart of encoding the coded information according to an embodiment.

[0461] Referring to Figure 47In step S4701, the action information of the SKIP mode is encoded. In step S4702, it is determined whether or not the action of the SKIP mode is performed. When the action of the SKIP mode is determined in step S4702, in step S4703, the action information of the DMVD mode is encoded. In step S4704, it is determined whether or not the action of the DMVD mode is performed. When the action of the DMVD mode is determined in step S4704, in step S4705, the DMVD information is encoded, and then the present flowchart is ended. The DMVD information is flag information indicating whether the bi-directional matching mode or the template matching mode is selected. If none of the top and left template blocks of the current block is included in the template prediction area, the DMVD information is not encoded and is determined as the bi-directional matching mode. When the action of the DMVD mode is not determined in step S4704, in step S4706, the MERGE candidate index information for the SKIP mode is encoded, and then the present flowchart is ended. When the action of the SKIP mode is not determined in step S4702, in step S4707, the prediction mode information is encoded. In step S4708, it is determined whether or not the prediction mode is the inter prediction. When the prediction mode is determined as the intra prediction in step S4708, in step S4709, the intra prediction information is encoded, and then the present flowchart is ended. When the prediction mode is determined as the inter prediction in step S4708, in step S4710, the MERGE mode action information is encoded. In step S4711, it is determined whether or not the action of the MERGE mode is performed. When the action of the MERGE mode is determined in step S4711, in step S4712, the DMVD mode action information is encoded. When the action of the DMVD mode is determined in step S4713, in step S4714, the DMVD information is encoded, and then the present flowchart is ended. When the action of the DMVD mode is not determined in step S4713, in step S4706, the MERGE candidate index information for the MERGE mode is encoded, and then the present flowchart is ended. When the action of the MERGE mode is not determined in step S4711, steps S4715 to S4723 are executed, which is the same as the process of steps S4701 to S4709. Figure 36The steps S3609 to S3617 in FIG. 36 are the same as those of steps S3609 to S3617 in FIG. 35. After the step S4723 is completed, the present flowchart will be ended.

[0462] Figure 48 FIG. 48 is a flowchart of an embodiment of a process of decoding coding information.

[0463] Referring to Figure 48 In the step S4801, the action information of the SKIP mode will be decoded. In the step S4802, it will be determined whether or not the action of the SKIP mode is performed. When the action of the SKIP mode is determined in the step S4802, in the step S4803, the action information of the DMVD mode will be decoded. In the step S4804, it will be determined whether or not the action of the DMVD mode is performed. When the action of the DMVD mode is determined in the step S4804, in the step S4805, the DMVD information will be decoded, and then the present flowchart will be ended. When the action of the DMVD mode is not determined in the step S4804, in the step S4806, the MERGE candidate index information for the SKIP mode will be decoded, and then the present flowchart will be ended. When the action of the SKIP mode is not determined in the step S4802, in the step S4807, the prediction mode information will be decoded. In the step S4808, it will be determined whether or not the prediction mode is the inter prediction. When the prediction mode is determined to be the intra prediction in the step S4808, in the step S4809, the intra prediction information will be decoded, and then the present flowchart will be ended. When the prediction mode is determined to be the inter prediction in the step S4808, in the step S4710, the MERGE mode action information will be decoded. In the step S4811, it will be determined whether or not the action of the MERGE mode is performed. When the action of the MERGE mode is determined in the step S4811, in the step S4812, the DMVD mode action information will be decoded. When the action of the DMVD mode is determined in the step S4813, in the step S4814, the DMVD information will be decoded, and then the present flowchart will be ended. When the action of the DMVD mode is not determined in the step S4813, in the step S4806, the MERGE candidate index information for the MERGE mode will be decoded, and then the present flowchart will be ended. When the action of the MERGE mode is not determined in the step S4811, the steps S4815 to S4823 will be executed, which are the same as those of steps S3619 to S3627 in FIG. 36.Figure 39 The descriptions of steps S3909 to S3917 are the same. This flowchart will end after step S4823 is completed.

[0464] Next, when determining the template region from the reconstructed region surrounding the current block in this embodiment, multiple sub-template blocks can be used instead of a single template block. The method for deriving the motion information of each sub-template block using the template matching pattern described above and using the corresponding motion information as candidate motion information for skip, merge, and advanced motion vector prediction (AMVP) predictions will be explained in detail below.

[0465] Figure 49 This is a flowchart illustrating a method for deriving candidate motion information for skip and merge modes using template matching, according to one embodiment of the present invention.

[0466] See Figure 49 In step S4901, the reconstructed motion information in the sub-template block is used as the initial motion information and processed as described above. Figure 43 The processes 4305, 4306, 4304, and 4307 in the text derive the optimal motion information for the sub-template blocks. At this point, the number of sub-template blocks can be L (L is an integer ≥ 1).

[0467] Figure 51 This is a schematic diagram illustrating an example of determining sub-template blocks from the reconstruction area surrounding the current block.

[0468] See Figure 51 The sub-template blocks can be set according to the examples in 5101 to 5103.

[0469] exist Figure 51 In template 5101, sub-template blocks can be determined at the bottom left (template A), top left (template B), top left (template C), and top right (template D) of the current block. Each sub-template block can be determined to have a variety of different sizes and shapes.

[0470] Figure 51 The method in 5102 is the same as that in 5101. The sub-template blocks can be determined from the lower left (template A), upper left (template B), upper left (template C), and upper right (template D) of the current block. However, the difference is that all the reconstruction areas on the left and upper sides adjacent to the current block are used.

[0471] Figure 51 Method 5103 is a method for generating sub-template blocks that takes into account both the sub-template block generation methods in 5101 and 5102.

[0472] In addition, sub-template blocks can be generated from the reconstruction areas surrounding the current block using various methods, such as determining the reconstruction areas to the left and top of the current block as a sub-template block.

[0473] In step S4902, M space candidates can be selected (M is an integer ≥ 1). The generation method is similar to that used for... Figure 31 The description of step S3101 is the same. In step S4903, N time candidates can be selected (N is an integer ≥ 1). The generation method is the same as for... Figure 31 The description of step S3102 is the same. Steps S4904 and S4905 are the same as those in the previous section. Figure 31 The descriptions of steps S3103 and S3104 in the flowchart are the same. The order of each step can be changed in this flowchart. Furthermore, if previously derived motion information in each step is the same as the currently derived motion information, the corresponding sufficient candidates can be deleted.

[0474] Figure 50 This is a flowchart illustrating a method for deriving candidate motion information for an Advanced Motion Vector Prediction (AMVP) mode using a template matching pattern, applicable to one embodiment of the present invention.

[0475] See Figure 50 The explanation of step S5001 and the Figure 49 The description of step S4901 is the same. In step S5002, M space candidates can be selected (M is an integer ≥ 1). The generation method is the same as for... Figure 32 The description of step S3201 is the same. In step S5003, N time candidates can be selected (N is an integer ≥ 1). The generation method is the same as for... Figure 32 The description of step S3202 is the same. Step S5004 is the same as that of... Figure 32 The description of step S3204 is the same. The order of each step can be changed in this flowchart. Furthermore, if previously derived motion information in each step is the same as the currently derived motion information, the corresponding sufficient candidates can be deleted.

[0476] Next, in this embodiment, we will describe the method of deriving corrected candidate motion information from candidate motion information in skip and merge modes using decoder-side motion vector derivation (DMVD) mode (template matching mode or bidirectional matching mode), and the method of rearranging candidate motion information in skip and merge modes using the cost of template matching.

[0477] Figure 52 is a table for explaining a method of correcting candidate motion information of a SKIP, MERGE mode using a decoder side motion vector derivation (DMVD) mode.

[0478] Referring to Figure 52 , Figure 52 Table 5201 in Figure 31 or Figure 49 is a result of deriving candidate motion information of a SKIP, MERGE mode using a method in In the present example, the number of candidate motion information is 5. The derived candidate motion information can have motion information of a history / future direction.

[0479] Different candidate motion information can obtain corrected motion information through the above-described process in Figure 43 In Table 5202 in Figure 52 , some of the candidate motion information of Table 5201 is corrected. Among them, the correction of the candidate motion information can be performed on all of the candidate motion information, or can be performed on only some of the candidate motion information by, for example, performing the correction on only a part of the candidate motion information having a high priority order.

[0480] In Table 5202 in Figure 52 , three candidate motion information of a higher level is corrected. In Table 5202 in Figure 52 , the motion information of the 1st and 3rd candidate motion information is corrected, but the 2nd is not. This can mean that the 2nd candidate motion information does not need to be corrected. As an example of the 1st candidate motion information, only the motion vector in the motion information can be corrected, and as an example of the 3rd candidate motion information, the motion vector and the reference picture index in the motion information can be corrected.

[0481] As described above, the candidate motion information corrected using the decoder side motion vector derivation (DMVD) mode can be used for motion compensation of the current block. In addition, in order to be used for a block encoded / decoded after the current block, the candidate motion information corrected using the decoder side motion vector derivation (DMVD) mode can be stored. In the case described above, the candidate motion information corrected using the decoder side motion vector derivation (DMVD) mode can be used for deriving motion information of a block encoded / decoded after the current block.

[0482] In addition, in the case where the reference picture is changed, the motion search area in step 4303 of the above-described Figure 43 may also be changed compared to the original reference picture. For example, referring to Figure 53In the case where the original candidate motion information of the SKIP and MERGE modes is corrected using the template matching mode, the reference picture can also be corrected. In this case, the motion search area can also be changed, and specifically, the motion search area can be moved according to the linear directionality of the motion vector. When the reference picture is farther from the current picture, the motion search area can be further expanded or reduced on the premise of containing the current motion search area.

[0483] Figure 54 Table 5401 in

[0484] Figure 54 Table 5401 in Figure 52 Figure 31 Figure 49 is the result of deriving the candidate motion information of the SKIP and MERGE modes using the method in

[0485] In Table 5402 in Figure 54 , the template matching mode cost (hereinafter referred to as "cost") is calculated using the candidate motion information derived in Table 5401. Table 5402 is the result of rearranging the candidate motion information in order from low to high according to the corresponding cost. When the candidate motion information contains both historical and future motion information, the cost is calculated using the candidate motion information in different directions, and the average cost in different directions is determined as the cost of the corresponding candidate motion information. Alternatively, the cost of the candidate motion information with a lower cost in different directions can be determined as the cost of the corresponding candidate motion information.

[0486] Referring to the above-described Figure 32 , in the candidate motion information of the AMVP mode, the best candidate motion information, i.e., the MVP information, can be encoded in the entropy encoding unit and decoded in the entropy decoding unit. According to the above-described method, the template matching mode cost of each candidate motion information of the AMVP mode is calculated using the template matching mode. Next, the candidate motion information can be arranged in order from low to high according to the template matching mode cost, and the candidate motion information with the lowest corresponding cost is determined as the MVP information. In this way, the MVP information can be reconstructed in the video decoding device without being encoded in the entropy encoding unit.

[0487] ​​Next, the method of calculating the weighted cost value from the template region when calculating the cost value of the template matching mode in the template matching mode commonly referred to in the above embodiments will be described in detail in this embodiment.

[0488] Figure 55 is a schematic diagram for explaining the method of dividing the template of the template matching mode into sub-templates and applying different weighted values to the cost value of each sub-template according to an embodiment of the present application.

[0489] The weighted values of the cost values calculated in different sub-templates can be different from each other. Mathematical expression 13 is a general mathematical expression for calculating the cost value.

[0490] [mathematical expression 13]

[0491]

[0492] In mathematical expression 13, TempDist refers to the prediction error amount in the cost value of the motion estimation of the template matching mode. The prediction error amount is equal to the sum of the product of the prediction error amount of each sub-template block, i.e., SubTempDis ti, and the weighted value of the corresponding sub-template block, i.e., Wi. i represents the index information of the sub-template block.

[0493] Figure 55 5501 in FIG. 55 is an example of a template block formed using sub-template blocks of the same size. The sub-template blocks can be set to the same size in a variety of different ways.

[0494] Figure 55 5502 in FIG. 55 is an example of a template block formed using sub-template blocks based on the actual partition boundary line of the reconstructed block.

[0495] Figure 55 5503 in FIG. 55 is an example of a template block formed using sub-template blocks similar in the characteristics of the reconstructed pixels. Here, the characteristics of the reconstructed pixels can be the reconstructed pixel values themselves or the motion information of the reconstructed pixels. In addition, the template block can be formed using sub-template blocks partitioned by a variety of different methods.

[0496] According to the positions of the sub-template blocks decided by the above-described methods, the weighted values of the prediction error amounts can be changed.

[0497] As the weighted value decision method according to an embodiment of the present application, the weighted value of the error amount can be decreased as the sub-template block is closer to the pixel located at the upper left end of the current block, and the weighted value of the error amount can be increased as the sub-template block is farther from the pixel located at the upper left end of the current block. The opposite method can also be used.

[0498] As another embodiment of the application, the weighting value determination method can change the weighting value of each sub-template block according to the similarity between the motion information of each sub-template block and the motion information currently being used for motion estimation. The weighting value can be decreased as the similarity increases and increased as the similarity decreases, or vice versa.

[0499] Figure 56 is a flowchart illustrating a video decoding method according to an embodiment of the application.

[0500] Referring to Figure 56 In step 5601, the video decoding device can determine an initial motion vector of the current block using the motion vector of the reconstructed region.

[0501] The initial motion vector can include a motion vector of a history direction and a motion vector of a future direction. That is, the initial motion vector can be bi-directional predicted motion information.

[0502] In addition, the motion vector of the reconstructed region can be a candidate motion information of a merge mode. The candidate motion information of the merge mode can include at least one of motion information of a spatial candidate block of the current block and motion information of a temporal candidate block.

[0503] Next, in step 5602, the video decoding device can search for a motion vector of the current block based on the initial motion vector determined in step 5601.

[0504] Specifically, the step of searching for the motion vector can search for the motion vector using a difference between a prediction block of the history direction indicated by the motion vector of the history direction and a prediction block of the future direction indicated by the motion vector of the future direction.

[0505] As an example, the step of searching for the motion vector can search for the motion vector based on a prediction error generated by applying a difference between the motion vector of the history direction and the prediction block of the future direction to a sum of absolute difference (SAD) calculation method.

[0506] In addition, the step of searching for the motion vector can search for the motion vector within a search region of a predefined size.

[0507] Next, in step 5603, the video decoding device can generate a prediction sample of the current block based on the motion vector searched for in step 5602.

[0508] In addition, Figure 56The image decoding method may further include a step of decoding decoder-side motion vector derivation (DMVD) mode motion information. This step can be performed after step S2801. If the decoder-side motion vector derivation (DMVD) mode motion information indicates a decoder-side motion vector derivation (DMVD) mode action, a step of exploring the motion vector of the current block based on the initial motion vector can be performed.

[0509] exist Figure 56 The motion vectors explored in the image decoding method can be used in the motion vector derivation of blocks to be decoded after the current block. The explored motion vectors can refer to those corrected using the decoder-side motion vector derivation (DMVD) mode.

[0510] Figure 57 This is a flowchart illustrating an image encoding method applicable to one embodiment of the present invention.

[0511] See Figure 57 In step 5701, the image encoding device can use the motion vector of the reconstructed region to determine the initial motion vector of the current block.

[0512] The initial motion vector can include motion vectors from historical directions and motion vectors from future directions. That is, the initial motion vector can be predicted motion information in both directions.

[0513] Furthermore, the motion vector of the reconstructed region can be candidate motion information from the MERGE pattern. Specifically, the candidate motion information from the MERGE pattern can include at least one of the motion information of spatial candidate blocks and the motion information of temporal candidate blocks for the current block.

[0514] Next, in step 5702, the image encoding device can explore the optimal motion vector for the current block based on the initial motion vector.

[0515] Specifically, the steps for exploring the optimal motion vector can be to utilize the difference between the predicted block of the historical direction indicated by the motion vector of the historical direction and the predicted block of the future direction indicated by the motion vector of the future direction.

[0516] As an example, the steps for exploring the optimal motion vector can be based on the prediction error generated by applying the difference between the motion vector in the historical direction and the predicted block in the future direction to the sum of absolute errors (SAD).

[0517] Furthermore, the step of exploring the optimal motion vector can be performed within a predefined exploration area.

[0518] Next, in step 5703, the image encoding device can encode the motion information of the decoder-side motion vector derivation (DMVD) mode by determining whether or not the optimal motion vector is used.

[0519] exist Figure 57 The optimal motion vector discovered in the image coding method can be used in the motion vector derivation of blocks encoded after the current block. The optimal motion vector discovered can refer to the motion vector corrected using the decoder-side motion vector derivation (DMVD) mode.

[0520] pass Figure 57 The bitstream (or encoded data) generated by the image encoding method can be stored on a computer-readable, non-transitory recording medium. Furthermore, the bitstream may also include decoder-side motion vector derivation (DMVD) mode motion information.

[0521] Figure 58 This is a flowchart illustrating the image encoding device.

[0522] See Figure 58 The image encoding apparatus is an apparatus for encoding images, and may include a block segmentation unit 5801, a prediction unit 5802, a transform unit 5803, a quantization unit 5804, an entropy encoding unit 5805, an inverse quantization unit 5806, an inverse transform unit 5807, an addition unit 5808, a loop filter unit 5809, a memory unit 5810, and a subtraction unit 5811.

[0523] Figure 58 The various components illustrated are shown separately to illustrate different features and functions of the image encoding apparatus, and do not represent that each component is composed of separate hardware or software units. That is, although the various components are listed for ease of explanation, at least two components can be combined into one component, or a component can be divided into multiple components to perform corresponding functions. The embodiments where the various components are integrated or separated as described above are included within the scope of the claims of this invention without departing from the essence of the invention.

[0524] Furthermore, some of the elements of an embodiment of the present application can not be required; thus, not all of the elements are required to practice an embodiment of the application. An embodiment of the application can be specifically adapted to persons of ordinary skill in the art.

[0525] The block division unit 5801 can divide the input image into at least one block. At this time, the input image can have various forms and sizes such as an image, a slice, a tile, a brick, or a segment. The block can refer to a coding unit (CU), a prediction unit (PU), or a transform unit (TU). The above-described division can be performed based on at least one of a quad tree, a binary tree, and a ternary tree division.

[0526] The quad tree refers to a division method in which a current coding block is exactly divided into four. The binary tree refers to a division method in which a coding block is exactly divided into two along a horizontal direction or a vertical direction. The ternary tree refers to a division method in which the above-described block is divided into three lower-level blocks. For example, the three lower-level blocks can be obtained by dividing the width or height of the above-described upper-level block in a ratio of 1:2:1. By performing division based on the binary tree as described above, a block can have not only a square form but also a non-square form. In addition, a method of simultaneously using a plurality of division methods for division can be used.

[0527] The prediction unit 5802 can generate a prediction block using a pixel of a surrounding of a block in which prediction is currently required in a current original block (hereinafter, referred to as a prediction block) or a pixel in a reference image in which encoding / decoding has been completed.

[0528] As the prediction block, one or more prediction blocks can be generated in the coding block. In the case in which the prediction block in the coding block is one, the prediction block can be in the same form as the coding block.

[0529] The prediction technique of a video signal includes intra prediction and inter prediction.

[0530] The intra prediction is a method of generating a prediction block using a surrounding pixel of a current block.

[0531] The inter prediction is a method of searching for a block most similar to a current block from a reference image in which encoding / decoding has been completed and generating a prediction block.

[0532] Further, the prediction section 5802 can determine the optimum prediction mode of the prediction block in the residual block obtained by subtracting the prediction block from the original block after generating the prediction block, using a variety of different techniques such as rate-distortion optimization (RDO).

[0533] [Equation 14]

[0534]

[0535] where D, R, J represent degradation due to quantization, code rate of the compressed stream, and rate-distortion (RD) cost, respectively, Φ represents the encoding mode, λ represents a Lagrangian multiplier, and can be used as a scaling correction coefficient for matching the unit of the error amount and the bit amount. In order to be able to select the optimum encoding mode in the encoding process, the J, i.e., the rate-distortion (RD) cost, when the corresponding mode is applied should be smaller than in the case where other modes are applied, and in the formula for calculating the rate-distortion (RD) cost, it can be calculated while considering the bit rate and the error at the same time.

[0536] Figure 59 is a schematic diagram for describing a prediction section of an image encoding apparatus.

[0537] In the case where the intra prediction 5901 is performed using the original information and the reconstructed information, the optimum intra prediction mode 5902 can be determined using the rate-distortion (RD) cost for each prediction mode and the prediction block can be generated.

[0538] In the case where the inter prediction 5903 is performed using the original information and the reconstructed information, the rate-distortion (RD) cost can be calculated for the SKIP mode, the MERGE mode, and the AMVP mode.

[0539] In the MERGE candidate search section 5904, a candidate motion information set for the SKIP mode and the MERGE mode can be constructed. Further, in the corresponding candidate motion information set, the optimum motion information 5905 can be determined using the rate-distortion (RD) cost.

[0540] In the AMVP candidate search section 5906, a candidate motion information set for the AMVP mode can be constructed. Further, the optimum motion information can be determined by performing the motion estimation 5907 using the corresponding candidate motion information set.

[0541] The prediction block can be generated by performing motion compensation 5908 using the best motion information decided in each mode.

[0542] The above-described inter-picture prediction can be constituted by three modes (SKIP mode, MERGE mode, and AMVP mode). Motion information (prediction direction information, reference picture information, and motion vector) can be encoded according to each prediction mode.

[0543] The SKIP mode can decide the best prediction information using motion information of a reconstructed area. The SKIP mode can constitute a motion information candidate group in the reconstructed area, and generate a prediction block by using a candidate having the minimum rate-distortion (RD) cost value in the corresponding candidate group as prediction information. Here, since the method of constituting the motion information candidate group is the same as the method of constituting the motion information candidate group of the MERGE mode described below, it will be omitted in the present description.

[0544] The MERGE mode is the same as the SKIP mode in terms of deciding the best prediction information using motion information of a reconstructed area. However, the difference between the two is that the SKIP mode explores motion information having a prediction error of 0 from a motion information candidate group, whereas the MERGE mode explores motion information having a prediction error not equal to 0 from a motion information candidate group. As in the SKIP mode, a motion information candidate group can be constituted in a reconstructed area, and a prediction block can be generated by using a candidate having the minimum rate-distortion (RD) cost value in the corresponding candidate group as prediction information.

[0545] The AMVP mode can decide the best motion information by performing motion estimation on reference pictures of different prediction directions, respectively. Here, the prediction direction can be either a single direction using only one of the past / future or a double direction using both the past and future directions. A prediction block can be generated by performing motion compensation using the best motion information decided by means of motion estimation. Here, a motion information candidate group for performing motion estimation can be derived for reference pictures of different prediction directions, respectively. The corresponding motion information candidate group can be used as a starting position for motion estimation.

[0546] For efficient encoding / decoding of the best intra prediction mode of a picture, a most probable mode (MPM) can be used. The most probable mode (MPM) refers to a method in which, when the best intra prediction mode of a current block is one of the intra prediction modes included in a most probable mode (MPM) candidate list configured using the reconstructed intra prediction modes of the periphery of the current block, only the most probable mode (MPM) candidate index information is transmitted and the intra prediction mode is encoded. The most probable mode (MPM) candidate list can be used to encode the best luma intra prediction mode. In contrast, when the chroma intra prediction mode is encoded, a chroma candidate list can be configured using the luma intra prediction modes. Among the modes included in the configured chroma candidate list, the prediction block of the best chroma intra prediction mode can be generated through rate-distortion (RD) cost calculation.

[0547] The transform unit 5803 can generate a transform block by transforming a residual block that is a difference between an original block and a prediction block. The transform block is the smallest unit used in the transform and quantization processes. The transform unit 5803 can generate a transform block having transform coefficients by transforming a residual signal into a frequency region. As a method of transforming a residual signal into a frequency region, various transform techniques such as a transform based on a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), etc. can be used, by which a residual signal can be transformed into a frequency region and transform coefficients can be generated. In order to conveniently use the transform techniques, a matrix operation using a basis vector is performed, and various transform techniques can be mixed and used when the matrix operation is performed, according to the prediction mode used when the prediction block is encoded. For example, when intra prediction is performed, a discrete cosine transform can be used in a horizontal direction and a discrete sine transform can be used in a vertical direction according to the prediction mode.

[0548] The quantization section 5804 can generate a quantized transform block by quantizing the transform block. That is, the quantization section 5804 can generate a quantized transform block (Quantized Transform Coefficient) having quantized transform coefficients by quantizing the transform coefficients of the transform block generated from the transform section 5803. As a quantization method, a method such as dead zone uniform threshold quantization (DZUTQ) or a quantization weighted matrix can be used, and various different quantization methods such as a modified quantization can also be used.

[0549] Furthermore, in the above description, the case where the transform section 5803 and the quantization section 5804 are included in the image encoding apparatus is illustrated and described, but the transform section and the quantization section can also be selectively included in the image encoding apparatus. That is, the image encoding apparatus can generate a transform block by transforming a residual block without performing a quantization process, can perform only a quantization process without transforming a residual block into frequency coefficients, or can not perform either a transform or a quantization process. Even in the case where only a part of the transform section or the quantization section is performed or all processes are not performed in the image encoding apparatus, the block input to the entropy encoding section is generally referred to as a "quantized transform block".

[0550] The entropy encoding section 5805 can encode the quantized transform block and output a bitstream. That is, the entropy encoding section 5805 can encode the coefficients of the quantized transform block output from the quantization section 5804 using various encoding techniques such as entropy encoding, and generate and output a bitstream including additional information (for example, information related to a prediction mode, quantized coefficients, and the like) required to decode the corresponding block in the image decoding apparatus described later.

[0551] That is, the entropy encoding section 2905 can encode the coefficients of the quantized transform block output from the quantization section 2904 using various encoding techniques such as entropy encoding, and generate and output a bitstream including additional information (for example, information related to a prediction mode (the information related to a prediction mode can include motion information or in-picture prediction mode information decided in the prediction section, and the like), quantized coefficients, and the like) required to decode the corresponding block in the image decoding apparatus described later.

[0552] The inverse quantization section 5806 can reconstruct an inverse quantized transform block by inversely performing a quantization technique used in the quantization process on the quantized transform block.

[0553] The inverse transform unit 5807 reconstructs the residual block by performing inverse transform on the inverse quantization transform block using the same method as that used in the transform process. The inverse transform can be performed by inversely performing the transform technique used in the transform unit.

[0554] In addition, the inverse quantization unit and the inverse transform unit described above can perform inverse quantization and inverse transform by inversely using the quantization method and the transform method used in the quantization unit and the transform unit. Alternatively, in the case where only quantization is performed without transform in the transform unit and the quantization unit, only inverse quantization can be performed without inverse transform. Also, in the case where neither transform nor quantization is performed, the inverse quantization unit and the inverse transform unit can not perform inverse transform and inverse quantization, or can be omitted directly from the image encoding apparatus.

[0555] The addition unit 5808 can reconstruct the current block by performing addition on the residual signal generated in the inverse transform unit and the prediction block generated by prediction.

[0556] The filter unit 5809 is a process of performing filtering on the entire image after reconstructing all blocks in the current image, and can include deblocking filtering and sample adaptive offset (SAO), etc. Deblocking filtering refers to an operation for reducing blocking distortion generated in the process of encoding an image in a block unit, and sample adaptive offset (SAO) refers to an operation of minimizing the difference between a reconstructed image and an original image by performing addition or subtraction on a reconstructed pixel using a specific value.

[0557] The memory 5810 stores the reconstructed current block on which filtering is additionally performed in the loop filter unit after addition of the residual signal generated in the inverse transform unit and the prediction block generated by prediction, and can be used for prediction of a next block or a next image, etc.

[0558] The subtraction unit 5811 can generate a residual block by performing subtraction on the current original block and the prediction block.

[0559] Figure 60 is a flowchart illustrating an encoding method of prediction information.

[0560] Referring to Figure 60In step S6001, the action information of the SKIP mode is encoded. In step S6002, it is determined whether the action of the SKIP mode is performed. When it is determined in step S6002 that the action of the SKIP mode is performed, in step S6007, the MERGE candidate index information for the SKIP mode is encoded, and then the present flowchart is ended.

[0561] When it is determined in step S6002 that the action of the SKIP mode is not performed, in step S6003, the prediction mode is encoded. In step S6003, it is determined whether the prediction mode is an inter prediction mode or an intra prediction mode. When it is determined in S6004 that the prediction mode is the inter prediction mode, in step S6005, the action information of the MERGE mode is encoded.

[0562] In step S6006, it is determined whether the action of the MERGE mode is performed. When it is determined in step S6006 that the action of the MERGE mode is performed, the process goes to step S6007, and the MERGE candidate index information for the MERGE mode is encoded, and then the present flowchart is ended. When it is determined in step S6006 that the action of the MERGE mode is not performed, in step S6008, the prediction direction is encoded. The prediction direction can be one of a past direction, a future direction, and a bi-direction.

[0563] In step S6009, it is determined whether the prediction direction is the future direction. When it is determined in step S6009 that the prediction direction is not the future direction, in step S6010, the reference picture index information of the past direction is encoded.

[0564] In step S6011, the MVD (Motion Vector Difference) information of the past direction is encoded. In step S6012, the MVP (Motion Vector Predictor) information of the past direction is encoded.

[0565] When it is determined in step S6009 that the prediction direction is the future direction or the bi-direction, after step S6012 is completed, in step S6013, it is determined whether the prediction direction is the past direction. When it is determined in step S6013 that the prediction direction is not the past direction, in step S6014, the reference picture index information of the future direction is encoded.

[0566] In step S6015, motion vector difference (MVD) information for a future direction is encoded. In step S6016, motion vector prediction (MVP) information for a future direction is encoded, and then the flowchart ends.

[0567] When it is determined in S6004 that the prediction mode is the intra prediction mode, in step S6017, motion information for a most probable mode (MPM) is encoded. The most probable mode (MPM) refers to a method of transmitting index information of a most probable mode (MPM) candidate intra prediction mode when the most probable mode (MPM) candidate intra prediction mode includes optimal intra prediction mode information of the current block decided in the image encoding apparatus, after the most probable mode (MPM) candidate intra prediction mode is constituted as the most probable mode (MPM) candidate intra prediction mode using the reconstructed image of the periphery of the current block.

[0568] In step S6018, it is determined whether or not the most probable mode (MPM) operates. When it is determined in step S6018 that the most probable mode (MPM) operates, in step S6019, index information of the most probable mode (MPM) candidate intra prediction mode is encoded. When it is determined in step S6018 that the most probable mode (MPM) does not operate, in step S6020, optimal intra prediction mode information of the luminance (Luma) in the remaining intra prediction mode other than the most probable mode (MPM) candidate intra prediction mode is encoded.

[0569] After step S6019 and step S6020 end, in step S6021, optimal intra prediction mode information of the chroma (Chroma) is encoded, and then the flowchart ends. The section for performing steps S6008 to S6016 is referred to as a motion information encoding section 601. Further, the section for performing steps S6017 to S6020 is referred to as a luminance intra prediction mode encoding section 602.

[0570] Figure 61 is a flowchart illustrating an image decoding apparatus.

[0571] Referring to Figure 61 , the image decoding apparatus is an apparatus for decoding an image, and can include an entropy decoding section 6101, an inverse quantization section 6102, an inverse transform section 6103, a prediction section 6104, an addition operation section 6105, a loop filter section 6106, and a memory section 6107.

[0572] Figure 61The respective constituent elements illustrated in the drawings are individually illustrated in order to represent different features and functions in the image decoding device, and do not represent that the respective constituent elements are constituted by mutually separate hardware or one software unit. That is, although the respective constituent elements are described in order to facilitate the explanation, at least two of the respective constituent elements can be combined into one constituent element, or one constituent element can be divided into a plurality of constituent elements to perform the corresponding functions, and the embodiment in which the respective constituent elements are integrated and the embodiment in which the respective constituent elements are separated as described above are included in the scope of the claims of the present application without departing from the essence of the present application.

[0573] Furthermore, a part of the constituent elements can not be a constituent element necessary for performing the essential functions in the present application, but can be an optional constituent element for improving performance. The present application can include only the constituent elements necessary for implementing the essence of the present application except for the constituent elements for improving performance, and a structure including the essential constituent elements except for the optional constituent elements for improving performance is also included in the scope of the claims of the present application.

[0574] The encoding block in the image encoding device can be referred to as a decoding block or can continue to be referred to as an encoding block in the image decoding device.

[0575] The entropy decoding unit 6101 can read various information necessary when decoding the corresponding block and the quantized transform coefficient by analyzing the bitstream transmitted from the image encoding device.

[0576] The inverse quantization unit 6102 can reconstruct an inverse quantization block having an inverse quantized coefficient by inversely performing the quantization technique used in the quantization process on the quantized coefficient decoded in the entropy decoding unit.

[0577] The inverse transform unit 6103 reconstructs a residual block having a difference signal by performing inverse transform on the inverse quantization transform block using the same method as that used in the transform process, and can perform the inverse transform by inversely performing the transform technique used in the transform unit.

[0578] The prediction unit 6104 generates a prediction block using the prediction mode information decoded in the entropy decoding unit, and at this time, can use the same method as that performed in the prediction unit of the image encoding device.

[0579] The addition unit 6105 can reconstruct the current block by performing addition on the residual signal reconstructed in the inverse transform unit and the prediction block generated by the prediction.

[0580] The filter unit 6106 is a process of performing filtering on the entire region of the image after reconstructing all blocks within the current image, and can include deblocking filtering, sample adaptive offset (SAO), and the like. Details thereof are the same as those described above with respect to the loop filter unit of the image encoding apparatus.

[0581] The memory 6107 stores a reconstructed current block in which filtering is additionally performed on a residual signal generated in the inverse transform unit and a prediction block generated by prediction in the loop filter unit after performing addition operation thereon, and can be used to predict a next block or a next image, and the like.

[0582] Figure 62 is a schematic diagram for describing a prediction unit of an image decoding apparatus.

[0583] Referring to Figure 62 When the prediction mode is intra prediction, optimal intra prediction mode information 6201 can be decided, and a prediction block can be generated by performing intra prediction 6202.

[0584] When the prediction mode is inter prediction, optimal prediction mode 6203 can be decided from SKIP, MERGE, and AMVP modes. Next, in the case of decoding by the SKIP mode or the MERGE mode, candidate motion information sets for the SKIP mode and the MERGE mode can be constituted in a MERGE candidate search unit 6204. In the corresponding candidate motion information sets, optimal motion information 6205 can be decided.

[0585] In the case of decoding by the AMVP, candidate motion information sets for the AMVP mode can be constituted in an AMVP candidate search unit 6206. In the corresponding candidate motion information sets, optimal motion information 6207 can be decided using transmitted motion vector prediction (MVP) information. Next, a prediction block can be generated by performing motion compensation 6208 using the optimal motion information decided in each mode.

[0586] Figure 63 is a flowchart illustrating a method of decoding prediction information.

[0587] Referring to Figure 63In step S6301, the action information of the SKIP mode is decoded. In step S6302, it is determined whether the action of the SKIP mode is performed. When it is determined in step S6302 that the action of the SKIP mode is performed, in step S6307, the MERGE candidate index information for the SKIP mode is decoded, and then the flowchart ends.

[0588] When it is determined in step S6302 that the action of the SKIP mode is not performed, in step S6303, the prediction mode is decoded. In step S6303, it is determined whether the prediction mode is an inter prediction mode or an intra prediction mode. When it is determined in S6304 that the prediction mode is the inter prediction mode, in step S6305, the action information of the MERGE mode is decoded.

[0589] In step S6306, it is determined whether the action of the MERGE mode is performed. When it is determined in step S6306 that the action of the MERGE mode is performed, it can be jumped to step S6307 and the MERGE candidate index information for the MERGE mode is decoded, and then the flowchart ends. When it is determined in step S6306 that the action of the MERGE mode is not performed, in step S6308, the prediction direction is decoded. The prediction direction can be one of a past direction, a future direction, and a bi-direction. In step S6309, it is determined whether the prediction direction is the future direction. When it is determined in step S6309 that the prediction direction is not the future direction, in step S6310, the reference picture index information of the past direction is decoded.

[0590] In step S6311, the MVD (Motion Vector Difference) information of the past direction is decoded. In step S6312, the MVP (Motion Vector Predictor) information of the past direction is decoded.

[0591] When it is determined in step S6309 that the prediction direction is the future direction or the bi-direction, after step S6312 is completed, in step S6313, it is determined whether the prediction direction is the past direction. When it is determined in step S6313 that the prediction direction is not the past direction, in step S6314, the reference picture index information of the future direction is decoded.

[0592] In step S6315, motion vector difference (MVD) information for a future direction is decoded. In step S6316, motion vector prediction (MVP) information for the future direction is decoded, and the flowchart ends.

[0593] When it is determined in S6304 that the prediction mode is the intra prediction mode, in step S6317, motion information for a most probable mode (MPM) is decoded. In step S6318, it is determined whether or not the most probable mode (MPM) is in operation. When it is determined in step S6318 that the most probable mode (MPM) is in operation, in step S6319, index information for a most probable mode (MPM) candidate intra prediction mode is decoded. When it is determined in step S6318 that the most probable mode (MPM) is not in operation, in step S6320, information for a best intra prediction mode for luma (Luma) among the remaining intra prediction modes other than the most probable mode (MPM) candidate intra prediction mode is decoded.

[0594] After step S6319 and step S6320 are completed, in step S6321, information for a best intra prediction mode for chroma (Chroma) is decoded, and the flowchart ends. The section for performing steps S6308 to S6316 is referred to as a motion information encoding section 631. Further, the section for performing steps S6317 to S6320 is referred to as a luma (Luma) intra prediction mode encoding section 632.

[0595] Next, various embodiments to which the present application is applied will be described in more detail with reference to the accompanying drawings.

[0596] Figure 64 is a schematic diagram for explaining a prediction section of an image encoding apparatus to which an embodiment of the present application is applied.

[0597] In a case where intra prediction is performed using original information and reconstructed information 6401, it can be determined whether or not the intra prediction mode is to be decided using a decoder side intra prediction mode derivation (DIMD) mode by a rate-distortion (RD) cost. In a case where the decoder side intra prediction mode derivation (DIMD) mode is not used, a luma (Luma) intra prediction mode is decided 6402. At this time, the intra prediction mode having the minimum rate-distortion (RD) cost can be decided from among the intra prediction modes.

[0598] In contrast, in the case of using the decoder-side intra mode derivation (DIMD) mode, the intra prediction mode information can not be transmitted, and the intra prediction mode of the luma is determined using the same prediction method as the image decoding device. Regarding the decoder-side intra mode derivation (DIMD) prediction method, please refer to Figure 72 Detailed description is made.

[0599] Next, the luma intra prediction mode can be used to determine the candidate mode for the chroma intra prediction mode, and the intra prediction mode with the minimum rate-distortion (RD) cost in the corresponding candidate mode is determined as the chroma intra prediction mode 6404. The image encoding device prediction unit 6400 can generate the intra prediction block of the current block using the determined luma, chroma intra prediction mode.

[0600] In the case of performing inter prediction 6405 using the original information and the reconstructed information, the rate-distortion (RD) cost can be calculated for the skip (SKIP) mode, the merge (MERGE) mode, and the advanced motion vector prediction (AMVP) mode.

[0601] In the merge (MERGE) candidate exploration unit 6406, the candidate motion information set 6406 for the skip (SKIP) mode and the merge (MERGE) mode can be constructed. Among the constructed candidate motion information set, the best motion information 6407 can be determined using the rate-distortion (RD) cost.

[0602] In the advanced motion vector prediction (AMVP) candidate exploration unit 6408, the candidate motion information set 6408 for the advanced motion vector prediction (AMVP) mode can be constructed. Among the constructed candidate motion information set, the best motion information can be determined by performing motion estimation 6409.

[0603] After the best motion information of the skip (SKIP), merge (MERGE), and advanced motion vector prediction (AMVP) modes is determined, whether to apply the regional luma compensation can be determined 6410. In the skip (SKIP) and merge (MERGE) modes, whether to apply the regional luma compensation to the prediction block reconstructed by the best candidate motion information can be determined directly, or can be determined by comparing the rate-distortion (RD) cost of applying the regional luma compensation and the rate-distortion (RD) cost of not using it. In the advanced motion vector prediction (AMVP) mode, whether to apply the regional luma compensation can be determined by comparing the rate-distortion (RD) cost of applying the regional luma compensation and the rate-distortion (RD) cost of not applying it.

[0604] After deciding the best motion information (including motion vector precision), whether or not to apply the regional luma compensation, the decided information can be used to perform the motion compensation 6411. The prediction unit 6400 of the video encoding device can generate the inter-predicted block by the motion compensation 6411.

[0605] Figure 65 is a schematic diagram for explaining the prediction unit of the video decoding device to which an embodiment of the present application is applied.

[0606] In the case of performing the intra-prediction 6501 by using the reconstructed information, it can be decided whether or not to decide the intra-prediction mode by using the decoder side intra prediction mode derivation (DIMD) mode.

[0607] At this time, in the case of not using the decoder side intra prediction mode derivation (DIMD) mode, the luma (Luma) intra-prediction mode can be decided by the reconstructed information 6502. In the case of using the decoder side intra prediction mode derivation (DIMD) mode, the luma (Luma) intra-prediction mode can be decided by the reconstructed information by using the same prediction method as the video encoding device.

[0608] Next, the chroma (Chroma) intra-prediction mode can be decided by the reconstructed information 6504. The prediction unit 6500 of the video decoding device can generate the intra-predicted block of the current block by using the decided luma (Luma), chroma (Chroma) intra-prediction modes.

[0609] In the case of performing the inter-prediction 6505 by using the reconstructed information, it is necessary to decide whether the current prediction mode in the inter-prediction mode is the SKIP mode, the MERGE mode, or the AMVP mode.

[0610] In the case of the current prediction mode being one of the SKIP mode and the MERGE mode, the candidate motion information set for the SKIP mode and the MERGE mode can be constructed in the MERGE candidate exploration unit 6506. By using the reconstructed information, the best motion information 6507 can be decided from the corresponding candidate motion information set.

[0611] In the AMVP candidate exploration unit 6508, the candidate motion information set for the AMVP mode can be constructed. By using the reconstructed information, the best candidate can be selected from the corresponding candidate motion information set, and the best motion information 6509 can be decided by using the reconstructed motion information.

[0612] After deciding the best motion information of SKIP, MERGE, and AMVP mode, it is possible to decide whether to apply the local illumination compensation by using the reconstructed information 6510. After deciding the best motion information including the motion vector precision and whether to apply the local illumination compensation, it is possible to perform the motion compensation using the decided information 6511. The prediction unit 6500 of the video decoding device can generate the inter picture predicted block by the motion compensation.

[0613] Figure 66 is a flowchart illustrating an encoding method of the coded information to which an embodiment of the present application is applied.

[0614] Next, the Figure 66 The method of encoding each coded information in the entropy encoding unit of the video encoding device to which an embodiment of the present application is applied will be described in detail. The description of the steps S6601 to S6607 in Figure 66 may be the same as the description of the steps S6001 to S6007 in Figure 60 .

[0615] When it is decided in step S6606 that the MERGE mode does not operate or after completing step S6607, it is possible to encode the local illumination compensation operation information in step S6608. However, in the case where the prediction mode is SKIP, MERGE, and whether to directly apply the local illumination compensation of the predicted block reconstructed by the best candidate motion information, it is possible to omit this step.

[0616] After completing step S6608, it is possible to encode the motion information by the motion information encoding unit 601 Figure 60 (S6008 to S6016).

[0617] When the prediction mode in step S6604 is the intra prediction mode but not the inter prediction mode, it is possible to encode the decoder side intra mode derivation (DIMD) mode operation information in step S6609. In step S6610, it is possible to decide whether the decoder side intra mode derivation (DIMD) mode operates. When the decoder side intra mode derivation (DIMD) mode operates, it is possible to not encode the additional intra prediction mode information. When the decoder side intra mode derivation (DIMD) mode does not operate, it is possible to encode the best intra prediction mode of the Luma by the Luma intra prediction mode encoding unit 602 Figure 60 (S6017 to S6020).

[0618] In step S6611, the best intra prediction mode of chroma can be encoded. Next, in step S6612, the fixed application / non-application information of the transform type can be encoded.

[0619] In step S6613, it can be determined whether the transform type is fixed. The present flowchart can be ended when the fixed transform type is used, and the present flowchart can be ended after the transform type information of different transform directions is encoded when the transform type is adaptively decided.

[0620] Figure 67 is a flowchart illustrating a decoding method of coded information to which an embodiment of the present application is applied.

[0621] Next, the decoding method of coded information to which an embodiment of the present application is applied will be described with reference to Figure 67 The method of decoding each coded information in the entropy decoding section of the image decoding apparatus to which an embodiment of the present application is applied will be described in detail. The description of steps S6701 to S6707 can be the same as the description of steps S6301 to S6307 in Figure 63 .

[0622] When it is determined in step S6706 that the MERGE mode does not operate or after step S6707 is completed, the region illumination compensation operation information can be decoded in step S6708. However, in the case where the prediction mode is SKIP, MERGE, and whether the region illumination compensation of the prediction block reconstructed by the best candidate motion information is directly applied, the present step can be omitted.

[0623] After step S6708 is completed, the motion information can be decoded by the motion information decoding section 631 (S6308 to S6316) in Figure 63 . When the prediction mode in step S6704 is the intra prediction mode, the decoder side intra mode derivation (DIMD) mode operation information can be decoded in step S6709.

[0624] In step S6710, it can be determined whether the decoder side intra mode derivation (DIMD) mode operates. When the decoder side intra mode derivation (DIMD) mode operates, the additional intra prediction mode information can not be decoded. When the decoder side intra mode derivation (DIMD) mode does not operate, the best luma intra prediction mode can be decoded by the luma intra prediction mode encoding section 632 (S6317 to S6320) in Figure 63 .

[0625] In step S6711, the optimal intra-prediction mode of the optimal chroma can be decoded. Next, in step S6712, the fixed application information of the transform type can be decoded.

[0626] In step S6713, it can be determined whether the transform type is fixed. The present flowchart can be ended when the fixed transform type is used, and the present flowchart can be ended after the transform type information of different transform directions is decoded when the transform type is adaptively determined.

[0627] Next, the method of encoding / decoding the motion vector difference (MVD) using the motion vector difference information in the inter-prediction mode will be described in detail.

[0628] Next, the motion vector difference can refer to a vector value used to derive the final motion vector. For example, the motion vector difference of the current block can be added to the initial motion vector of the current block, thereby being used to derive the final motion vector of the current block.

[0629] The motion vector difference information can be information used to derive the motion vector difference of the current block. The video decoding device can decode the motion vector difference information in order to derive the motion vector difference of the current block.

[0630] The inter-prediction mode to which the present embodiment is applied can be one of a MERGE mode, an AMVP mode, and a SKIP mode.

[0631] As an example, in the case where the inter-prediction mode of the current block is the AMVP mode, the motion vector difference can be added to the motion vector predictor (MVP), thereby deriving the final motion vector of the current block.

[0632] As another example, in the case where the inter-prediction mode of the current block is the MERGE mode, the motion vector difference can be added to the initial motion vector derived through the MERGE mode, thereby deriving the final motion vector of the current block. That is, the motion vector difference can be a vector value used to correct the initial motion vector derived through the MERGE mode. For example, the motion vector difference can refer to precision information of the initial motion vector. In the MERGE mode, the initial motion vector can be derived using a merge candidate list of the current block.

[0633] Figure 68is a flowchart illustrating a method of encoding / decoding motion vector difference information in an inter-picture prediction mode.

[0634] In Figure 68 In the method 6801 and the method 6802, the actions in encoding (step S6011, S6015) / decoding (step S6311, S6315) motion vector difference information in the motion information encoding section 601, the motion information decoding section 631, respectively, are illustrated in detail.

[0635] As another example, in the method 6801 and the method 6802, the actions in encoding (step S305) / decoding (step S605) MERGE mode motion information are illustrated in detail. Figure 68

[0636] As an example, the motion vector difference information can include information indicating whether a motion vector difference exists in the current block. The information related to the presence or absence of the motion vector difference can be expressed using a flag value.

[0637] As an example, the motion vector difference information can include at least one of X component absolute value information or Y component absolute value information of the motion vector difference.

[0638] In step S6803, the image encoding apparatus can encode the absolute value information of the X component of the motion vector difference, and in step S6804, the absolute value information of the Y component can be encoded. As another example, the image encoding apparatus can first encode the absolute value information of the Y component of the motion vector difference, and then encode the absolute value information of the X component of the motion vector difference.

[0639] On the contrary, in step S6807, the image decoding apparatus can decode the absolute value information of the X component of the motion vector difference, and in step S6808, the absolute value information of the Y component can be decoded. As another example, the image decoding apparatus can first decode the absolute value information of the Y component of the motion vector difference, and then decode the absolute value information of the X component of the motion vector difference.

[0640] As another example, the image encoding apparatus / decoding apparatus can simultaneously encode / decode the absolute value information of the X component and the Y component of the motion vector difference. In order to simultaneously encode / decode the X component and the Y component of the motion vector difference, the motion vector difference information can include motion vector absolute value group information composed of at least one of the X component absolute value or the Y component absolute value information.

[0641] ​Further, the motion vector difference value information can include sign group information constituted by one of the sign of the X component and the sign of the Y component of the motion vector difference value. For example, the sign group information can be information for indicating one of the sign groups constituted by one of the sign of the X component and the sign of the Y component of the motion vector difference value. At this time, the binary information can be respectively assigned to different sign groups, and the sign group information can be expressed using an index for indicating one of the above-described sign groups.

[0642] As an example, the sign group information can be an index for indicating one of the sign groups defined according to Table 2 below.

[0643]

Table 2

[0644]

[0645]

[0646] In step S6805, the encoding apparatus can calculate the template matching cost value for different sign groups of the motion vector difference value, respectively, and assign binary information or an index to different sign groups of the motion vector difference value, respectively.

[0647] Figure 69 is a table illustrating a method of deciding binary information using a template matching cost value when encoding motion vector difference value information according to an embodiment of the present application.

[0648] As an example, in the case where the absolute values of the X and Y components of the motion vector difference value are not 0, the binary information can be assigned to the motion vector difference value sign groups according to 6901. Figure 69 At this time, the template matching cost value can be calculated for different motion vector difference value sign groups, respectively.

[0649] As another example, in the case where the absolute value of one of the X and Y components of the motion vector difference value is 0, the binary information can be assigned to the motion vector difference value sign groups according to 6902 and 6903. 6902 can be the case where the absolute value of the X component of the motion vector difference value is 0, and 6903 can be the case where the absolute value of the Y component of the motion vector difference value is 0. In the case as described above, the template matching cost value can be calculated for only 2 kinds of sign groups.

[0650] The video encoding device can efficiently determine the binary information allocated to each of the symbol groups in the order from low to high of the calculated template matching cost in different cases 6901, 6902, 6903, thereby improving the coding efficiency. In the cases as shown in 6902, 6903, the binary number of the allocated binary information is one bit in each of the different motion vector difference value symbol groups, but the coding efficiency can be improved by using the binary information of the lower template matching cost as the context adaptive binary arithmetic coding (CABAC) probability information corresponding to each of the binary information in the entropy encoding section and the entropy decoding section.

[0651] Figure 69 The template matching cost and the allocated binary information are merely examples, and the cost can be changed, and the method of allocating the binary information can also be changed.

[0652] Figure 70 is a schematic diagram for explaining the template matching technique used when determining the binary information of the motion vector difference value information according to an embodiment of the present application.

[0653] Next, the template matching technique as described above will be explained in detail. Figure 70 The template region can be determined in the reconstructed region of the current block in the current picture. The template region can be determined in the (Lh x Lw, where Lh and Lw are integers greater than 0) region of the top (Aw x Ah, where Aw and Ah are integers greater than 0) of the current block as shown in Figure 70

[0654] The region most similar to the template of the current picture, which is searched from the motion search exploration region in the reference picture, can be used as the template region of the reference picture. At this time, the error amount between the template region of the current block periphery and the template region in the motion search exploration region can be determined as the template matching cost. The error amount can be derived by a calculation method such as the sum of absolute difference (SAD), the sum of square difference (SSD), etc. The following mathematical expression 15 and mathematical expression 16 are mathematical expressions describing the calculation method of the sum of absolute difference (SAD) and the sum of square difference (SSD), respectively.

[0655]

Mathematical Expression 15

[0656] Diff(i, j) = Template(i, j) - PredBlk(i, j) ​

[0657] [Equation 16]

[0658] Diff(i, j) = Template(i, j) - PredBlk(i, j)

[0659] where i, j represent the position of a pixel, Template(i, j) represents the pixel of a template block, and PredBlk(i, j) represents the pixel of a prediction block.

[0660] Next, the binary information or the index of the decided sign group of the motion vector difference value can be encoded / decoded in steps S6806 and S6810.

[0661] Next, the method of deciding the binary information using the template matching technique in the case of encoding / decoding the best Luma, Chroma intra prediction mode in the intra prediction mode will be explained in detail.

[0662] Figure 71 is a schematic diagram for explaining the method of assigning binary information to each candidate in the most probable mode (MPM) candidate list for the template matching cost value using the template matching technique applied to the decoder-side intra mode derivation (DIMD) mode applied to one embodiment of the present application and the method of assigning binary information to the Chroma candidate mode.

[0663] Figure 71 is a schematic diagram for explaining the method of deciding the binary information of the index information based on the template matching cost value in the case of the most probable mode (MPM) action in the Luma intra prediction mode encoding section in Figure 60 and the Luma intra prediction mode decoding section in Figure 63 . The template matching technique in the intra prediction mode and the template matching technique in the inter prediction mode can be different from each other.

[0664] Figure 72 is a schematic diagram for explaining the template matching technique of the decoder-side intra mode derivation (DIMD) mode applied to one embodiment of the present application.

[0665] Next, the method of deciding the binary information using the template matching technique in the case of encoding / decoding the best Luma, Chroma intra prediction mode in the intra prediction mode will be explained in detail. Figure 72 Figure 72 ​As shown, the template region can be determined from the reconstructed region around the current block.

[0666] The template region can be determined to be different sizes. Next, a reference region of the template region can be determined from the periphery of the template region. The template region can be intra-predicted using reconstructed pixels of the reference region, and a difference between the predicted value and the reconstructed value of the template region, i.e., a prediction error amount, can be determined as a template matching cost value. As a method of calculating the prediction error amount, various methods such as the sum of absolute differences (SAD) and the sum of squared differences (SSD) can be used.

[0667] In Figure 71 In 7101, in the case of encoding the best Luma intra-prediction mode, the template matching cost values of the intra-prediction modes included in the most probable mode (MPM) candidate list can be calculated first.

[0668] Next, the intra-prediction modes can be assigned smaller binary information in order of the template matching cost values from small to large.

[0669] For example, as shown in Table 7101, in the case where the vertical (Ver) mode, the mean (DC) mode, the planar (Planar) mode, the horizontal (Hor) mode, and the angular (Angular) 20th mode are included in the most probable mode (MPM) candidate list, and the template matching cost values of the respective intra-prediction modes are 100, 120, 80, 140, and 150, the assigned binary information can be determined as 0, 10, 110, 1110, and 1111 in order of the template matching cost values from small to large. The template matching cost values and the assigned binary information in Table 7101 are merely illustrative, and the cost values can be changed, and the method of assigning binary information can also be changed.

[0670] In Figure 71 In 7102, in the case of encoding the best Chroma intra-prediction mode, the template matching cost values of the intra-prediction modes in the Chroma candidate mode list can be calculated first.

[0671] Next, the Chroma intra-prediction modes can be assigned smaller binary information in order of the template matching cost values from small to large.

[0672] As the intra prediction mode list of the chroma candidate mode list, the intra prediction mode reconstructed in the luma block can be used. In the video of the 4:2:0 format, the chroma block is 1 / 4 size of the luma block. In addition, because the block partitioning of the luma block and the chroma block can be independently performed in the intra only picture, the block partitioning of the luma block and the chroma block can be different from each other as shown in 7102.

[0673] In the construction of the chroma candidate mode list, the intra prediction modes reconstructed from the five different blocks Al, A2, A3, A4, A5 of the luma block can be used to construct the chroma candidate mode list. In the constructed chroma candidate mode list, the template matching cost values of the intra prediction modes of the blocks Al, A2, A3, A4, A5 can be calculated, and the smaller binary information can be assigned in the order from small to large of the corresponding cost values as shown in Table 7102. The template matching cost values in Table 7102 and the assigned binary information are only illustrative, and the cost values can be changed, and the method of assigning the binary information can also be changed.

[0674] Next, the method of using the template region in the reconstructed region around the current block when performing the region-wise illumination compensation process in the inter prediction mode will be described in detail.

[0675] Figure 73 is a schematic diagram for explaining the method of deciding whether to use the template in the reconstructed region around the prediction block according to whether to use the template in the reconstructed region around the current block when performing the region-wise illumination compensation technique according to one embodiment of the present application.

[0676] In the case of performing the region-wise illumination compensation, the compensation process for minimizing the difference between the template region of the reconstructed region around the prediction block pointed by the optimal motion information and the template region of the reconstructed region around the current block can be performed. The compensation process can be a process of deriving the parameters a, β that can minimize the least square sum through linear regression analysis. The region-wise illumination compensation using a, β can be performed through Mathematical Formula 17.

[0677]

Mathematical Formula 17

[0678] FinalPred = a * CurPred + β

[0679] In the case of deriving a, β, the a, β can be derived in the order of Figure 73The illustrated manner defines the template region of the current block peripheral reconstruction region. The definition as described above is to design an efficient parallel processing structure of the image.

[0680] In deciding whether or not to use the template region of the current block peripheral reconstruction region, the region exceeding the maximum coding block including the current block can not be used as the template region. In addition, a window of an arbitrary size (K x L) can be set and the template region exceeding the corresponding window can not be used. The size of the above-mentioned window can be transmitted in the upper header (e.g., video parameter level, sequence parameter level, picture parameter level, slice level, etc.) or the same size can be used by pre-setting in the image encoding / decoding apparatus.

[0681] In addition, whether or not to use the template region can not be decided in a state of violating the division boundary such as the slice and the parallel block.

[0682] In the template region of the prediction block peripheral reconstruction region, only the available template region of the current block periphery can also be used.

[0683] Figure 73 is a schematic diagram for explaining a method of deciding whether or not to use the template region in the prediction block peripheral reconstruction region according to whether or not to use the template in the current block peripheral reconstruction region when performing the regional brightness compensation technique according to an embodiment of the present application.

[0684] Figure 73 7301 in is an example diagram illustrating a case where the template region of the upper end region of the prediction block cannot also be used because the template region of the upper end region of the current block peripheral reconstruction region cannot be used. Figure 73 7302 in is an example diagram illustrating a state where the template region of the left side region of the prediction block cannot also be used because the template region of the left side region of the current block peripheral reconstruction region cannot be used. Figure 73 7303 in is an example diagram illustrating a case where the template region of the upper end and the left side region of the prediction block cannot also be used because the template region of the upper end and the left side region of the current block peripheral reconstruction region cannot be used.

[0685] In the case illustrated in 7303, the regional brightness compensation process can not be performed. In the case as described above, in Figure 66 the region brightness compensation action information can not be encoded in step S6608 of 6600, and in Figure 67 the region brightness compensation action information can also not be decoded in step S6708 of 6700. At this time, the region brightness compensation process action information can be decided as false.

[0686] Next, a method of deciding binary information of a transform type using a template matching technique in a case of encoding / decoding the transform type for a block unit transform of an image will be described in detail.

[0687] Figure 74 is a transform section of an image encoding apparatus to which an embodiment of the present application is applied.

[0688] The transform section 7400 of the image encoding apparatus can derive transform coefficients by transforming residual information into a frequency domain. When performing the residual information transform, it can fix the transform type by the fixed decision 7410 or adaptively decide 7402 the transform type.

[0689] In a case of fixing the transform type, the transform coefficients can be obtained by performing the horizontal direction transform 7404 after performing the vertical direction transform 7405 on the residual signal using the fixed transform type. At this time, as the transform type, one of a plurality of different transform types such as a discrete cosine transform-2 (DCT-2) type can be fixedly used.

[0690] On the contrary, in a case of adaptively deciding the transform type, the transform type can be adaptively decided from a plurality of different transform types. For example, one of transform types such as a discrete cosine transform-2 (DCT-2), a discrete cosine transform-5 (DCT-5), a discrete cosine transform-8 (DCT-8), a discrete sine transform-1 (DST-1), and a discrete sine transform-7 (DST-7) type can be adaptively decided. At this time, the transform type 7403 can be decided in different transform directions. By performing the horizontal direction transform 7404 and the vertical direction transform 7405 using the decided transform type in different transform directions, the transform coefficients can be derived. At this time, the transform type in different directions can be transmitted in a block unit or can be previously set and used in a higher level header in consideration of a size and a shape of a transform block, etc.

[0691] Figure 75 is an inverse transform section of an image encoding / decoding apparatus to which an embodiment of the present application is applied.

[0692] In the inverse transform section 7500 of the image encoding / decoding apparatus, reconstructed residual coefficients can be derived by transforming inverse quantization coefficients into a pixel domain. When performing the inverse transform on the inverse quantization coefficients, it can fix the inverse transform type by the fixed decision 7410 or adaptively decide 7502 the inverse transform type.

[0693] If the transform is performed in the transform section 7400 of the image encoding apparatus in a fixed transform type manner, the inverse transform can also be performed in the inverse transform section 7500 of the image encoding / decoding apparatus in a fixed inverse transform type manner. As the inverse transform type, the reconstructed residual coefficients can be derived by performing the vertical direction inverse transform 7504 and the horizontal direction inverse transform 7505 of the transform type decided in the transform section.

[0694] In the case of adaptively deciding the transform type, as the transform type, the reconstructed residual coefficients can be derived by performing the vertical direction inverse transform 7504 and the horizontal direction inverse transform 7505 of the transform type in different transform directions decided in the transform section.

[0695] Figure 76 is a schematic diagram for explaining a method of using a template matching technique for binary information for deciding a transform type in a transform section / inverse transform section of an image encoding / decoding apparatus according to an embodiment of the present application.

[0696] Next, a description will be given of Figure 76 A method of deciding binary information when encoding / decoding transform type information in different transform directions in the case of adaptively deciding a transform type in different transform directions in the transform section and a method of deciding whether to use a template region from a reconstructed region around a current block in the process of deciding binary information using a template matching technique will be described in detail.

[0697] As illustrated in 7604, five transform type groups are defined and the horizontal / vertical transform type is decided for each transform type group. At this time, the template matching cost value can be calculated for each transform group using the template matching technique.

[0698] In the template matching technique in the transform section, only the template region of the reconstructed region around the current block can be required to calculate the template matching cost value. In the example of 7601 to 7603, the reconstructed pixel values of the left side and the upper end 1 row of the current block can be calculated using each transform type group. After the residual values corresponding to the reconstructed region are transformed / quantized and then added to the prediction values after inverse quantization / inverse transform, the reconstructed pixel values of the reconstructed region can be obtained. The template matching cost value can be calculated using the reconstructed region and the template region by the following mathematical expression 18.

[0699] [Mathematical Expression 18]

[0700]

[0701] In Math. 18, TempDist represents the template matching cost value. In Math. 18, i represents the coordinate of the left template region, and j represents the coordinate of the upper template region. Width and Height represent the horizontal and vertical lengths of the current block, respectively.

[0702] By Math. 5, the template matching cost value can be calculated for each of the different transform type groups, and the smaller binary information can be assigned in the order of the corresponding cost value from small to large. The transform type, the template matching cost value, and the assigned binary information of the horizontal / vertical transform in Table 7604 are merely illustrative, and the transform type can be changed to a variety of different combinations, the cost value can be changed, and the method of assigning the binary information can also be changed.

[0703] In calculating the template matching cost value, the template region of the surrounding reconstructed region of the current block can be limited as in the examples of 7601, 7602, and 7603. The region beyond the maximum coding block including the current block can not be used as the template region. In addition, a window of any size (K x L) can be set, and the template region beyond the corresponding window can not be used. The size of the window can be transmitted in the upper header, or the same size set in advance can be used in the image encoding / decoding apparatus.

[0704] In addition, it cannot be determined whether the size beyond the partition boundary determined in the upper header such as the slice and the parallel block is used.

[0705] Figure 76 7601 in Math. 7601 illustrates a case where the upper region of the surrounding reconstructed region of the current block cannot be used as the template region. Figure 76 7602 in Math. 7602 illustrates a case where the left region of the surrounding reconstructed region of the current block cannot be used as the template region. Figure 76 7603 in Math. 7603 illustrates a case where the upper and left regions of the surrounding reconstructed region of the current block cannot be used as the template region.

[0706] In the example of 7603, the transform type can be used fixedly. Therefore, in step S6612 of Math. 6600, Figure 66 the transform type fixed use or not information can not be encoded in step S6712 of Math. 6700, and the transform type can be used fixedly. Figure 67 the transform type fixed use or not information can not be decoded in step S6712 of Math. 6700, and the transform type can be used fixedly.

[0707] Figure 77 is a flowchart for explaining an image decoding method to which an embodiment of the present application is applied.

[0708] Referring to Figure 77In step S7701, the video decoding apparatus can derive an initial motion vector of the current block.

[0709] The initial motion vector can be derived using a merge candidate list of the current block. For example, the initial motion vector can be derived using candidate motion information derived by a merge mode. The merge candidate list can be derived using at least one of motion information of a spatial candidate block of the current block and motion information of a temporal candidate block of the current block.

[0710] Next, in step S7702, the video decoding apparatus can decode motion vector difference information of the initial motion vector derived in step S7701, and in step S7703, can derive a motion vector difference of the initial motion vector using the decoded motion vector difference information. Next, in step S7704, the video decoding apparatus can add the initial motion vector derived in step S7701 and the motion vector difference derived in step S7703, thereby deriving a final motion vector of the current block.

[0711] The motion vector difference information can include sign group information constituted by one of an X component sign and a Y component sign of the motion vector difference. In addition, the motion vector difference information can include at least one of X component absolute value information or Y component absolute value information of the motion vector difference.

[0712] At this time, the sign group information can indicate one of sign groups constituted by one of the X component sign and the Y component sign, and can be expressed in the form of an index.

[0713] Figure 78 is a flowchart for explaining a video encoding method to which an embodiment of the present application is applied.

[0714] Referring to Figure 78 In step S7801, the video decoding apparatus can decide a final motion vector of the current block.

[0715] At this time, the video encoding apparatus can derive the final motion vector of the current block by a method such as rate-distortion (RD) cost comparison.

[0716] Next, in step S7802, the video encoding apparatus can decide a motion vector difference of the final motion vector decided in step S7801. The video encoding apparatus can encode motion vector difference information of the motion vector difference decided in step S7801.

[0717] The motion vector difference value can be added to the initial motion vector of the current block in the video decoding apparatus to derive the final motion vector determined in step S7801.

[0718] The motion vector difference information can include sign group information constituted by one of an X component sign and a Y component sign of the motion vector difference. In addition, the motion vector difference information can include at least one of X component absolute value information or Y component absolute value information of the motion vector difference.

[0719] At this time, the sign group information can indicate one of sign groups constituted by one of the X component sign and the Y component sign, and can be expressed in the form of an index.

[0720] The bitstream (or encoded data) generated by the video encoding method of Figure 78 The bitstream (or encoded data) generated by the video encoding method of

[0721] The above-described embodiments can be performed in the same method or a corresponding method in the video encoding apparatus and the video decoding apparatus.

[0722] The video can be encoded / decoded using at least one of the above-described embodiments or a combination of at least one of the above-described embodiments.

[0723] The order in which the above-described embodiments are applied can be different in the video encoding apparatus and the video decoding apparatus, and the order in which the above-described embodiments are applied can be the same in the video encoding apparatus and the video decoding apparatus.

[0724] The above-described embodiments can be performed on the luminance and the color difference signals respectively, or can be performed on the luminance and the color difference signals in the same manner.

[0725] The exemplary methods in the disclosure are described in the form of a sequence of actions for the sake of clarity and explicitness of illustration, but this is not intended to limit the execution order of the steps, and each step can be executed simultaneously or in a different order if necessary. In order to implement the methods in the disclosure, other steps can be additionally included on the basis of the steps of the examples, or only the remaining steps except for a part of the steps can be included, or other steps can be additionally included after excluding a part of the steps.

[0726] The various embodiments in the disclosure are not the result of listing all possible combinations, but are intended to illustrate representative forms of the disclosure, and matters described in the various embodiments can be independently applied or applied in a combination of two or more.

[0727] Furthermore, various embodiments of the present disclosure can be implemented by hardware, firmware, software, or a combination of the above. When implemented by hardware, one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general processors, controllers, microcontrollers, and microprocessors can be used.

[0728] The scope of the present disclosure includes software or device executable instructions (e.g., operating systems, applications, firmware, programs, etc.) that cause actions in methods of various embodiments to be performed on devices or computers, and non-transitory computer-readable media (e.g., memory, storage, etc.) that stores software or instructions as described above, which are executable by devices or computers.

[0729] Industrial Applicability

[0730] The present application can be used for encoding / decoding an image.

Claims

1. An image decoding method comprising the steps of: generating a first prediction block by performing inter prediction on a current block; generating a second prediction block by performing intra prediction on the current block; and generating a final prediction block by weightedly summing the first prediction block and the second prediction block, wherein the second prediction block being predicted using a fixed intra prediction mode, a weighting value of the first prediction block and a weighting value of the second prediction block being determined based on a prediction mode of a surrounding block of the current block, the surrounding block being a rightmost top surrounding block among top blocks of the current block and a lowermost left surrounding block among left blocks of the current block, the fixed intra prediction mode being a planar mode.

2. The image decoding method according to claim 1, wherein the first prediction block is predicted in a merge mode.

3. An image encoding method comprising the steps of: generating a first prediction block by performing inter prediction on a current block; generating a second prediction block by performing intra prediction on the current block; and generating a final prediction block by weightedly summing the first prediction block and the second prediction block, wherein the second prediction block being predicted using a fixed intra prediction mode, a weighting value of the first prediction block and a weighting value of the second prediction block being determined based on a prediction mode of a surrounding block of the current block, the surrounding block being a rightmost top surrounding block among top blocks of the current block and a lowermost left surrounding block among left blocks of the current block, the fixed intra prediction mode being a planar mode.

4. The image encoding method according to claim 3, wherein the first prediction block is predicted in a merge mode.

5. A method of transmitting a bitstream, the method comprising the steps of: transmitting a bitstream generated by an image encoding method, wherein the image encoding method comprising the steps of: generating a first prediction block by performing inter prediction on a current block; generating a second prediction block by performing intra prediction on the current block; and generating a final prediction block by weightedly summing the first prediction block and the second prediction block, wherein the second prediction block is predicted using a fixed intra prediction mode, a weighting value of the first prediction block and a weighting value of the second prediction block are determined based on a prediction mode of a surrounding block of the current block, the surrounding block being a rightmost top surrounding block among top blocks of the current block and a lowermost left surrounding block among left blocks of the current block, the fixed intra prediction mode being a planar mode.

Images