Video encoding and decoding method and device using affine model-based prediction

Abstract

The embodiment discloses a video coding and decoding method and device using affine model-based prediction. In this embodiment, a video decoding device decodes all or part of affine model information, control point motion vector information, and a reference picture index from a bitstream. The video decoding device determines a form of an affine model based on the affine model information, and derives a control point motion vector based on the affine model and the control point motion vector information. The video decoding device generates a motion vector for each prediction unit of the current block using the control point motion vector, and generates a prediction value for each prediction unit using the motion vector and the reference picture indicated by the reference picture index, thereby generating a prediction block of the current block.

Description

Technical Field

[0001] The present invention relates to a video encoding and decoding method and device using prediction based on an affine model. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.

[0003] Since video data has a large amount of data compared to audio data or still image data, video data requires a large amount of hardware resources (including memory) to store or transmit the uncompressed video data.

[0004] Accordingly, an encoder is typically used to compress and store or transmit video data. A decoder receives the compressed video data, decompresses the received compressed video data, and plays the decompressed video data. Video compression technologies include H.264 / Advanced Video Codec (AVC), High Efficiency Video Coding (HEVC), and Versatile Video Codec (VVC), which improves codec efficiency by approximately 30% or more compared to HEVC.

[0005] However, as image size, resolution, and frame rate continue to increase, the amount of data to be encoded will also increase. Accordingly, new compression technologies are needed that provide higher encoding and decoding efficiency and improved image enhancement effects than existing compression technologies.

[0006] As a method for improving encoding / decoding efficiency, an affine model processes the changing object signal or background signal in the video after following the movement of the camera or object in space and time to derive the geometric relationship between the object signal or background signal and the camera or object, model this relationship, and apply the modeled relationship to the reference signal and the original signal. In theory, if the relationship is perfectly derived as a three-dimensional affine model representing the same object, perfect prediction can be achieved using this relationship. However, such perfect prediction is only achievable in theory. In video encoding and decoding, prediction errors can be compensated by utilizing a prediction signal based on the modeling prediction and using a differential signal. Prediction methods based on affine models have the effect of improving encoding and decoding efficiency due to their prediction accuracy, but they also suffer from increased computational complexity for calculating the affine model. Therefore, in order to improve video encoding and decoding efficiency and enhance video quality, a method is needed that reduces computational complexity when performing affine model calculations and efficiently transmits relevant coding information. Summary of the Invention

[0007] Technical issues

[0008] The present invention is directed to providing a video encoding and decoding method and apparatus that efficiently transmit information about an affine model when performing affine model-based prediction of a current block.

[0009] Technical Solution

[0010] At least one aspect of the present invention provides a method for decoding a current block by a video decoding device. The method includes decoding all or part of affine model information, control point motion vector information, and a reference picture index from a bitstream. The affine model information indicates the form of the affine model, and the control point motion vector information includes a prediction method for the control point motion vector and a control point motion vector difference. The method also includes determining the form of the affine model based on the affine model information. The method also includes deriving the control point motion vector based on the affine model and the control point motion vector information. The method also includes generating a motion vector for each prediction unit of the current block by using the control point motion vector. The method also includes generating a prediction value for each prediction unit of the current block by using the motion vector and a reference picture indicated by the reference picture index, thereby generating a prediction block for the current block. In decoding the control point motion vector information, the method includes deriving the control point motion vector at the overlapping vertex in the current block by using the control point motion vector at the overlapping vertex in the reconstructed block at the overlapping vertex between the reconstructed block and the current block.

[0011] Another aspect of the present invention provides a method for encoding a current block using a video encoding device. The method includes determining affine model information and a prediction method for control point motion vectors. The affine model information indicates a form of the affine model. The method also includes determining the form of the affine model based on the affine model information. The method includes deriving control point motion vectors based on the affine model. The method also includes generating a motion vector for each prediction unit of the current block by using the control point motion vectors. The method also includes generating a prediction value for each prediction unit by using the motion vector and a reference picture, thereby generating a first prediction block of the current block. The method also includes deriving control point motion vector prediction values ​​based on the affine model and the prediction method for the control point motion vectors, and generating control point motion vector differences by subtracting each control point motion vector prediction value from each control point motion vector. The method also includes encoding the affine model information, an index indicating a reference picture, the prediction method for the control point motion vectors, and the control point motion vector differences. When generating control point motion vector differences, the method includes deriving control point motion vectors at overlapping vertices in the current block by utilizing control point motion vectors at overlapping vertices in the encoded block at overlapping vertices between the encoded block and the current block.

[0012] Yet another aspect of the present invention provides a computer-readable recording medium storing a bitstream generated by a video encoding method. The video encoding method includes determining affine model information and a prediction method for control point motion vectors. The affine model information indicates a form of the affine model. The video encoding method further includes determining the form of the affine model based on the affine model information. The video encoding method further includes deriving control point motion vectors based on the affine model. The video encoding method further includes generating a motion vector for each prediction unit of a current block by using the control point motion vectors. The video encoding method further includes generating a prediction value for each prediction unit by using the motion vectors and a reference picture, thereby generating a first prediction block for the current block. The video encoding method further includes deriving control point motion vector prediction values ​​based on the affine model and the prediction method for the control point motion vectors, and generating control point motion vector differences by subtracting each control point motion vector prediction value from each control point motion vector. The video encoding method further includes encoding the affine model information, an index indicating a reference picture, the prediction method for the control point motion vectors, and the control point motion vector differences. When generating the control point motion vector difference, the video encoding method includes deriving the control point motion vector at the overlapping vertex in the current block by using the control point motion vector at the overlapping vertex in the encoded block at the overlapping vertex between the encoded block and the current block.

[0013] Beneficial effects

[0014] As described above, the present invention provides a video coding method and apparatus that efficiently transmits information about the affine model when performing affine model-based prediction of the current block. Therefore, the video coding method and apparatus improve video coding efficiency and enhance video quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a block diagram of a video encoding device that can implement the technology of the present invention.

[0016] Figure 2 A method of partitioning a block using a quadtree plus binary tree ternary tree (QTBTTT) structure is shown.

[0017] Figure 3a and Figure 3b A plurality of intra prediction modes including a wide-angle intra prediction mode are shown.

[0018] Figure 4 Shows the neighboring blocks of the current block.

[0019] Figure 5 is a block diagram of a video decoding device that can implement the technology of the present invention.

[0020] Figure 6is a schematic diagram illustrating pixel-by-pixel computation of an affine model according to at least one embodiment of the present invention.

[0021] Figure 7 is a schematic diagram illustrating affine model calculation using a center vector according to at least one embodiment of the present invention.

[0022] Figure 8 is a schematic diagram illustrating sub-block-by-sub-block computation of an affine model according to at least one embodiment of the present invention.

[0023] Figure 9 is a schematic diagram illustrating a portion of a video decoding apparatus for affine model-based prediction according to at least one embodiment of the present invention.

[0024] Figures 10a to 10d is a schematic diagram illustrating motion vectors of sub-block units according to at least one embodiment of the present invention.

[0025] Figures 11a to 11c is a schematic diagram illustrating the derivation of overlapping control point motion vectors according to at least one embodiment of the present invention.

[0026] Figure 12a and Figure 12b is a flowchart of a method of encoding a current block by a video encoding apparatus according to at least one embodiment of the present invention.

[0027] Figure 13 is a flowchart of a method for reconstructing a current block by a video decoding device according to at least one embodiment of the present invention. DETAILED DESCRIPTION

[0028] Hereinafter, some embodiments of the present invention will be described in detail with reference to the accompanying illustrative drawings. In the following description, the same reference numerals represent the same elements, even though the elements are shown in different drawings. In addition, in the following description of some embodiments, when it is believed that the detailed description of related known components and functions obscures the subject matter of the present invention, the detailed description of the related known components and functions may be omitted for the sake of clarity and brevity.

[0029] Figure 1 FIG. 1 is a block diagram of a video encoding device that can implement the technology of the present invention. Figure 1 , a video encoding device and components of the device are described.

[0030] The encoding device may include: an image splitter 110, a predictor 120, a subtractor 130, a transformer 140, a quantizer 145, a rearrangement unit 150, an entropy encoder 155, an inverse quantizer 160, an inverse transformer 165, an adder 170, a loop filtering unit 180 and a memory 190.

[0031] Each component of the encoding device can be implemented as hardware or software, or as a combination of hardware and software. In addition, the function of each component can be implemented as software, and the microprocessor can also be implemented to execute the function of the software corresponding to each component.

[0032] A video consists of one or more sequences including multiple images. Each image is divided into multiple regions, and encoding is performed on each region. For example, an image is divided into one or more tiles or / and slices. Here, one or more tiles can be defined as a tile group. Each tile or / and slice is divided into one or more coding tree units (CTUs). In addition, each CTU is divided into one or more coding units (CUs) through a tree structure. The information applied to each coding unit (CU) is encoded as the syntax of the CU, and the information commonly applied to the CUs included in a CTU is encoded as the syntax of the CTU. In addition, the information commonly applied to all blocks in a slice is encoded as the syntax of the slice header, and the information applied to all blocks constituting one or more images is encoded as a picture parameter set (PPS) or a picture header. In addition, information commonly referenced by multiple images is encoded as a sequence parameter set (SPS). In addition, information commonly referenced by one or more SPSs is encoded as a video parameter set (VPS). In addition, information commonly applied to one tile or tile group may also be encoded as syntax of the tile or tile group header.The syntax included in the SPS, PPS, slice header, tile or tile group header may be referred to as a high-level syntax.

[0033] The image divider 110 determines the size of a coding tree unit (CTU). Information on the size of the CTU (CTU size) is encoded as a syntax of an SPS or PPS and transmitted to a video decoding device.

[0034] The image splitter 110 splits each image constituting a video into a plurality of coding tree units (CTUs) of a predetermined size, and then recursively splits the CTUs using a tree structure. Leaf nodes in the tree structure become coding units (CUs), which are basic units of coding.

[0035] The tree structure can be a quadtree (QT), in which a higher node (or parent node) is split into four lower nodes (or child nodes) of the same size. The tree structure can also be a binary tree (BT), in which a higher node is split into two lower nodes. The tree structure can also be a ternary tree (TT), in which a higher node is split into three lower nodes at a ratio of 1:2:1. The tree structure can also be a structure in which two or more structures of the QT structure, the BT structure, and the TT structure are mixed. For example, a quadtree plus binary tree (QTBT) structure can be used, or a quadtree plus binary tree ternary tree (QTBTTT) structure can be used. Here, a binary tree ternary tree (BTTT) is added to the tree structure to be called a multiple-type tree (MTT).

[0036] Figure 2 It is a schematic diagram for describing a method of dividing a block by using a QTBTTT structure.

[0037] like Figure 2 As shown, the CTU can first be split into a QT structure. The quadtree splitting can be recursive until the size of the split block reaches the minimum block size (MinQTSize) of the leaf node allowed in QT. The first flag (QT_split_flag) indicating whether each node of the QT structure is split into four nodes of the lower layer is encoded by the entropy encoder 155 and notified to the video decoding device with a signal. When the leaf node of QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in BT, the leaf node can be further split into at least one of a BT structure or a TT structure. There can be multiple splitting directions in the BT structure and / or the TT structure. For example, there can be two directions, namely, the direction of splitting the blocks of the corresponding node horizontally and the direction of splitting the blocks of the corresponding node vertically. As shown in FIG. Figure 2 As shown, when MTT splitting starts, the entropy encoder 155 encodes a second flag (mtt_split_flag) indicating whether the node is split, and a flag indicating the splitting direction (vertical or horizontal) and / or a flag indicating the splitting type (binary or trifurcated) when the node is split, and notifies the video decoding device of the same with a signal.

[0038] Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node is split into four nodes in the lower layer, a CU split flag (split_cu_flag) indicating whether the node is split may also be encoded. When the value of the CU split flag (split_cu_flag) indicates that each node is not split, the block of the corresponding node becomes a leaf node in the partition tree structure and becomes a CU, which is the basic unit of encoding. When the value of the CU split flag (split_cu_flag) indicates that each node is split, the video encoding device first starts encoding the first flag in the above-mentioned scheme.

[0039] When QTBT is used as another example of a tree structure, there may be two types, namely, a type in which the block of the corresponding node is horizontally split into two blocks of the same size (i.e., symmetrical horizontal splitting) and a type in which the block of the corresponding node is vertically split into two blocks of the same size (i.e., symmetrical vertical splitting). The entropy encoder 155 encodes a split flag (split_flag) indicating whether each node of the BT structure is split into blocks of the lower layer and split type information indicating the split type, and transmits them to the video decoding device. On the other hand, there may also be a type in which the block of the corresponding node is split into two blocks that are asymmetric to each other. The asymmetric form may include a form in which the block of the corresponding node is split into two rectangular blocks with a size ratio of 1:3, or may also include a form in which the block of the corresponding node is split in a diagonal direction.

[0040] The CU can have various sizes depending on the QTBT or QTBTTT partitioned from the CTU. Hereinafter, the block corresponding to the CU to be encoded or decoded (i.e., the leaf node of the QTBTTT) is referred to as the "current block." When QTBTTT partitioning is adopted, the shape of the current block can also be a rectangular shape in addition to a square shape.

[0041] The predictor 120 predicts the current block to generate a predicted block. The predictor 120 includes an intra predictor 122 and an inter predictor 124.

[0042] Typically, each current block in an image can be predictively encoded. Typically, prediction of the current block can be performed using intra-frame prediction techniques (which utilize data from the image that includes the current block) or inter-frame prediction techniques (which utilize data from an image that was encoded before the image that includes the current block). Inter-frame prediction includes both unidirectional prediction and bidirectional prediction.

[0043] The intra-frame predictor 122 predicts pixels in the current block by using pixels (reference pixels) located adjacent to the current block in the current image including the current block. There are multiple intra-frame prediction modes according to the prediction direction. For example, Figure 3aAs shown, the plurality of intra prediction modes may include two non-directional modes including a planar mode and a DC mode, and may include 65 directional modes. Neighboring pixels to be used and algorithm equations are defined differently according to each prediction mode.

[0044] In order to perform efficient directional prediction for a current block with a rectangular shape, we can additionally use Figure 3b The directional modes are shown in the figure with dotted arrows (#67 to #80, intra-frame prediction modes #-1 to #-14). The directional modes can be called "wide angle intra-frame prediction modes". Figure 3b , the arrows indicate the corresponding reference samples used for prediction, rather than representing the prediction direction. The prediction direction is opposite to the direction indicated by the arrow. When the current block has a rectangular shape, the wide-angle intra prediction mode is a mode in which prediction is performed in a direction opposite to the specific direction mode without additional bit transmission. In this case, in the wide-angle intra prediction mode, some wide-angle intra prediction modes available for the current block can be determined by the ratio of the width to the height of the current block having a rectangular shape. For example, when the current block has a rectangular shape with a height less than the width, a wide-angle intra prediction mode (intra prediction modes #67 to #80) with an angle less than 45 degrees is available. When the current block has a rectangular shape with a width greater than the height, a wide-angle intra prediction mode with an angle greater than -135 degrees is available.

[0045] The intra-frame predictor 122 can determine the intra-frame prediction to be used for encoding the current block. In some examples, the intra-frame predictor 122 can encode the current block by utilizing multiple intra-frame prediction modes, and can also select an appropriate intra-frame prediction mode to use from a test mode. For example, the intra-frame predictor 122 can calculate the rate-distortion value by utilizing a rate-distortion analysis of multiple test intra-frame prediction modes, and can also select the intra-frame prediction mode with the best rate-distortion characteristics from the test mode.

[0046] The intra-frame predictor 122 selects an intra-frame prediction mode from a plurality of intra-frame prediction modes and predicts the current block by using adjacent pixels (reference pixels) and an algorithm equation determined according to the selected intra-frame prediction mode. The entropy encoder 155 encodes information about the selected intra-frame prediction mode and transmits it to the video decoding device.

[0047] The inter-frame predictor 124 generates a prediction block for the current block by utilizing motion compensation processing. The inter-frame predictor 124 searches for a block that is most similar to the current block in a reference image that has been encoded and decoded earlier than the current image, and generates a prediction block for the current block by utilizing the searched block. In addition, a motion vector (MV) is generated, which corresponds to the displacement between the current block in the current image and the prediction block in the reference image. Typically, motion estimation is performed on the luminance (luma) component, and the motion vector calculated based on the luminance component is used for both the luminance component and the chrominance component. The motion information including the information of the reference image and the information about the motion vector used to predict the current block is encoded by the entropy encoder 155 and transmitted to the video decoding device.

[0048] The inter-frame predictor 124 may also perform interpolation of a reference image or reference block to increase prediction accuracy. In other words, subsamples are interpolated between two consecutive integer samples by applying filter coefficients to a plurality of consecutive integer samples. When searching for a block most similar to the current block in the interpolated reference image, fractional precision rather than integer sample precision may be used for the motion vector. The precision or resolution of the motion vector may be set differently for each target region to be encoded, such as a unit such as a slice, tile, CTU, or CU. When such adaptive motion vector resolution (AMVR) is applied, information regarding the motion vector resolution to be applied to each target region should be signaled for each target region. For example, when the target region is a CU, information regarding the motion vector resolution to be applied to each CU is signaled. The information regarding the motion vector resolution may be information representing the precision of the motion vector difference, as described below.

[0049] On the other hand, the inter-frame predictor 124 can perform inter-frame prediction using bidirectional prediction. Bidirectional prediction uses two reference images and two motion vectors representing the block positions most similar to the current block in each reference image. The inter-frame predictor 124 selects a first reference image and a second reference image from reference image list 0 (RefPicList0) and reference image list 1 (RefPicList1), respectively. The inter-frame predictor 124 also searches for blocks most similar to the current block in the corresponding reference images to generate first and second reference blocks. Furthermore, a prediction block for the current block is generated by averaging or weighted averaging the first and second reference blocks. Motion information, including information about the two reference images used to predict the current block and information about the two motion vectors, is transmitted to the entropy encoder 155. Reference image list 0 may consist of images that precede the current image in display order among pre-reconstructed images, and reference image list 1 may consist of images that follow the current image in display order among pre-reconstructed images. However, while not particularly limited to this, pre-reconstructed images that follow the current image in display order may also be included in reference image list 0. Conversely, a pre-reconstructed image preceding the current image may also be additionally included in the reference image list 1 .

[0050] To minimize the amount of bits consumed for encoding motion information, various methods may be used.

[0051] For example, when the reference image and motion vector of the current block are the same as those of a neighboring block, information identifying the neighboring block is encoded to transmit the motion information of the current block to the video decoding device. This method is called merge mode.

[0052] In the merge mode, the inter predictor 124 selects a predetermined number of merge candidate blocks (hereinafter, referred to as “merge candidates”) from neighboring blocks of the current block.

[0053] As the adjacent blocks for deriving the merge candidate, all or some of the left block A0, the lower left block A1, the upper block B0, the upper right block B1 and the upper left block B2 adjacent to the current block in the current image may be used, such as Figure 4 As shown. In addition, in addition to the current image where the current block is located, blocks located in the reference image (which may be the same as or different from the reference image used to predict the current block) can also be used as merge candidates. For example, the co-located block of the current block in the reference image or a block adjacent to the co-located block can be additionally used as a merge candidate. If the number of merge candidates selected by the above method is less than the preset number, a zero vector is added to the merge candidates.

[0054] The inter-frame predictor 124 configures a merge list including a predetermined number of merge candidates using neighboring blocks. From the merge candidates included in the merge list, a merge candidate to be used as motion information for the current block is selected, and merge index information is generated to identify the selected candidate. The generated merge index information is encoded by the entropy encoder 155 and transmitted to the video decoding device.

[0055] Merge skip mode is a special case of merge mode. After quantization, when all transform coefficients used for entropy coding are close to zero, only the neighboring block selection information is transmitted, without the residual signal. By utilizing merge skip mode, relatively high coding efficiency can be achieved for images with minimal motion, still images, and images with on-screen content.

[0056] Hereinafter, merge mode and merge skip mode are collectively referred to as merge / skip mode.

[0057] Another method for encoding motion information is the advanced motion vector prediction (AMVP) mode.

[0058] In the AMVP mode, the inter-frame predictor 124 derives a motion vector prediction candidate for the motion vector of the current block by using the neighboring blocks of the current block. As the neighboring blocks for deriving the motion vector prediction candidate, the neighboring blocks may be used. Figure 4 All or some of the left block A0, lower left block A1, upper block B0, upper right block B1, and upper left block B2 adjacent to the current block in the current image shown. Furthermore, in addition to the current image in which the current block is located, blocks within a reference image (which may be the same as or different from the reference image used to predict the current block) may also be used as adjacent blocks for deriving motion vector prediction candidates. For example, a co-located block of the current block within the reference image or a block adjacent to the co-located block may be used. If the number of motion vector candidates selected by the above method is less than a preset number, a zero vector is added to the motion vector candidates.

[0059] The inter-frame predictor 124 derives motion vector prediction candidates by using the motion vectors of the neighboring blocks, and determines the motion vector prediction of the motion vector of the current block by using the motion vector prediction candidates. In addition, the motion vector difference is calculated by subtracting the motion vector prediction from the motion vector of the current block.

[0060] Motion vector predictions can be obtained by applying a predefined function (e.g., median and mean calculations) to motion vector prediction candidates. In this case, the video decoding device also knows the predefined function. In addition, since the neighboring blocks used to derive motion vector prediction candidates are blocks that have already been encoded and decoded, the video decoding device may also already know the motion vectors of the neighboring blocks. Therefore, the video encoding device does not need to encode information for identifying motion vector prediction candidates. Accordingly, in this case, information about the motion vector difference and information about the reference image used to predict the current block are encoded.

[0061] Alternatively, the motion vector prediction may be determined by selecting any one of the motion vector prediction candidates. In this case, information identifying the selected motion vector prediction candidate is additionally encoded together with information about the motion vector difference used to predict the current block and information about the reference image.

[0062] The subtractor 130 generates a residual block by subtracting a prediction block generated by the intra predictor 122 or the inter predictor 124 from the current block.

[0063] The transformer 140 transforms the residual signal in the residual block having pixel values ​​in the spatial domain into transform coefficients in the frequency domain. The transformer 140 may transform the residual signal in the residual block by using the entire size of the residual block as a transform unit, or may divide the residual block into multiple sub-blocks and perform the transform using the sub-blocks as transform units. Alternatively, the residual block is divided into two sub-blocks, namely a transform region and a non-transform region, to transform the residual signal using only the transform region sub-block as a transform unit. Here, the transform region sub-block may be one of two rectangular blocks with a size ratio of 1:1 based on the horizontal axis (or vertical axis). In this case, the entropy encoder 155 encodes a flag (cu_sbt_flag) indicating that only the sub-block is transformed, as well as direction (vertical / horizontal) information (cu_sbt_horizontal_flag) and / or position information (cu_sbt_pos_flag), and signals these to the video decoding device. In addition, the size of the transform region subblock may have a size ratio of 1:3 based on the horizontal axis (or vertical axis). In this case, the entropy encoder 155 additionally encodes a flag (cu_sbt_quad_flag) for dividing the corresponding partition and signals it to the video decoding apparatus.

[0064] On the other hand, the transformer 140 can perform transformations of the residual block separately in the horizontal direction and the vertical direction. For this transformation, various types of transformation functions or transformation matrices can be used. For example, paired transformation functions for horizontal transformation and vertical transformation can be defined as a multiple transform set (MTS). The transformer 140 can select a transformation function pair with the highest transformation efficiency in the MTS and can transform the residual block in each of the horizontal and vertical directions. The information (mts_idx) about the transformation function pair in the MTS is encoded by the entropy encoder 155 and notified to the video decoding device with a signal.

[0065] The quantizer 145 quantizes the transform coefficients output from the transformer 140 using a quantization parameter and outputs the quantized transform coefficients to the entropy encoder 155. The quantizer 145 can also immediately quantize the relevant residual block without transforming any block or frame. The quantizer 145 can also apply different quantization coefficients (scaling values) depending on the position of the transform coefficient in the transform block. The quantization matrix applied to the quantized transform coefficients arranged in two dimensions can be encoded and signaled to the video decoding device.

[0066] The rearrangement unit 150 may perform rearrangement of coefficient values ​​on the quantized residual value.

[0067] The rearrangement unit 150 can change the 2D coefficient array into a 1D coefficient sequence by using coefficient scanning. For example, the rearrangement unit 150 can use a zigzag scan or a diagonal scan to scan the DC coefficient to the coefficient of the high-frequency region to output a 1D coefficient sequence. Depending on the size of the transform unit and the intra-frame prediction mode, the zigzag scan can also be replaced by a vertical scan that scans the 2D coefficient array in the column direction and a horizontal scan that scans the 2D block type coefficients in the row direction. In other words, depending on the size of the transform unit and the intra-frame prediction mode, the scanning method to be used can be determined from zigzag scanning, diagonal scanning, vertical scanning, and horizontal scanning.

[0068] The entropy encoder 155 encodes the sequence of 1D quantized transform coefficients output from the rearrangement unit 150 by using various encoding schemes including Context-based Adaptive Binary Arithmetic Code (CABAC), Exponential Golomb, etc. to generate a bitstream.

[0069] In addition, the entropy encoder 155 encodes information related to block partitioning (e.g., CTU size, CTU partition flag, QT partition flag, MTT partition type, and MTT partition direction, etc.) so that the video decoding device can partition the block in the same manner as the video encoding device. In addition, the entropy encoder 155 encodes information about the prediction type indicating whether the current block is encoded by intra-frame prediction or inter-frame prediction. The entropy encoder 155 encodes intra-frame prediction information (i.e., information about the intra-frame prediction mode) or inter-frame prediction information (merge index in the case of merge mode, and information about reference image index and motion vector difference in the case of AMVP mode) according to the prediction type. In addition, the entropy encoder 155 encodes information related to quantization (i.e., information about quantization parameters and information about quantization matrices).

[0070] The inverse quantizer 160 inversely quantizes the quantized transform coefficient output from the quantizer 145 to generate a transform coefficient. The inverse transformer 165 transforms the transform coefficient output from the inverse quantizer 160 from the frequency domain to the spatial domain to reconstruct a residual block.

[0071] The adder 170 reconstructs the current block by adding the reconstructed residual block and the prediction block generated by the predictor 120. When performing intra prediction on the next block, the pixels in the reconstructed current block are used as reference pixels.

[0072] The loop filtering unit 180 performs filtering on the reconstructed pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. that occur due to block-based prediction and transformation / quantization. The loop filtering unit 180, as an in-loop filter, may include all or some of the deblocking filter 182, the sample adaptive offset (SAO) filter 184, and the adaptive loop filter (ALF) 186.

[0073] The deblocking filter 182 filters the boundaries between reconstructed blocks to remove blocking artifacts caused by block-based encoding / decoding, and the SAO filter 184 and ALF 186 perform additional filtering on the deblocked video. The SAO filter 184 and ALF 186 are filters used to compensate for the difference between reconstructed and original pixels caused by lossy coding. The SAO filter 184 applies an offset per CTU to enhance subjective image quality and coding efficiency. On the other hand, the ALF 186 performs block-based filtering and applies different filters to compensate for distortion based on the boundaries of the corresponding blocks and the degree of change. Information about the filter coefficients to be used for the ALF can be encoded and signaled to the video decoding device.

[0074] The reconstructed blocks filtered by the deblocking filter 182, the SAO filter 184, and the ALF 186 are stored in the memory 190. When all blocks in one image are reconstructed, the reconstructed image can be used as a reference image for inter-frame prediction of blocks within an image to be subsequently encoded.

[0075] The video encoding apparatus may store a bit stream of encoded video data in a non-volatile storage medium or transmit the bit stream to a video decoding apparatus through a communication network.

[0076] Figure 5 is a functional block diagram of a video decoding device that can implement the technology of the present invention. Figure 5 , describes a video decoding device and components of the device.

[0077] The video decoding apparatus may include an entropy decoder 510 , a rearrangement unit 515 , an inverse quantizer 520 , an inverse transformer 530 , a predictor 540 , an adder 550 , a loop filtering unit 560 , and a memory 570 .

[0078] Similar to Figure 1 Each component of the video encoding device and the video decoding device can be implemented as hardware or software, or as a combination of hardware and software. In addition, the function of each component can be implemented as software, and the microprocessor can also be implemented to execute the function of the software corresponding to each component.

[0079] The entropy decoder 510 extracts information related to block partitioning by decoding a bitstream generated by a video encoding apparatus to determine a current block to be decoded, and extracts prediction information required to reconstruct the current block and information about a residual signal.

[0080] The entropy decoder 510 extracts information about the CTU size from a sequence parameter set (SPS) or a picture parameter set (PPS) to determine the size of the CTU and divides the image into CTUs of the determined size. In addition, the CTU is determined as the highest level (i.e., the root node) of the tree structure, and partition information of the CTU can be extracted to divide the CTU using the tree structure.

[0081] For example, when a CTU is segmented using a QTBTTT structure, a first flag (QT_split_flag) related to QT segmentation is first extracted to segment each node into four nodes in the lower layer. Furthermore, a second flag (mtt_split_flag) related to MTT segmentation, a segmentation direction (vertical / horizontal), and / or a segmentation type (binary / trifurcated) are extracted with respect to nodes corresponding to QT leaf nodes to segment the corresponding leaf nodes into an MTT structure. As a result, each node below the QT leaf node is recursively segmented into a BT or TT structure.

[0082] As another example, when a CTU is split using the QTBTTT structure, a CU split flag (split_cu_flag) indicating whether the CU is split is extracted. When the corresponding block is split, a first flag (QT_split_flag) may also be extracted. During the splitting process, for each node, zero or more recursive MTT splits may occur after zero or more recursive QT splits. For example, for a CTU, MTT splits may occur immediately, or, conversely, multiple QT splits may occur.

[0083] As another example, when a CTU is split using a QTBT structure, a first flag (QT_split_flag) related to the splitting of the QT is extracted to split each node into four nodes in the lower layer. In addition, a split flag (split_flag) indicating whether a node corresponding to a leaf node of the QT is further split into a BT and split direction information are extracted.

[0084] On the other hand, when the entropy decoder 510 determines the current block to be decoded by partitioning using a tree structure, the entropy decoder 510 extracts information about the prediction type indicating whether the current block is intra-predicted or inter-predicted. When the prediction type information indicates intra-prediction, the entropy decoder 510 extracts syntax elements of intra-prediction information (intra-prediction mode) for the current block. When the prediction type information indicates inter-prediction, the entropy decoder 510 extracts information representing syntax elements of inter-prediction information, that is, a motion vector and a reference image referenced by the motion vector.

[0085] In addition, the entropy decoder 510 extracts quantization-related information and extracts information on a quantized transform coefficient of the current block as information on a residual signal.

[0086] The rearrangement unit 515 may change the sequence of 1D quantized transform coefficients entropy-decoded by the entropy decoder 510 into a 2D coefficient array (ie, block) again in the reverse order of the coefficient scanning order performed by the video encoding device.

[0087] The inverse quantizer 520 inversely quantizes the quantized transform coefficients and inversely quantizes the quantized transform coefficients by using a quantization parameter. The inverse quantizer 520 can also apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in 2D. The inverse quantizer 520 can perform inverse quantization by applying a matrix of quantization coefficients (scaling values) from the video encoding device to the 2D array of quantized transform coefficients.

[0088] The inverse transformer 530 reconstructs a residual signal by inversely transforming the inversely quantized transform coefficient from the frequency domain to the spatial domain to generate a residual block of the current block.

[0089] In addition, when the inverse transformer 530 inverse-transforms a partial region (sub-block) of a transform block, the inverse transformer 530 extracts a flag (cu_sbt_flag) indicating that only the sub-block of the transform block is transformed, direction (vertical / horizontal) information (cu_sbt_horizontal_flag) of the sub-block, and / or position information (cu_sbt_pos_flag) of the sub-block. The inverse transformer 530 also inverse-transforms the transform coefficients of the corresponding sub-block from the frequency domain to the spatial domain to reconstruct the residual signal, and fills the uninverse-transformed region with a value of "0" as the residual signal to generate the final residual block of the current block.

[0090] In addition, when MTS is applied, the inverse transformer 530 determines the transform index or transform matrix to be applied in each of the horizontal and vertical directions by using the MTS information (mts_idx) signaled from the video encoding device. The inverse transformer 530 also performs inverse transform on the transform coefficients in the transform block in the horizontal and vertical directions by using the determined transform function.

[0091] The predictor 540 may include an intra predictor 542 and an inter predictor 544. The intra predictor 542 is activated when the prediction type of the current block is intra prediction, and the inter predictor 544 is activated when the prediction type of the current block is inter prediction.

[0092] The intra predictor 542 determines an intra prediction mode of a current block from among a plurality of intra prediction modes according to syntax elements of the intra prediction mode extracted from the entropy decoder 510. The intra predictor 542 also predicts the current block by using neighboring reference pixels of the current block according to the intra prediction mode.

[0093] The inter predictor 544 determines a motion vector of a current block and a reference image to which the motion vector refers by using a syntax element of the inter prediction mode extracted from the entropy decoder 510 .

[0094] The adder 550 reconstructs the current block by adding the residual block output from the inverse transformer 530 to the prediction block output from the inter predictor 544 or the intra predictor 542. When intra-predicting a subsequent block to be decoded, pixels within the reconstructed current block are used as reference pixels.

[0095] The loop filter unit 560, which serves as an in-loop filter, may include a deblocking filter 562, an SAO filter 564, and an ALF 566. The deblocking filter 562 performs deblocking filtering on the boundaries between reconstructed blocks to remove blocking artifacts caused by block-by-block decoding. The SAO filter 564 and the ALF 566 perform additional filtering on the reconstructed blocks after deblocking filtering to compensate for differences between reconstructed and original pixels caused by lossy encoding. The filter coefficients of the ALF are determined by utilizing information about the filter coefficients decoded from the bitstream.

[0096] The reconstructed blocks filtered by the deblocking filter 562, the SAO filter 564, and the ALF 566 are stored in the memory 570. When all blocks in one image are reconstructed, the reconstructed image can be used as a reference image for inter-frame prediction of blocks within an image to be subsequently encoded.

[0097] In some embodiments, the present invention relates to encoding and decoding video images as described above. More specifically, the present invention provides a video encoding and decoding method and apparatus that efficiently transmits information about an affine model when predicting a current block based on the affine model.

[0098] The following embodiments may be performed by the inter-frame predictor 124 in the video encoding apparatus. The following embodiments may also be performed by the inter-frame predictor 544 in the video decoding apparatus.

[0099] The video encoding device can generate signaling information associated with this embodiment from the perspective of optimizing rate distortion when encoding the current block. The video encoding device can use the entropy encoder 155 to encode the signaling information and send the encoded signaling information to the video decoding device. The video decoding device can use the entropy decoder 510 to decode the signaling information associated with decoding the current block from the bitstream.

[0100] In the following description, the term “target block” may be used interchangeably with a current block or a coding unit (CU), or may refer to some regions of a coding unit.

[0101] Furthermore, a flag value of true indicates a case where the flag is set to 1. Furthermore, a flag value of false indicates a case where the flag is set to 0.

[0102] I. Prediction based on affine model

[0103] Figure 6 is a schematic diagram illustrating pixel-by-pixel computation of an affine model according to at least one embodiment of the present invention.

[0104] Figure 6 The embodiment utilizes the possible identity or similarity of pixel values ​​and motion information between the current block and adjacent blocks. The video decoding device uses the motion vectors of the adjacent blocks corresponding to the vertices (A, B, C) of the current block to predict the corresponding control point motion vectors (CPMVs). The video decoding device uses the CPMVs to model the geometric transformation relationship, i.e., the affine model, between the current block and the prediction block, and then performs prediction of the current block based on the modeled transformation relationship. Figure 6 A 6-parameter model using control point motion vectors at three control points A, B, and C is shown, but some embodiments may adopt a 4-parameter model using control point motion vectors at two control points A and B or A and C. According to the 4-parameter model and the 6-parameter model, the motion vector (mv) at the pixel (x, y) of the target block is obtained by using the control point motion vector and the position of each pixel. x , mv y ) can be expressed as shown in Equation 1 and Equation 2 respectively.

[0105] [Equation 1]

[0106]

[0107] [Equation 2]

[0108]

[0109] Here, W and H represent the width and height of the current block. (cpmv ix , cpmv iy ) represents the i-th control point motion vector. The prediction value for each pixel of the current block can be predicted by using the motion vector calculated according to Equation 1 or Equation 2. Although CPMVP4 is not included in the affine model, CPMVP4 can be defined for position D, such as Figure 6 Since position D is a pixel that has not yet been reconstructed, embodiments may use the motion vector of the co-located pixel in the reference picture for CPMVP4.

[0110] exist Figure 6In the example, CPMVP stands for the control point motion vector prediction value in the affine AMVP mode (affine advanced motion vector prediction mode). In affine merge mode, the motion vector difference (MVD) is not transmitted, so CPMVP is the same as CPMV (control point motion vector). In affine AMVP mode, the motion vector difference is transmitted, so CPMV can be calculated by summing CPMVP and MVD. Figure 6 The example in shows a method for generating control point motion vector prediction values ​​in the affine AMVP mode using the construction. The affine merge mode and the affine AMVP mode are described below.

[0111] Figure 7 is a schematic diagram illustrating affine model calculation using a center vector according to at least one embodiment of the present invention.

[0112] Furthermore, to reduce computational complexity in some embodiments, the video decoding apparatus may perform block-by-block prediction by utilizing each control point motion vector as a center vector, such as Figure 7 As shown. At this time, by assuming that the current block has four control point motion vectors, the current block can be divided into four blocks. Figure 7 In the method shown, a block with each control point motion vector as its center vector can be predicted based on the common motion vector. This can reduce computational complexity, although the prediction accuracy may be lower than an implementation that performs pixel-by-pixel calculations.

[0113] Figure 8 is a schematic diagram illustrating sub-block-by-sub-block computation of an affine model according to at least one embodiment of the present invention.

[0114] In another embodiment, in order to reduce computational complexity, the video decoding apparatus may perform sub-block-by-sub-block prediction on each sub-block, such as Figure 8 As shown in . If the horizontal or vertical size of the current block is larger than the horizontal or vertical size of the sub-block, the video decoding device may divide the current block into sub-blocks. The video decoding device may derive the control point motion vector at each vertex position of the divided sub-block by using the control point motion vector of the current block, and then derive a representative motion vector of the control point motion vector of each sub-block. The video decoding device generates a prediction block in units of sub-blocks by using the derived representative motion vector, and then combines the sub-block-by-sub-block prediction blocks to generate the first prediction block of the current block.

[0115] In some embodiments, the video decoding device may subject the first prediction block to filtering to generate the second prediction block.Then, the video decoding device may generate a final prediction block of the current block by using one of the first prediction block and the second prediction block.

[0116] In order to reduce the number of bits required to encode the control point motion vectors, some embodiments may adopt the conventional inter-frame prediction (translational motion prediction) method described above, namely the affine merge mode and the affine AMVP mode. Hereinafter, the affine merge mode and the affine AMVP mode are referred to as the control point motion vector prediction method.

[0117] As an example, in the affine merge mode, the inter-frame predictor 124 of the video encoding apparatus composes a list of a predefined number (e.g., 5) of affine merge candidates. First, the video encoding apparatus derives inherited affine merge candidates from the neighboring blocks of the target block. For example, the video encoding apparatus derives the inherited affine merge candidates from the neighboring blocks of the target block. Figure 4 The neighboring samples (A0, A1, B0, B1, B2) of the target block shown in the figure derive a predefined number of inherited affine merging candidates to generate a merging candidate list. Each of the inherited affine merging candidates included in the candidate list corresponds to a combination of two or three CPMVs.

[0118] The video encoding device derives inherited affine merge candidates from the control point motion vectors of the neighboring blocks of the target block predicted in affine mode. Some embodiments may limit the number of merge candidates derived from the neighboring blocks predicted in affine mode. For example, the video encoding device may derive two inherited affine merge candidates, one of A0 and A1, plus one of B0, B1, and B2, from the neighboring blocks predicted in affine mode. The priority may be in the order of A0, A1, then B0, B1, and B2.

[0119] On the other hand, if the total number of merging candidates is three or more, the video encoding apparatus may additionally derive an insufficient number of constructed affine merging candidates from the translation motion vectors of the adjacent blocks, such as Figure 6 as shown in the example.

[0120] The video encoding device derives control point motion vectors CPMV1, CPMV2 and CPMV3 from each of the adjacent block group {B2, B3, A2}, the adjacent block group {B1, B0} and the adjacent block group {A1, A0}. As an example, the priority within each adjacent block group can be in the order of B2, B3, A2, in the order of B1, B0, and in the order of A1, A0. In addition, the video encoding device derives another control point motion vector CPMV4 from the co-located block C0 in the reference picture. The video encoding device combines two or three of the four control point motion vectors to additionally generate an insufficient number of constructed affine merge candidates. The priority of the combination is as follows. The elements within each group are listed in the following order: upper left, upper right and lower left control point motion vectors.

[0121] {CPMV1, CPMV2, CPMV3}, {CPMV1, CPMV2, CPMV4}, {CPMV1, CPMV3, CPMV4},

[0122] {CPMV2, CPMV3, CPMV4}, {CPMV1, CPMV2}, {CPMV1, CPMV3}

[0123] If the merge candidate list cannot be filled by using the inherited affine merge candidate and the constructed affine merge candidate, the video encoding apparatus may add a zero motion vector as a candidate.

[0124] The video encoding device selects a merge candidate from the merge candidate list to optimize codec efficiency and determines a merge index indicating the selected merge candidate. The video encoding device performs affine motion prediction on the target block using the selected merge candidate. If the merge candidate consists of two control point motion vectors, affine motion prediction is performed using a 4-parameter model. On the other hand, if the merge candidate consists of three control point motion vectors, affine motion prediction is performed using a 6-parameter model. The video encoding device encodes the merge index and signals the encoded merge index to the video decoding device.

[0125] The video decoding device decodes the merge index. The inter-frame predictor 544 of the video decoding device composes a merge candidate list in the same manner as the video encoding device, and performs affine motion prediction by using a control point motion vector corresponding to the merge candidate indicated by the merge index.

[0126] As another example, in the affine AMVP mode, from the perspective of optimizing codec efficiency, the inter-frame predictor 124 of the video encoding device determines the form of the affine model for the target block and its accompanying actual control point motion vector. For each control point, the video encoding device calculates the motion vector difference (MVD) and then encodes the MVD for each control point. The MVD is the difference between the actual control point motion vector and the motion vector prediction value of each control point (the MVP of each control point). To derive the motion vector prediction value for each control point, the inter-frame predictor 124 assembles a list of a predefined number (e.g., 2) of affine AMVP candidates. If the target block is of a 4-parameter type, the candidates included in the list each consist of a pair of two control point motion vectors. On the other hand, if the target block is of a 6-parameter type, the candidates in the list each consist of a set of three control point motion vectors.

[0127] The following describes a method for forming a candidate list in the affine AMVP mode. The affine AMVP candidate list can be derived similarly to the method for forming the affine merged candidate list described above.

[0128] The video encoding apparatus checks whether the reference picture of the inherited affine AMVP candidate is the same as the reference picture of the current block. Here, the inherited affine AMVP candidate can be Figure 4The blocks predicted in the affine mode among the neighboring blocks (A0, A1, B0, B1, B2) of the target block shown in , as in the aforementioned affine merge mode.

[0129] If the reference picture of the inherited affine AMVP candidate is the same as the reference picture of the current block, the video encoding apparatus adds the corresponding inherited affine AMVP candidate.

[0130] On the other hand, if the reference picture of the inherited affine AMVP candidate is different from the reference picture of the current block, the video encoding apparatus checks whether the reference pictures of all CPMVs of the constructed affine AMVP candidate are the same as the reference picture of the current block. Figure 6 The motion vectors of the adjacent samples shown in derive all CPMVPs of the constructed affine AMVP candidates, as in the above-mentioned affine merge mode. If the reference pictures of all CPMVPs of the constructed affine AMVP candidates are the same as the reference pictures of the current block, the video encoding device adds the corresponding constructed affine AMVP candidates.

[0131] At this point, the affine model type of the target block needs to be considered. The video encoding device checks whether the affine model of the target block is of the 4-parameter type. If so, it uses the affine models of neighboring blocks to derive two control point motion vectors: the upper left and upper right control point motion vectors of the target block. If the affine model of the target block is of the 6-parameter type, the video encoding device uses the affine models of neighboring blocks to derive three control point motion vectors: the upper left, upper right, and lower left control point motion vectors of the target block.

[0132] If the reference pictures of all CPMVPs are different from the reference picture of the current block, the video encoding apparatus adds a translational motion vector as an affine AMVP candidate.

[0133] If the candidate list cannot be filled, ie, the preset number of candidates is not met even using all the above steps, the video encoding apparatus will add a zero motion vector for the affine AMVP candidate.

[0134] The video encoding device selects a candidate from the affine AMVP list and determines a candidate index indicating the selected candidate. At this time, each of the control point motion vectors of the selected candidate corresponds to each of the control point motion vector prediction values. From the perspective of optimizing codec efficiency, the video encoding device determines an actual control point motion vector for each control point in the target block and then calculates the MVD between the actual control point motion vector and the control point motion vector prediction value. The video encoding device encodes the affine model form of the target block, the candidate index, and the MVD of each control point, and notifies the video decoding device of the encoded affine model form of the target block, the encoded candidate index, and the encoded MVD of each control point using a signal.

[0135] The video decoding device decodes the affine model form, candidate index, and MVD for each control point. The inter-frame predictor 544 of the video decoding device generates an affine AMVP list in the same manner as the video encoding device and selects a candidate indicated by the candidate index from the affine AMVP list. The video decoding device sums the motion vector prediction value of each control point of the selected candidate and the corresponding MVD to reconstruct the motion vector of each control point. The video decoding device uses the reconstructed control point motion vector to perform affine motion prediction.

[0136] The following embodiments are described with respect to a video decoding apparatus, but they may also be implemented by a video encoding apparatus in the same or similar manner.

[0137] II. Embodiments according to the present invention

[0138] Figure 9 is a schematic diagram illustrating a portion of a video decoding apparatus for affine model-based prediction according to at least one embodiment of the present invention.

[0139] The video decoding apparatus according to some embodiments of the present invention may determine an affine model, perform prediction of the current block based on the determined affine model, and ultimately generate a reconstructed block of the current block. This may be performed by the entropy decoder 510, the inter-frame predictor 544, and the adder 550 of the video decoding apparatus. Figure 9 On the other hand, the image segmenter 110, the predictor 120 and the adder 170 of the video encoding device may perform the following steps: Figure 9 In this case, the video decoding device can use the coding information parsed from the bitstream, while the video encoding device can use the coding information set at a higher level from the perspective of minimizing rate distortion. In the following, for ease of description, the embodiment is described with the video decoding device as the center.

[0140] like Figure 9 As shown, the inter predictor 544 may include all or part of an affine model determiner 910 , a control point motion vector generator 920 , a motion vector generator 930 , and a prediction performer 940 .

[0141] In some embodiments, the video encoding device transmits an affine model application flag, information about the affine model, information about the control point motion vector, and all or part of the residual block. Here, the affine model application flag indicates whether the current block is a prediction block based on prediction based on the affine model.

[0142] The entropy decoder 510 may decode all or part of an affine model application flag, affine model information, information on a control point motion vector prediction method, a residual block, and a reference picture index from a bitstream transmitted by a video encoding apparatus.

[0143] The affine model application flag indicates whether the current block is predicted using an affine model. The affine prediction model information indicates the form of the affine model, i.e., whether it is a 4-parameter model or a 6-parameter model. The control point motion vector information includes information about the control point motion vectors depending on the affine model, such as the number of control point motion vectors, the control point motion vector prediction method, and the control point motion vector difference. Here, the control point motion vector prediction method can be the affine merge mode or the affine AMVP mode.

[0144] If the affine model application flag regarding the affine model-based prediction is true, the affine model determiner 910 may utilize the affine model information as a basis for determining the form of the affine model to derive the number of control point motion vectors.

[0145] The control point motion vector generator 920 predicts the control point motion vector for the current block. Based on relevant syntax and prediction mode information of decoded neighboring blocks of the current block, the control point motion vector generator 920 determines the number of control point motion vectors in the current block and generates corresponding control point motion vector prediction values. Here, the relevant syntax includes the control point motion vector prediction method, reference picture index, etc.

[0146] After generating the control point motion vector prediction value, the control point motion vector generator 920 adds the control point motion vector difference value to the control point motion vector prediction value to calculate the control point motion vector. If the control point motion vectors of the affine model have already been transmitted in the affine merge mode, the process of adding the control point motion vector difference value to the control point motion vector prediction value can be omitted. Here, the affine merge mode refers to a method of determining the control point motion vector in the same manner as adjacent vectors without motion vector differences.

[0147] The motion vector generator 930 calculates a motion vector by using a control point motion vector in a unit used for the current block when calculating the motion vector of the current block. For example, if a subblock is a unit for calculating a motion vector, the motion vector can be calculated in subblock units.

[0148] The prediction executor 940 may perform interpolation filtering on the reference block according to the accuracy of the motion vector. The prediction executor 940 may generate a prediction block for the current block by performing motion prediction compensation using the motion vector. The prediction executor 940 may then sum the prediction block and the decoded residual block to generate a reconstructed block for the current block.

[0149] Figures 10a to 10d is a schematic diagram illustrating motion vectors of sub-block units according to at least one embodiment of the present invention.

[0150] In one example, for a sub-block of size 4×4, target blocks A, B, C, and D of size 8×8 establish a geometric relationship with the reference block, such as Figure 10a To perform decoding based on the right affine model, decoding based on the 6-parameter model needs to be combined with Figure 10b The dotted arrows in the figure correspond to the vector information, and the decoding based on the 4-parameter model requires the same Figure 10c In addition, the motion vector of the sub-block unit calculated based on the corresponding vector information can be as follows: Figure 10d is conceptualized as in the example above.

[0151] exist Figure 10b and Figure 10c In the example of , some of the control point motion vectors of blocks A, B, C, and D may overlap with each other, as at the magnified vertices. That is, if the geometric relationship is continuous from block to block, overlapping information may occur. Therefore, if the affine model information is transmitted separately for each of blocks A, B, C, and D, retransmission of overlapping information will occur. Here, the overlapping information may be, for example, a control point motion vector, which may include a control point motion vector prediction value and / or a control point motion vector difference value. At the overlapping vertices, by utilizing the overlapping information of the reconstructed blocks, the video decoding device may derive the control point motion vector of the current block, i.e., the control point motion vector prediction value or the control point motion vector difference value.

[0152] Figures 11a to 11c is a schematic diagram illustrating the derivation of overlapping control point motion vectors according to at least one embodiment of the present invention.

[0153] When the target block has Figure 11a When the four control point motion vectors are defined in the example, the four target blocks A, B, C and D can have the following characteristics: Figure 11b The dashed arrows in show the defined control point motion vectors.

[0154] In one example, a video decoding apparatus performs affine model-based prediction by utilizing overlapping control point motion vectors derived from the control point motion vectors of the reconstructed block, without additionally transmitting a motion vector difference (MVD). According to the decoding sequence, affine model-based decoding may be performed on block A, and then affine model-based decoding may be performed on block B. When the MVD of CPMVP1 of block B is resolved to 0 under the application of affine AMVP and CPMVP3 of block B is predicted from the motion vector of block A or a sub-block of block A, the video decoding apparatus may omit the transmission of the MVD of CPMVP3 of block B.

[0155] Thereafter, affine model-based prediction may be applied to blocks A and B, and then affine model-based prediction may be applied to block C. When the MVD of CPMVP1 of block C is 0 and the MVD of CPMVP3 of block B is omitted or 0 under the application of affine AMVP, the video decoding apparatus may omit transmitting the MVD of CPMVP2 of block C.

[0156] Thereafter, the affine model-based prediction may be applied to all A, B, and C blocks, and then the affine model-based prediction may be applied to the D block. When the MVD of the CPMVP3 of the B block is omitted or is 0 and the MVD of the CPMVP2 of the C block is 0 or omitted, the video decoding device may completely omit transmitting the MVD of the control point motion vector of the D block. For example, when the MVD of the CPMVP3 of the B block is omitted or is 0 and the CPMVP2 of the D block is predicted from the motion vector of the B block or a sub-block of the B block, the transmission of the MVD of the CPMVP3 for the D block may be omitted. In addition, if the MVD of the CPMVP2 of the block C is omitted or is zero, and the CPMVP3 of the block D is the motion vector of the block C (or the sub-block of the block C), the video decoding device may omit transmitting the MVD of the CPMVP2 of the D block.

[0157] In summary, the present invention can omit the transmission of the MVD corresponding to the CPMVP at the overlapping vertices in the target block when predicting the target block according to the affine AMVP mode, the MVD of the CPMVP at the overlapping vertices in the reconstructed block is omitted or is 0, and the CPMVP in the target block is predicted from the motion vector of the reconstructed block or a sub-block of the reconstructed block based on the overlapping vertices. Here, the second condition of omitting the MVD of the CPMVP at the overlapping vertices in the reconstructed block indicates that the reconstructed block is predicted according to the affine merge mode. The alternative condition that the MVD of the CPMVP at the overlapping vertices in the reconstructed block is 0 indicates that the reconstructed block is predicted according to the affine AMVP mode, and the MVD of the CPMVP at the overlapping vertices is 0.

[0158] Figure 11c The enlarged vertices shown in represent vertices for which transmission of the MVD can be omitted based on the overlap as described above.

[0159] In some embodiments, a weight may be further transmitted for calculating the motion vector of the sub-block within the current block. The video decoding device may utilize the weight w to scale the motion vector of the sub-block calculated based on the corresponding control point motion vector.

[0160] For example, for a 4-parameter model, the sub-block motion vector can be predicted as shown in Equation 3.

[0161] [Equation 3]

[0162]

[0163] Here, w 2x +w 1x =1, so only w can be transmitted 1x and w 2x One of them, and the other can be derived. In addition, since w 2y +w 1y =1, only w can be transmitted 1y and w 2y of one, and can deduce the other.

[0164] Furthermore, for the 6-parameter model, the sub-block motion vector can be predicted as shown in Equation 4.

[0165] [Equation 4]

[0166]

[0167] As another example, for efficiency in weight transmission, if the size of the current block is at least N times wider horizontally or vertically and at least M times longer than the size of the subblock (for which the motion vector is calculated in the affine model-based prediction), the current block may be weighted, where N and M are integers greater than or equal to 2. For example, there may be a subblock of size 4×4 (for which the motion vector is calculated). The current block may have an area greater than 16 times that of the subblock, and if the weight w is transmitted when the current block is divided into two or more subblocks in the horizontal and vertical directions, the minimum area of ​​the current block may be 16×16, 32×8, or 8×32.

[0168] Now refer to Figure 12a 、 Figure 12b and Figure 13 , describes the method of predicting the current block based on the affine model.

[0169] Figure 12a and Figure 12b is a flowchart of a method of encoding a current block by a video encoding apparatus according to at least one embodiment of the present invention.

[0170] The video encoding apparatus determines affine model information and a prediction method for control point motion vectors (S1200). Here, the affine model information refers to the form of the affine model, i.e., whether it is a 4-parameter model or a 6-parameter model. The control point motion vector prediction method refers to either the affine merge mode or the affine AMVP mode. The affine model information and the prediction method for control point motion vectors can be determined from the perspective of rate-distortion optimization.

[0171] The video encoding apparatus determines the form of the affine model based on the affine model information ( S1202 ).

[0172] The video encoding apparatus derives a control point motion vector based on an affine model ( S1204 ).

[0173] The video encoding apparatus generates a motion vector for each prediction unit of the current block using the control point motion vector (S1206). Here, the prediction unit may be a pixel, a block containing each control point motion vector, or a subblock within the current block.

[0174] The video encoding apparatus generates a prediction value of each prediction unit by using a motion vector and a reference picture, thereby generating a first prediction block of the current block ( S1208 ).

[0175] The video encoding apparatus determines a motion vector of a current block by using a reference picture (S1210). The motion vector of the current block may be determined from a rate-distortion optimization perspective.

[0176] The video encoding apparatus uses the motion vector to generate a second prediction block of the current block ( S1212 ).

[0177] The video encoding apparatus determines an affine model application flag based on the first prediction block and the second prediction block (S1214). Here, the affine model application flag indicates whether the current block is to be predicted based on the prediction based on the affine model. For example, if the first prediction block is optimal, the affine model application flag may be determined to be true. On the other hand, if the second prediction block is optimal, the affine model application flag may be determined to be false.

[0178] The video encoding apparatus encodes an affine model application flag ( S1216 ).

[0179] The video encoding apparatus checks the affine model application flag (S1218).

[0180] If the affine model application flag is true (Yes in S1218 ), the video encoding apparatus performs the following steps.

[0181] The video encoding apparatus derives a control point motion vector prediction value according to a prediction method for the affine model and the control point motion vector (S1220).

[0182] The video encoding apparatus subtracts each of the control point motion vector prediction values ​​from each of the control point motion vectors to generate a control point motion vector difference value ( S1222 ).

[0183] On the other hand, at the overlapping vertex between the coded block and the current block, the video encoding apparatus may use the control point motion vector at the overlapping vertex in the coded block to derive the control point motion vector at the overlapping vertex in the current block. The control point motion vector may include a control point motion vector prediction value and / or a control point motion vector difference value.

[0184] The video encoding apparatus encodes the affine model information, the index indicating the reference picture, the prediction method for the control point motion vector, and the control point motion vector difference (S1224).

[0185] When the prediction method for the control point motion vectors is the affine merge mode, the generation and encoding of the control point motion vector differences can be omitted.

[0186] In addition, when the control point motion vector difference of the control point motion vector prediction value at the overlapping vertex in the encoded block and the current block is omitted or is 0, and the control point motion vector prediction value at the overlapping vertex in the current block is predicted based on the motion vector of the encoded block or the sub-block of the encoded block, the encoding of the control point motion vector difference corresponding to the control point motion vector prediction value at the overlapping vertex in the current block can be omitted.

[0187] If the affine model application flag is false (No in S1218), the video encoding apparatus encodes motion vector information of the current block (S1230). Here, the motion vector information may include an index indicating a reference picture, a prediction method for a motion vector (e.g., merge mode or AMVP mode), a motion vector difference, etc.

[0188] Then, the video encoding apparatus may subtract the prediction block from the original block of the current block to generate a residual block, and encode the generated residual block.

[0189] Figure 13 is a flowchart of a method for reconstructing a current block by a video decoding device according to at least one embodiment of the present invention.

[0190] The video decoding apparatus decodes an affine model application flag from a bitstream (S1300). Here, the affine model application flag indicates whether a current block is to be predicted according to prediction based on an affine model.

[0191] The video decoding apparatus checks an affine model application flag ( S1302 ).

[0192] If the affine model application flag is true (Yes in S1302 ), the video decoding apparatus takes the following steps.

[0193] The video decoding apparatus decodes all or part of the affine model information, control point motion vector information, and reference picture index from the bitstream (S1304). Here, the affine model information refers to the form of the affine model, that is, whether it is a 4-parameter model or a 6-parameter model. The control point motion vector information includes the prediction method for the control point motion vector and the control point motion vector difference. The prediction method for the control point motion vector is the affine merge mode or the affine AMVP mode.

[0194] At the overlapping vertices between the reconstructed block and the current block, the video decoding apparatus may utilize the control point motion vectors at the overlapping vertices in the reconstructed block to derive the control point motion vectors at the overlapping vertices in the current block. The control point motion vectors may include control point motion vector prediction values ​​and / or control point motion vector differences.

[0195] For example, when the control point motion vector difference of the control point motion vector prediction value at the overlapping vertex in the reconstructed block is omitted or is 0, and the control point motion vector prediction value at the overlapping vertex in the current block is predicted from the motion vector of the reconstructed block or a sub-block of the reconstructed block based on the overlapping vertex, the decoding of the control point motion vector difference corresponding to the control point motion vector prediction value at the overlapping vertex in the current block can be omitted.

[0196] The video decoding apparatus determines the form of the affine model based on the affine model information ( S1306 ).

[0197] The video decoding apparatus derives a control point motion vector based on the affine model and the control point motion vector information ( S1308 ).

[0198] The video decoding apparatus determines the number of control point motion vectors based on an affine model, and generates a predicted value for each control point motion vector based on a prediction method for the control point motion vector and prediction mode information of a decoded block adjacent to the current block. The video decoding apparatus generates each control point motion vector by summing the predicted value for each control point motion vector and a corresponding control point motion vector difference.

[0199] The video decoding apparatus generates a motion vector for each prediction unit of the current block by using the control point motion vector (S1310). Here, the prediction unit may be a pixel, a block including each control point motion vector, or a subblock within the current block.

[0200] The video decoding apparatus generates a prediction value for each prediction unit using the motion vector and the reference picture indicated by the reference picture index, and thereby generates a prediction block for the current block ( S1312 ).

[0201] If the affine model application flag is false (No in S1302 ), the video decoding apparatus generates a prediction block of the current block by using another inter prediction method instead of performing prediction based on the affine model ( S1320 ).

[0202] Then, the video decoding apparatus may sum the prediction block and the decoded residual block to generate a reconstructed block of the current block.

[0203] Although the steps in the various flowcharts are described as being performed sequentially, these steps merely illustrate the technical concepts of some embodiments of the present invention. Therefore, a person skilled in the art to which the present invention pertains can implement the steps by changing the order in which they are described in the various figures or by performing two or more steps in parallel. Therefore, the steps in the various flowcharts are not limited to the chronological order shown.

[0204] It should be understood that the above description presents illustrative embodiments that can be implemented in various other ways. The functions described in some embodiments can be implemented by hardware, software, firmware, and / or a combination thereof. It should also be understood that the functional components described in the present invention are labeled as "units" to highlight their ability to be implemented independently.

[0205] On the other hand, the various methods or functions described in some embodiments can be implemented as instructions stored in a non-volatile recording medium, which can be read and executed by one or more processors. The non-volatile recording medium can include, for example, various types of recording devices that store data in a form readable by a computer system. For example, the non-volatile recording medium can include storage media such as erasable programmable read-only memory (EPROM), flash drives, optical disk drives, magnetic hard drives, and solid-state drives (SSDs), etc.

[0206] Although exemplary embodiments of the present invention have been described for illustrative purposes, it will be understood by those skilled in the art that various modifications, additions, and substitutions may be made without departing from the spirit and scope of the present invention. Therefore, embodiments of the present invention have been described for brevity and clarity. The scope of the technical ideas of the embodiments of the present invention is not limited by the examples. Accordingly, it will be understood by those skilled in the art that the scope of the present invention should not be limited by the embodiments clearly described above, but by the claims and their equivalents.

[0207] Reference numerals

[0208] 124: Inter-frame predictor

[0209] 544: Interframe predictor

[0210] 910: Control Point Motion Vector Generator

[0211] 920: Motion Vector Generator

[0212] 930: Predictive Executor.

[0213] CROSS-REFERENCE TO RELATED APPLICATIONS

[0214] This application claims priority to and the benefit of Korean Patent Application No. 10-2022-0162308, filed on November 29, 2022, and Korean Patent Application No. 10-2023-0135316, filed on October 11, 2023, the entire contents of each of which are incorporated herein by reference.

Claims

1. A method for decoding a current block by a video decoding device, the method comprising: decoding all or part of affine model information, control point motion vector information, and a reference picture index from a bitstream, the affine model information indicating a form of the affine model, and the control point motion vector information including a prediction method for a control point motion vector and a control point motion vector difference; determining a form of the affine model based on the affine model information; Derivation of control point motion vectors based on the affine model and control point motion vector information; generating a motion vector for each prediction unit of the current block by using the control point motion vector; as well as generating a prediction block of the current block by generating a prediction value for each prediction unit of the current block using a motion vector and a reference picture indicated by a reference picture index, thereby generating a prediction block of the current block, The decoding of the control point motion vector information includes: At the overlapping vertices between the reconstructed block and the current block, the control point motion vectors at the overlapping vertices in the current block are derived by using the control point motion vectors at the overlapping vertices in the reconstructed block.

2. The method according to claim 1, further comprising: decoding an affine model application flag from a bitstream, the affine model application flag indicating whether a current block is to be predicted according to a prediction based on an affine model; as well as Check the affine model application flag, The method further includes, when the affine model application flag is true: performing prediction based on the affine model.

3. The method according to claim 1, wherein Determining the form of the affine model includes: The affine model is determined as a 4-parameter model using two control point motion vectors, or a 6-parameter model using three control point motion vectors.

4. The method according to claim 1, wherein Derivation of the control point motion vector includes: Determine the number of control point motion vectors according to the affine model; generating a motion vector prediction value for each control point based on a prediction method and prediction mode information of neighboring decoded blocks of the current block; and Each of the control point motion vectors is generated by summing each control point motion vector prediction value and the corresponding control point motion vector difference value.

5. The method according to claim 1, wherein The prediction unit includes: The pixel, block containing the motion vector for each control point, or a sub-block within the current block.

6. The method according to claim 5, wherein: Derivation of the control point motion vector includes: When the prediction method is in affine merge mode, each control point motion vector prediction value is used for each of the control point motion vectors.

7. The method according to claim 1, wherein Decoding control point motion vector information includes: In response to causing a control point motion vector difference value of a control point motion vector prediction value at an overlapping vertex in a reconstructed block to be skipped or zero, and in response to predicting the control point motion vector prediction value at an overlapping vertex in a current block based on a motion vector from the reconstructed block or a sub-block of the reconstructed block, decoding of a control point motion vector difference value corresponding to the control point motion vector prediction value at the overlapping vertex in the current block is skipped.

8. The method according to claim 1, further comprising: Decode weights from the bitstream; as well as Generating a motion vector for each prediction unit includes: The motion vector of each prediction unit is scaled by using a weight.

9. A method for encoding a current block by a video encoding apparatus, the method comprising: determining affine model information and a prediction method for a control point motion vector, the affine model information indicating a form of an affine model; determining a form of the affine model based on the affine model information; Derivation of control point motion vectors based on affine model; generating a motion vector for each prediction unit of the current block by using the control point motion vector; generating a first prediction block of the current block by generating a prediction value for each prediction unit using a motion vector and a reference picture; deriving control point motion vector prediction values ​​based on an affine model and a prediction method for control point motion vectors, and generating control point motion vector differences by subtracting each of the control point motion vector prediction values ​​from each of the control point motion vectors; as well as Encodes affine model information, an index indicating a reference picture, a prediction method for control point motion vectors, and control point motion vector differences, The generation of the control point motion vector difference includes: At the overlapping vertices between the coded block and the current block, the control point motion vectors at the overlapping vertices in the current block are derived by using the control point motion vectors at the overlapping vertices in the coded block.

10. The method according to claim 9, further comprising: Determine the motion vector of the current block by using the reference picture; as well as A second prediction block of the current block is generated by using the motion vector.

11. The method according to claim 10, further comprising: determining an affine model application flag based on the first prediction block and the second prediction block, the affine model application flag indicating whether to predict the current block according to prediction based on the affine model; as well as Encodes the affine model application flag.

12. The method according to claim 11, further comprising: Check the affine model application flag, The method further comprises, when the affine model application flag is true: The control point motion vector prediction value is further derived by encoding the control point motion vector difference.

13. The method according to claim 9, wherein: Encoding the control point motion vector differences includes: In response to making the control point motion vector difference of the control point motion vector prediction value at the overlapping vertex in the coded block and the current block skipped or zero, and in response to predicting the control point motion vector prediction value at the overlapping vertex in the current block based on the motion vector from the coded block or the sub-block of the coded block, encoding of the control point motion vector difference corresponding to the control point motion vector prediction value at the overlapping vertex in the current block is skipped.

14. A computer-readable recording medium storing a bitstream generated by a video encoding method, the video encoding method comprising: determining affine model information and a prediction method for a control point motion vector, the affine model information indicating a form of an affine model; determining a form of the affine model based on the affine model information; Derivation of control point motion vectors based on affine model; generating a motion vector for each prediction unit of the current block by using the control point motion vector; generating a first prediction block of the current block by generating a prediction value for each prediction unit using a motion vector and a reference picture; deriving control point motion vector prediction values ​​based on an affine model and a prediction method for control point motion vectors, and generating control point motion vector differences by subtracting each of the control point motion vector prediction values ​​from each of the control point motion vectors; as well as Encodes affine model information, an index indicating a reference picture, a prediction method for control point motion vectors, and control point motion vector differences, The generation of the control point motion vector difference includes: At the overlapping vertices between the coded block and the current block, the control point motion vectors at the overlapping vertices in the current block are derived by using the control point motion vectors at the overlapping vertices in the coded block.

Images