Video encoding and decoding method, storage medium, and transmission method of image data
By optimizing the construction process of the affine motion vector prediction sub-candidate list, the problem of low coding efficiency in high-resolution, high-quality image and video coding is solved, achieving more efficient video compression and storage, and reducing hardware costs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-09-10
- Publication Date
- 2026-03-17
AI Technical Summary
Existing technologies suffer from low coding efficiency in video coding of high-resolution, high-quality images, especially when constructing a candidate list for affine motion vector prediction. Insufficient candidate numbers or excessive complexity lead to increased transmission and storage costs.
By constructing an affine motion vector predictor (MVP) candidate list, affine MVP candidates are derived only when all candidate motion vectors are available. By utilizing inherited and constructed candidate motion vectors, the construction process of the affine MVP candidate list is optimized, reducing complexity and improving coding efficiency.
It improves the overall compression efficiency of video encoding, reduces hardware costs and storage requirements, and enhances encoding efficiency.
Smart Images

Figure CN116156189B_ABST
Abstract
Description
[0001] This application is a divisional application of the original invention patent application No. 201980006206.6 (International Application No.: PCT / KR2019 / 011733, Application Date: September 10, 2019, Invention Title: Image Decoding Method and Apparatus Based on Affine Motion Prediction Using Affine MVP Candidate List in Image Coding System). Technical Field
[0002] This disclosure relates to video coding technology, and more specifically, to a video decoding method and apparatus based on affine motion prediction in a video coding system. Background Technology
[0003] The demand for high-resolution, high-quality images, such as HD (high-definition) and UHD (ultra-high-definition) images, is growing across various fields. Because image data is high-resolution and high-quality, the amount of information or bits to be transmitted increases compared to traditional image data. Therefore, transmission and storage costs increase when using media such as traditional wired / wireless broadband lines to send image data or when storing image data using existing storage media.
[0004] Therefore, there is a need for efficient image compression technology to effectively transmit, store, and reproduce information from high-resolution, high-quality images. Summary of the Invention
[0005] Technical issues
[0006] The technical objective of this disclosure is to provide a method and apparatus for improving video encoding efficiency.
[0007] Another technical objective of this disclosure is to provide a video decoding method and apparatus that constructs an affine MVP candidate list for the current block by means of affine MVP candidates derived from neighboring blocks only when all candidate motion vectors of the CP are available, and performs prediction of the current block based on the constructed affine MVP candidate list.
[0008] Another technical objective of this disclosure is to provide a video decoding method and apparatus that derives affine MVP candidates by using candidate motion vectors derived from a process for deriving constructed affine MVP candidates as additional affine MVP candidates when the number of available inherited affine MVP candidates and constructed affine MVP candidates (i.e., the number of candidates in the MVP candidate list) is less than a maximum number; and performs prediction of the current block based on the constructed affine MVP candidate list.
[0009] Technical solution
[0010] According to one embodiment of this disclosure, a video decoding method performed by a decoding device is provided. The method includes: obtaining motion prediction information of a current block from a bitstream; constructing an affine motion vector predictor (MVP) candidate list for the current block; deriving a control point motion vector predictor (CPMVP) for a control point (CP) of the current block based on the affine MVP candidate list; deriving a control point motion vector difference (CPMVD) for the CP of the current block based on the motion prediction information; deriving a control point motion vector (CPMV) for the CP of the current block based on the CPMVP and CPVD; deriving a prediction sample for the current block based on the CPMV; and generating a current block prediction sample based on the derived prediction sample. The reconstructed image of the previous block, wherein the steps of constructing the affine MVP candidate list include: checking whether the inherited affine MVP candidates of the current block are available, wherein, if the inherited affine MVP candidates are available, the inherited affine MVP candidates are derived; checking whether the constructed affine MVP candidates of the current block are available, wherein, if the constructed affine MVP candidates are available, the constructed affine MVP candidates are derived, and the constructed affine MVP candidates include the candidate motion vectors of CP0, CP1, and CP2 of the current block; when the derived affine MVP candidates are available, the constructed affine MVP candidates are derived. When the number of MVP candidates is less than 2 and the motion vector of CP0 is available, a first affine MVP candidate is derived, wherein the first affine MVP candidate is an affine MVP candidate that includes the motion vector of CP0 as a candidate motion vector of CP; when the number of derived affine MVP candidates is less than 2 and the motion vector of CP1 is available, a second affine MVP candidate is derived, wherein the second affine MVP candidate is an affine MVP candidate that includes the motion vector of CP1 as a candidate motion vector of CP; when the number of derived affine MVP candidates is less than 2 and the motion vector of CP2 is available, a third affine MVP candidate is derived, wherein the third affine MVP candidate is an affine MVP candidate that includes the motion vector of CP2 as a candidate motion vector of CP; when the number of derived affine MVP candidates is less than 2, a fourth affine MVP candidate is derived, the fourth affine MVP candidate including the time MVP derived based on the time adjacent blocks of the current block as a candidate motion vector of CP; and when the number of derived affine MVP candidates is less than 2, a fifth affine MVP candidate is derived, the fifth affine MVP candidate including the zero motion vector as a candidate motion vector of CP.
[0011] According to another embodiment of this disclosure, a decoding apparatus for performing video encoding is provided. The decoding apparatus includes: an entropy decoder that obtains motion prediction information for a current block; a predictor that constructs a candidate list of affine motion vector predictors (MVPs) for the current block, derives control point motion vector predictors (CPMVPs) for control points (CPs) of the current block based on the affine MVP candidate list, derives control point motion vector differences (CPMVDs) for the CPs of the current block based on the motion prediction information; derives control point motion vectors (CPMVs) for the CPs of the current block based on the CPVDs; derives prediction samples for the current block based on the CPMVs; and an adder that, based on the derived prediction samples... This generates a reconstructed image of the current block, wherein the affine MVP candidate list is constructed based on the following operations: checking whether the inherited affine MVP candidates of the current block are available, wherein the inherited affine MVP candidates are derived when they are available; checking whether the constructed affine MVP candidates of the current block are available, wherein the constructed affine MVP candidates are derived when they are available, and the constructed affine MVP candidates include the candidate motion vectors of CP0, CP1, and CP2 of the current block; When the number of derived affine MVP candidates is less than 2 and the motion vector of CP0 is available, a first affine MVP candidate is derived, wherein the first affine MVP candidate is an affine MVP candidate that includes the motion vector of CP0, which is a candidate motion vector of CP; when the number of derived affine MVP candidates is less than 2 and the motion vector of CP1 is available, a second affine MVP candidate is derived, wherein the second affine MVP candidate is an affine MVP candidate that includes the motion vector of CP1, which is a candidate motion vector of CP; when the number of derived affine MVP candidates is less than 2 and the motion vector of CP2 .... When a vector is available, a third affine MVP candidate is derived, wherein the third affine MVP candidate is an affine MVP candidate that includes the motion vector of CP2 as a candidate motion vector of CP; when the number of derived affine MVP candidates is less than 2, a fourth affine MVP candidate is derived, wherein the fourth affine MVP candidate includes the time MVP derived based on the time neighboring blocks of the current block as a candidate motion vector of CP; and when the number of derived affine MVP candidates is less than 2, a fifth affine MVP candidate is derived, wherein the fifth affine MVP candidate includes the zero motion vector as a candidate motion vector of CP.
[0012] According to another embodiment of this disclosure, a video coding method performed by an encoding device is provided. The method includes: constructing an affine motion vector predictor (MVP) candidate list for the current block; deriving a control point motion vector predictor (CPMVP) for a control point (CP) of the current block based on the affine MVP candidate list; deriving a CPMV for the current block's CP; deriving a control point motion vector difference (CPMVD) for the current block's CP based on the CPMVP and CPMV; and encoding motion prediction information including information about the CPMVD. The step of constructing the affine MVP candidate list includes: checking whether inherited affine MVP candidates for the current block are available, wherein inherited affine MVP candidates are derived when they are available; checking whether constructed affine MVP candidates for the current block are available, wherein constructed affine MVP candidates are derived when they are available, and the constructed affine MVP candidates include candidate motion vectors for CP0, candidate motion vectors for CP1, and candidate motion vectors for CP2 of the current block; and when the number of derived affine MVP candidates is less than 2 and the motion vector of CP0 is available... The process involves deriving a first affine MVP candidate, where the first affine MVP candidate is an affine MVP candidate that includes the motion vector of CP0 as a candidate motion vector of CP; when the number of derived affine MVP candidates is less than 2 and the motion vector of CP1 is available, a second affine MVP candidate is derived, where the second affine MVP candidate is an affine MVP candidate that includes the motion vector of CP1 as a candidate motion vector of CP; when the number of derived affine MVP candidates is less than 2 and the motion vector of CP2 is available, a third affine MVP candidate is derived, where the third affine MVP candidate is an affine MVP candidate that includes the motion vector of CP2 as a candidate motion vector of CP; when the number of derived affine MVP candidates is less than 2, a fourth affine MVP candidate is derived, where the fourth affine MVP candidate includes the time MVP derived based on the time adjacent blocks of the current block as a candidate motion vector of CP; and when the number of derived affine MVP candidates is less than 2, a fifth affine MVP candidate is derived, where the fifth affine MVP candidate includes the zero motion vector as a candidate motion vector of CP.
[0013] According to another embodiment of this disclosure, a video encoding device is provided. The encoding device includes: a predictor that constructs a candidate list of affine motion vector predictors (MVPs) for the current block, derives the control point motion vector predictors (CPMVPs) for the control points (CPs) of the current block, and derives the CPMV of the current block's CP; a subtractor that derives the control point motion vector difference (CPMVD) of the current block's CP based on the CPMVPs and CPMVs; and an entropy encoder that encodes motion prediction information including information about the CPMVD, wherein the affine MVP candidate list is constructed based on the following operations: checking whether inherited affine MVP candidates for the current block are available, wherein inherited affine MVP candidates are derived when they are available; checking whether constructed affine MVP candidates for the current block are available, wherein constructed affine MVP candidates are derived when they are available, and the constructed affine MVP candidates include candidate motion vectors of CP0, candidate motion vectors of CP1, and candidate motion vectors of CP2 of the current block; when the number of derived affine MVP candidates is less than 2 and the motion vector of CP0 is less than 2, the affine MVP candidate list is constructed based on the following operations: checking whether inherited affine MVP candidates for the current block are available, wherein inherited affine MVP candidates are derived when they are available; and ... When a vector is available, a first affine MVP candidate is derived, wherein the first affine MVP candidate is an affine MVP candidate that includes the motion vector of CP0 as a candidate motion vector of CP; when the number of derived affine MVP candidates is less than 2 and the motion vector of CP1 is available, a second affine MVP candidate is derived, wherein the second affine MVP candidate is an affine MVP candidate that includes the motion vector of CP1 as a candidate motion vector of CP; when the number of derived affine MVP candidates is less than 2 and the motion vector of CP2 is available, a third affine MVP candidate is derived, wherein the third affine MVP candidate is an affine MVP candidate that includes the motion vector of CP2 as a candidate motion vector of CP; when the number of derived affine MVP candidates is less than 2, a fourth affine MVP candidate is derived, wherein the fourth affine MVP candidate includes the time MVP derived based on the time adjacent blocks of the current block as a candidate motion vector of CP; and when the number of derived affine MVP candidates is less than 2, a fifth affine MVP candidate is derived, wherein the fifth affine MVP candidate includes the zero motion vector as a candidate motion vector of CP.
[0014] Beneficial effects
[0015] According to this disclosure, the overall image / video compression efficiency can be improved.
[0016] According to this disclosure, the efficiency of video coding based on affine motion prediction can be improved.
[0017] According to this disclosure, in deriving the affine MVP candidate list, a constructed affine MVP candidate can only be added when all candidate motion vectors of the CP of the constructed affine MVP candidate are available. This reduces the complexity of deriving the constructed affine MVP candidate and constructing the affine MVP candidate list, and improves coding efficiency.
[0018] According to this disclosure, in deriving the affine MVP candidate list, additional affine MVP candidates can be derived based on the candidate motion vectors of the CP derived from the processing used to derive the constructed affine MVP candidates, thereby reducing the complexity of constructing the affine MVP candidate list and improving coding efficiency.
[0019] According to this disclosure, in deriving an inherited affine MVP candidate, the inherited affine MVP candidate can only be derived by using the upper neighbor block if the upper neighbor block is included in the current CTU. This reduces the storage of the row buffer used for affine prediction and minimizes hardware costs. Attached Figure Description
[0020] Figure 1 Examples of video / image coding systems to which this disclosure can be applied are illustrated.
[0021] Figure 2 This is a schematic diagram illustrating the configuration of a video / image encoding device to which the implementation methods described in this document can be applied.
[0022] Figure 3 This is a schematic diagram illustrating the configuration of a video / image decoding device to which the implementation methods described in this document can be applied.
[0023] Figure 4 An example is given of motion represented by an affine motion model.
[0024] Figure 5 An affine motion model using motion vectors with 3 control points is illustrated.
[0025] Figure 6 An affine motion model using motion vectors with two control points is illustrated.
[0026] Figure 7 An example is given of a method for deriving motion vectors based on sub-blocks using an affine motion model.
[0027] Figure 8 This is a flowchart illustrating an affine motion prediction method according to one embodiment of the present disclosure.
[0028] Figure 9An embodiment of the present disclosure illustrates a method for deriving a motion vector predictor at a control point.
[0029] Figure 10 An example of a method for deriving motion vector predictors at control points according to one embodiment of the present disclosure is illustrated.
[0030] Figure 11 An example of an affine prediction performed when neighboring block A is selected as an affine merge candidate is shown.
[0031] Figure 12 The example illustrates adjacent blocks used to derive affine candidates for inheritance.
[0032] Figure 13 Spatial candidates for affine candidates are illustrated.
[0033] Figure 14 An example of constructing an affine MVP list is shown.
[0034] Figure 15 An example of deriving a candidate construct is shown.
[0035] Figure 16 An example of deriving a candidate construct is shown.
[0036] Figure 17 This example illustrates the positions of adjacent blocks scanned to derive the affine candidate of inheritance.
[0037] Figure 18 An example is shown where a four-parameter affine motion model is applied to the current block to derive a candidate construction.
[0038] Figure 19 An example of deriving a candidate construct is shown when a six-parameter affine motion model is applied to the current block.
[0039] Figure 20a and Figure 20b An implementation method for deriving affine candidates for inheritance is illustrated.
[0040] Figure 21 An example of a video coding method performed by an encoding device according to the present disclosure is illustrated.
[0041] Figure 22 An encoding device for performing a video encoding method according to the present disclosure is illustrated.
[0042] Figure 23 An example of a video decoding method performed by a decoding device according to the present disclosure is illustrated.
[0043] Figure 24An example of a decoding device for performing a video decoding method according to the present disclosure is shown.
[0044] Figure 25 An example of a content flow system structure applying an embodiment of this disclosure is provided. Detailed Implementation
[0045] This disclosure can be modified in various forms, and specific embodiments thereof will be described and illustrated in the accompanying drawings. However, the embodiments are not intended to limit this disclosure. The terminology used in the following description is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. Singular expressions include plural expressions, provided that they are clearly understood in different ways. Terms such as “comprising” and “having” are intended to indicate the presence of the features, quantities, steps, operations, elements, components, or combinations thereof used in the following description, and therefore it should be understood that the possibility of having or adding one or more different features, quantities, steps, operations, elements, components, or combinations thereof is not excluded.
[0046] On the other hand, the elements in the accompanying drawings described in this disclosure are drawn independently for the convenience of explaining different specific functions, and do not imply that these elements are embodied by independent hardware or independent software. For example, two or more elements may be combined to form a single element, or a single element may be divided into multiple elements. The implementation of combining and / or dividing elements is part of this disclosure and does not depart from its concept.
[0047] In the following, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Furthermore, throughout the drawings, similar reference numerals are used to indicate similar elements, and identical descriptions of similar elements will be omitted.
[0048] Figure 1 Examples of video / image coding systems to which this disclosure can be applied are illustrated.
[0049] Reference Figure 1 A video / image encoding system may include a first device (source device) and a second device (receiving device). The source device may send encoded video / image information or data to the receiving device in the form of a file or stream via a digital storage medium or network.
[0050] The source device may include a video source, an encoding device, and a transmitter. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display, and the display may be configured as a separate device or an external component.
[0051] Video sources can acquire video / images through processes that capture, synthesize, or generate video / images. Video sources may include video / image capture devices and / or video / image generation devices. Video / image capture devices may include, for example, one or more cameras, video / image archives including previously captured video / images, etc. Video / image generation devices may include, for example, computers, tablets, and smartphones, and can generate video / images (electronically). For example, virtual video / images can be generated by computers, etc. In this case, the video / image capture process can be replaced by a process that generates related data.
[0052] Encoding devices can encode input video / images. They can perform a series of processes such as prediction, transformation, and quantization to achieve compression and encoding efficiency. The encoded data (encoded video / image information) can be output as a bitstream.
[0053] The transmitter can send encoded images / image information or data, output in bitstream form, to the receiver of the receiving device in the form of a file or stream via digital storage media or a network. Digital storage media can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter can include elements for generating media files according to a predetermined file format and may include elements for transmission via a broadcast / communication network. The receiver can receive / extract the bitstream and send the received bitstream to a decoding device.
[0054] Decoding devices can decode video / images by performing a series of processes such as inverse quantization, inverse transform, and prediction, which correspond to the operations of encoding devices.
[0055] The renderer can render decoded video / images. Rendered video / images can then be displayed on a monitor.
[0056] This document relates to video / image coding. For example, the methods / implementations disclosed in this document can be applied to methods disclosed in Multifunctional Video Coding (VVC), EVC (Essential Video Coding) standard, AOMedia Video 1 (AV1) standard, AVS2 (Audio Video 2) standard, or next-generation video / image coding standards (e.g., H.267, or H.268, etc.).
[0057] This document presents various implementations of video / image encoding, and unless otherwise noted, these implementations can be combined with each other.
[0058] In this document, video can refer to a series of images over a period of time. An image typically refers to a unit representing an image within a specific time region, while a slice / tile is a unit that makes up a part of an image during encoding. A slice / tile can include one or more Code Tree Units (CTUs). An image can include one or more slices / tiles. An image can include one or more groups of slices. A group of slices can include one or more slices. A brick can represent a rectangular area of CTU rows within a slice of an image. A slice can be divided into multiple bricks, each brick comprising one or more CTU rows within the slice. A slice that is not divided into multiple bricks can also be called a brick. A brick scan is a specific ordering of CTUs in which the CTUs in an image are consecutively ordered in the CTU raster scan of the bricks, where the bricks within a slice are consecutively arranged in the raster scan of the bricks of the slice, and the slices in an image are consecutively ordered in the raster scan of the slices of the image. A slice is a rectangular area of CTUs within a specific slice column and a specific slice row in an image. A slice column is a rectangular area of CTUs with a height equal to the height of the image and a width specified by the syntax element in the image parameter set. A slice row is a rectangular area of CTUs with a height specified by the syntax element in the image parameter set and a width equal to the width of the image. A slice scan refers to a specific order of CTUs that divide an image where the CTUs are ordered consecutively in a slice's CTU raster scan, and slices in an image are ordered consecutively in a slice's slice raster scan. A strip comprises an integer number of bricks of an image that can be uniquely contained within a single NAL unit. A strip can include multiple complete slices or a consecutive sequence of complete bricks of a single slice. Slice groups and stripes are used interchangeably in this document. For example, in this document, a slice group / slice group header may be referred to as a strip / strip header.
[0059] A pixel, or image pixel, can represent the smallest unit that makes up a picture (or image). Additionally, the term "sample" can be used as the counterpart to a pixel. A sample can typically represent a pixel or a pixel value, and can represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chrominance component.
[0060] A unit can represent the basic unit of image processing. A unit may include a specific region of an image and at least one of the information associated with that region. A unit may include a luminance block and two chrominance (e.g., cb, cr) blocks. In some cases, the term "unit" may be used interchangeably with terms such as "block" or "region". In general, an M×N block may include a set (or array) of samples (or sample arrays) or transform coefficients in M columns and N rows.
[0061] In this document, the terms " / " and "," should be interpreted as indicating "and / or". For example, the expression "A / B" can mean "A and / or B". Furthermore, "A, B" can mean "A and / or B". Additionally, "A / B / C" can mean "at least one of A, B, and / or C". Also, "A / B / C" can mean "at least one of A, B, and / or C".
[0062] Furthermore, in this document, the term "or" should be interpreted as indicating "and / or". For example, the expression "A or B" could include 1) only A, 2) only B, and / or 3) both A and B. In other words, the term "or" in this document should be interpreted as indicating "additionally or alternatively".
[0063] Figure 2 This is a schematic diagram illustrating the configuration of a video / image encoding apparatus to which the embodiments described in this document may be applied. In the following, a video encoding apparatus may include an image encoding apparatus.
[0064] Reference Figure 2 The encoding device 200 includes an image segmenter 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-frame predictor 221 and an intra-frame predictor 222. The residual processor 230 may include a transform 232, a quantizer 233, an inverse quantizer 234, and an inverse transform 235. The residual processor 230 may also include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstruction block generator. According to embodiments, the image segmenter 210, predictor 220, residual processor 230, entropy encoder 240, adder 250, and filter 260 may be constituted by at least one hardware component (e.g., an encoder chipset or a processor). Additionally, the memory 270 may include a decoded image buffer (DPB) or may be constituted by a digital storage medium. The hardware component may also include the memory 270 as an internal / external component.
[0065] Image segmenter 210 can segment an input image (or picture or frame) input to encoding device 200 into one or more processors. For example, a processor may be referred to as a coding unit (CU). In this case, coding units can be recursively segmented from coding tree units (CTUs) or maximum coding units (LCUs) according to a quadtree-binary-trinary tree (QTBTTT) structure. For example, a coding unit can be segmented into multiple deeper coding units based on a quadtree structure, a binary tree structure, and / or a ternary structure. In this case, for example, a quadtree structure can be applied first, followed by a binary tree structure and / or a ternary structure. Alternatively, a binary tree structure can be applied first. The encoding process according to this document can be performed based on the final coding unit that is no longer segmented. In this case, the maximum coding unit can be used as the final coding unit based on encoding efficiency according to image characteristics, or, if necessary, the coding unit can be recursively segmented into deeper coding units, and the coding unit with the optimal size can be used as the final coding unit. Here, the encoding process may include prediction, transformation, and reconstruction processes, which will be described later. As another example, the processor may also include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit can be separated or divided from the final encoding unit described above. The prediction unit may be a unit for predicting samples, and the transform unit may be a unit for deriving transform coefficients and / or a unit for deriving residual signals from transform coefficients.
[0066] In some cases, a unit can be used interchangeably with terms such as block or region. Generally, an M×N block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can typically represent a pixel or pixel value, and can represent only the pixel / pixel value of the luminance component, or only the pixel / pixel value of the chrominance component. A sample can be used as a term corresponding to a picture (or image) of pixels or pictographs.
[0067] In the encoding device 200, the predicted signal (prediction block, prediction sample array) output from the inter-frame predictor 221 or the intra-frame predictor 222 is subtracted from the input image signal (original block, original sample array) to generate a residual signal (residual block, residual sample array), and the generated residual signal is sent to the converter 232. In this case, as shown, the unit in the encoder 200 used to subtract the predicted signal (prediction block, prediction sample array) from the input image signal (original block, original sample array) can be called the subtractor 231. The predictor can perform prediction on the block to be processed (hereinafter referred to as the current block) and generate a prediction block that includes the prediction samples of the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction based on the current block or CU. As described later in the description of each prediction mode, the predictor can generate various information related to the prediction, such as prediction mode information, and send the generated information to the entropy encoder 240. The information about the prediction can be encoded in the entropy encoder 240 and output as a bitstream.
[0068] Intra-predictor 222 can predict the current block by referencing samples in the current image. Depending on the prediction mode, the referenced samples may be located near or far from the current block. In intra-prediction, the prediction mode can include multiple non-directional modes and multiple directional modes. Non-directional modes can include, for example, DC mode and planar mode. Depending on the level of detail in the prediction direction, the directional modes can include, for example, 33 or 65 directional prediction modes. However, this is just an example, and more or fewer directional prediction modes may be used depending on the settings. Intra-predictor 222 can determine the prediction mode to be applied to the current block by using the prediction modes applied to neighboring blocks.
[0069] Inter-frame predictor 221 can deduce the predicted block of the current block based on a reference block (reference sample array) specified by motion vectors on a reference image. Here, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation between motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference image indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, neighboring blocks may include spatially adjacent blocks existing in the current image and temporally adjacent blocks existing in the reference image. The reference image including the reference block and the reference image including the temporally adjacent block may be the same or different. The temporally adjacent block may be called a juxtaposed reference block, a co-located CU (colCU), etc., and the reference image including the temporally adjacent block may be called a juxtaposed image (colPic). For example, inter-frame predictor 221 can configure a motion information candidate list based on neighboring blocks and generate information indicating which candidate to use to deduce the motion vector and / or reference image index of the current block. Inter-frame prediction can be performed based on various prediction modes. For example, in skip mode and merge mode, the inter-frame predictor 221 can use motion information from neighboring blocks as motion information for the current block. In skip mode, unlike merge mode, residual signals may not be transmitted. In motion vector prediction (MVP) mode, motion vectors from neighboring blocks can be used as motion vector predictors, and the motion vector of the current block can be indicated by signaling the motion vector difference.
[0070] Predictor 220 can generate prediction signals based on various prediction methods described below. For example, the predictor can not only apply intra-frame prediction or inter-frame prediction to predict a block, but can also apply both intra-frame prediction and inter-frame prediction simultaneously. This can be called Inter-Frame Intra-Frame Combined Prediction (CIIP). Alternatively, the predictor can predict blocks based on an Intra-Frame Block Copy (IBC) prediction mode or a palette mode. IBC prediction mode or palette mode can be used for content image / video coding such as games, for example, Screen Content Coding (SCC). IBC essentially performs prediction in the current image, but can be performed similarly to inter-frame prediction because the reference block is derived in the current image. That is, IBC can use at least one of the inter-frame prediction techniques described herein. Palette mode can be considered as an example of intra-frame coding or intra-frame prediction. When applying palette mode, sample values within the image can be signaled based on information about the palette table and palette index.
[0071] The predicted signal generated by the predictor (including inter-frame predictor 221 and / or intra-frame predictor 222) can be used to generate a reconstructed signal or a residual signal. Transformer 232 can generate transform coefficients by applying transform techniques to the residual signal. For example, the transform technique can include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loève Transform (KLT), Graph-Based Transform (GBT), or Conditional Nonlinear Transform (CNT). Here, GBT refers to the transform obtained from a graph when the relationship information between pixels is represented by a graph. CNT refers to the transform generated based on the predicted signal generated using all previously reconstructed pixels. Furthermore, the transform processing can be applied to square pixel blocks of the same size, or it can be applied to blocks of variable size that are not square.
[0072] Quantizer 233 quantizes the transform coefficients and sends them to entropy encoder 240, which encodes the quantized signal (information about the quantized transform coefficients) and outputs a bitstream. This information about the quantized transform coefficients can be called residual information. Quantizer 233 can rearrange the block-type quantized transform coefficients into a one-dimensional vector form based on the coefficient scan order and generate information about the quantized transform coefficients based on this one-dimensional vector form. Entropy encoder 240 can perform various encoding methods, such as Golomb, Context Adaptive Variable Length Coding (CAVLC), and Context Adaptive Binary Arithmetic Coding (CABAC). Entropy encoder 240 can encode information required for video / image reconstruction other than the quantized transform coefficients (e.g., values of syntax elements) together or separately. Encoded information (e.g., encoded video / image information) can be sent or stored in units of NAL (Network Abstraction Layer) in the form of a bitstream. The video / image information may also include information about various parameter sets such as Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). Additionally, the video / image information may include general constraint information. In this document, information and / or syntax elements that send / signal notifications from the encoding device to the decoding device may be included in the video / image information. The video / image information may be encoded by the encoding process described above and included in a bitstream. The bitstream may be transmitted over a network or stored in a digital storage medium. The network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitter (not shown) that transmits the signal output from the entropy encoder 240 and / or a storage medium (not shown) that stores the signal may be included as internal / external components of the encoding device 200; alternatively, the transmitter may be included in the entropy encoder 240.
[0073] The quantization transform coefficients output from quantizer 233 can be used to generate a prediction signal. For example, the residual signal (residual block or residual sample) can be reconstructed by applying inverse quantization and inverse transform to the quantization transform coefficients using inverse quantizer 234 and inverse transformer 235. Adder 250 adds the reconstructed residual signal to the prediction signal output from inter-frame predictor 221 or intra-frame predictor 222 to generate a reconstructed signal (reconstructed image, reconstructed block, reconstructed sample array). If the block to be processed has no residual (such as when a skip mode is applied), the prediction block can be used as a reconstructed block. Adder 250 can be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current image, and can be used for inter-frame prediction of the next image by filtering as described below.
[0074] In addition, luminance mapping with chroma scaling (LMCS) can be applied during image encoding and / or reconstruction.
[0075] Filter 260 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 260 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image and store the modified reconstructed image in memory 270 (specifically, the DPB of memory 270). Various filtering methods may include, for example, deblocking filtering, sample adaptive offsetting, adaptive loop filtering, bilateral filtering, etc. Filter 260 can generate various filtering-related information and send the generated information to entropy encoder 240, as described later in the description of the various filtering methods. The filtering-related information can be encoded by entropy encoder 240 and output as a bitstream.
[0076] The modified reconstructed image sent to memory 270 can be used as a reference image in inter-frame predictor 221. When inter-frame prediction is applied through the encoding device, prediction mismatch between the encoding device 200 and the decoding device can be avoided, and encoding efficiency can be improved.
[0077] The DPB of memory 270 can store a modified reconstructed image used as a reference image in inter-frame predictor 221. Memory 270 can store motion information of blocks from which motion information in the current image is derived (or encoded) and / or motion information of reconstructed blocks in the image. The stored motion information can be sent to inter-frame predictor 221 and used as motion information for spatially adjacent blocks or temporally adjacent blocks. Memory 270 can store reconstructed samples of reconstructed blocks in the current image and can transmit these reconstructed samples to intra-frame predictor 222.
[0078] Figure 3 This is a schematic diagram illustrating the configuration of a video / image decoding device to which the implementation methods described in this document can be applied.
[0079] Reference Figure 3 The decoding device 300 may include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-frame predictor 332 and an intra-frame predictor 331. The residual processor 320 may include an inverse quantizer 321 and an inverse transformer 322. According to embodiments, the entropy decoder 310, residual processor 320, predictor 330, adder 340, and filter 350 may be constructed from hardware components (e.g., a decoder chipset or processor). Additionally, the memory 360 may include a decoded image buffer (DPB) or may be constructed from a digital storage medium. The hardware components may also include the memory 360 as an internal / external component.
[0080] When a bitstream including video / image information is input, the decoding device 300 can interact with... Figure 2 The processing of video / image information in the encoding device correspondingly reconstructs the image. For example, the decoding device 300 can deduce units / blocks based on block segmentation information obtained from the bitstream. The decoding device 300 can use a processor applied in the encoding device to perform decoding. Therefore, the decoding processor can be, for example, an encoding unit, and the encoding unit can be segmented from the encoding tree unit or the maximum encoding unit according to a quadtree structure, binary tree structure, and / or ternary tree structure. One or more transform units can be derived from the encoding unit. The reconstructed image signal decoded and output by the decoding device 300 can be reproduced by a reproduction device.
[0081] Decoding device 300 can receive data in bitstream form from... Figure 2The signal output by the encoding device can be decoded by the entropy decoder 310. For example, the entropy decoder 310 can parse the bitstream to derive the information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets such as Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). In addition, the video / image information may also include general constraint information. The decoding device can also decode the picture based on the information about the parameter sets and / or general constraint information. The signaling / receiving information and / or syntax elements described later herein can be decoded and obtained from the bitstream through the decoding process. For example, the entropy decoder 310 decodes the information in the bitstream based on encoding methods such as Exponential Golomb coding, CAVLC, or CABAC, and outputs the quantized values of the syntax elements and transform coefficients of the residuals required for image reconstruction. More specifically, the CABAC entropy decoding method can receive bins (bits) corresponding to each syntax element in the bitstream, determine the context model using information about the target syntax element, decoding information about the target block, or information about symbols / bins decoded in previous stages, and perform arithmetic decoding on the bin by predicting the occurrence probability of the bin based on the determined context model, generating symbols corresponding to the value of each syntax element. In this case, after determining the context model, the CABAC entropy decoding method can update the context model by using the information of the decoded symbols / bins for the context model of the next symbol / bin. The prediction-related information in the information decoded by the entropy decoder 310 can be provided to the predictors (inter-frame predictor 332 and intra-frame predictor 331), and the residual values (i.e., quantization transform coefficients and related parameter information) from which entropy decoding has been performed in the entropy decoder 310 can be input to the residual processor 320. The residual processor 320 can derive the residual signals (residual blocks, residual samples, residual sample arrays). Additionally, the filtering information in the information decoded by the entropy decoder 310 can be provided to the filter 350. Furthermore, a receiver (not shown) for receiving signals output from the encoding device may be further configured as an internal / external element of the decoding device 300, or the receiver may be a component of the entropy decoder 310. Additionally, the decoding device according to this document may be referred to as a video / image / picture decoding device, and the decoding device may be classified as an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoder 310, and the sample decoder may include at least one of an inverse quantizer 321, an inverse transformer 322, an adder 340, a filter 350, a memory 360, an inter-frame predictor 332, and an intra-frame predictor 331.
[0082] The dequantizer 321 can dequantize the quantized transform coefficients and output the transform coefficients. The dequantizer 321 can rearrange the quantized transform coefficients in the form of two-dimensional blocks. In this case, the rearrangement can be performed based on the coefficient scan order executed in the encoding device. The dequantizer 321 can perform dequantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information) and obtain the transform coefficients.
[0083] The inverse transformer 322 performs an inverse transformation on the transformation coefficients to obtain the residual signal (residual block, residual sample array).
[0084] The predictor can perform prediction on the current block and generate a prediction block that includes the prediction samples of the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction to the current block based on the prediction information output from the entropy decoder 310, and can determine the specific intra-frame / inter-frame prediction mode.
[0085] Predictor 320 can generate prediction signals based on various prediction methods described below. For example, the predictor can not only apply intra-frame prediction or inter-frame prediction to predict a block, but can also apply intra-frame prediction and inter-frame prediction simultaneously. This can be referred to as combined intra-frame and inter-frame prediction (CIIP). Alternatively, the predictor can predict blocks based on an intra-block copy (IBC) prediction mode or a palette mode. IBC prediction mode or palette mode can be used for content image / video coding such as in games, for example, screen content coding (SCC). IBC essentially performs prediction in the current frame, but can be performed similarly to inter-frame prediction because a reference block is derived in the current frame. That is, IBC can use at least one of the inter-frame prediction techniques described in this document. Palette mode can be considered an example of intra-frame coding or intra-frame prediction. When applying palette mode, sample values within the frame can be signaled based on information about the palette table and palette index.
[0086] Intra-predictor 331 can predict the current block by referencing samples in the current image. Depending on the prediction mode, the referenced samples may be located near or far from the current block. In intra-prediction, the prediction mode can include multiple non-directional modes and multiple directional modes. Intra-predictor 331 can determine the prediction mode applied to the current block by using prediction modes applied to neighboring blocks.
[0087] Inter-frame predictor 332 can deduce the predicted block for the current block based on a reference block (reference sample array) specified by motion vectors on a reference image. In this case, to reduce the amount of motion information transmitted in the inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation between motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference image indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, neighboring blocks may include spatially adjacent blocks existing in the current image and temporally adjacent blocks existing in the reference image. For example, inter-frame predictor 332 can configure a motion information candidate list based on neighboring blocks and deduce the motion vector and / or reference image index for the current block based on received candidate selection information. Inter-frame prediction can be performed based on various prediction modes, and the information about the prediction may include information indicating the mode of inter-frame prediction for the current block.
[0088] Adder 340 can generate a reconstruction signal (reconstructed image, reconstruction block, reconstruction sample array) by adding the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the predictor (including inter-frame predictor 332 and / or intra-frame predictor 331). If the block to be processed has no residual (e.g., when a skip mode is applied), the prediction block can be used as the reconstruction block.
[0089] Adder 340 can be called a reconstructor or reconstruction block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current image, through filtering output as described below, or it can be used for inter-frame prediction of the next image.
[0090] Furthermore, Luminance Mapping with Chroma Scaling (LMCS) can be applied in image decoding processing.
[0091] Filter 350 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 350 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image and store the modified reconstructed image in memory 360 (specifically, the DPB of memory 360). Various filtering methods may include, for example, deblocking filtering, adaptive sample shifting, adaptive loop filtering, bilateral filtering, etc.
[0092] The (modified) reconstructed image stored in the DPB of memory 360 can be used as a reference image in inter-frame predictor 332. Memory 360 can store motion information of blocks from which motion information in the current image is derived (or decoded) and / or motion information of reconstructed blocks in the image. The stored motion information can be sent to inter-frame predictor 260 to be used as motion information of spatially adjacent blocks or temporally adjacent blocks. Memory 360 can store reconstructed samples of reconstructed blocks in the current image and can transmit the reconstructed samples to intra-frame predictor 331.
[0093] In this disclosure, the embodiments described in the filter 260, inter-frame predictor 221, and intra-frame predictor 222 of the encoding device 200 can be the same as, or applied to, the filter 350, inter-frame predictor 332, and intra-frame predictor 331 of the decoding device 300, respectively. The same content can also be applied to the inter-frame predictor 332 and intra-frame predictor 331.
[0094] Furthermore, regarding inter-frame prediction, methods that consider image distortion have been proposed. More specifically, an affine motion model has been proposed that effectively derives the motion vectors of sub-blocks or sample points of the current block and improves the accuracy of inter-frame prediction, independent of deformations caused by image rotation, scaling, or reduction. In other words, an affine motion model for deriving the motion vectors of sub-blocks or sample points of the current block has been proposed. Prediction using an affine motion model can be called affine inter-frame prediction or affine motion prediction.
[0095] For example, affine inter-frame prediction using an affine motion model can effectively represent the four motions described below, namely, the four deformations.
[0096] Figure 4 This illustrates motion represented by an affine motion model. (See reference...) Figure 4 Motion that can be represented by an affine motion model can include translational motion, scaling motion, rotational motion, and shearing motion. That is, it can be represented as... Figure 3 Effectively represented as shown Figure 4 The image shown is a scaling motion where a portion of the image is scaled over time, a rotational motion where a portion of the image is rotated over time, a shearing motion where a portion of the image is deformed into a parallelogram over time, and a translational motion where a portion of the image is moved planarly over time.
[0097] Encoding / decoding devices can predict the distortion shape of an image based on the motion vector at the control point (CP) of the current block through affine inter-frame prediction, which can improve image compression performance by increasing prediction accuracy. Furthermore, since the motion vector of at least one control point of the current block can be derived using the motion vectors of neighboring blocks, the data burden regarding additional information can be reduced, and inter-frame prediction efficiency can be significantly improved.
[0098] As an example of affine inter-frame prediction, motion information at three control points (i.e., three reference points) may be required.
[0099] Figure 5 An affine motion model using motion vectors with three control points is illustrated.
[0100] When the top-left sample position in the current block 500 is (0, 0), the sample positions (0, 0), (w, 0), and (0, h) can be defined as control points, such as... Figure 5 As shown below, the control point at sample position (0, 0) can be denoted as CP0, the control point at sample position (w, 0) can be denoted as CP1, and the control point at sample position (0, h) can be denoted as CP2.
[0101] The formula for the affine motion model can be derived using the aforementioned control points and their corresponding motion vectors. The formula for the affine motion model can be expressed as follows.
[0102] [Formula 1]
[0103]
[0104] Here, w represents the width of the current block 500, h represents the height of the current block 500, and v 0x and v 0y Let v represent the x-component and y-component of the motion vector of CP0, respectively. 1x and v 1y Let x and y represent the motion vector of CP1, respectively, and v 2x and v 2y These represent the x and y components of the motion vector of CP2, respectively. Additionally, x represents the x-component of the target sample's position in the current block 500, y represents the y-component of the target sample's position in the current block 500, and v... x Let v represent the x-component of the motion vector of the target sample in the current block 500, and v y This represents the y-component of the motion vector of the target sample in the current block 500.
[0105] Since the motion vectors of CP0, CP1, and CP2 are known, the motion vector based on the sample position in the current block can be derived using Equation 1. That is, according to the affine motion model, the motion vector at the control point v0(v 0x v 0y v1(v) 1x v 1y ) and v2(v 2x v 2y The motion vector of a target sample can be derived based on the ratio of its coordinates (x, y) to the distances between the three control points, allowing scaling to be applied to the target sample's position. In other words, based on the affine motion model, the motion vector of each sample in the current block can be derived from the motion vectors of the control points. Furthermore, the set of motion vectors derived from the affine motion model for the samples in the current block can be called the affine motion vector field (MVF).
[0106] Furthermore, the six parameters of Equation 1 can be represented by a, b, c, d, e, and f as shown in the following equation, and the formula for the affine motion model represented by the six parameters can be as follows.
[0107] [Equation 2]
[0108]
[0109] Here, w represents the width of the current block 500, h represents the height of the current block 500, and v 0x and v 0y The x and y components of the motion vector of CP0, v 1x and v 1y Let x and y represent the motion vector of CP1, respectively, and v 2x and v 2y These represent the x and y components of the motion vector of CP2, respectively. Additionally, x represents the x-component of the target sample's position in the current block 500, y represents the y-component of the target sample's position in the current block 500, and v... x Let v represent the x-component of the motion vector of the target sample in the current block 500, and v y This represents the y-component of the motion vector of the target sample in the current block 500.
[0110] A affine motion model or affine inter-frame prediction using six parameters can be called a 6-parameter affine motion model or AF6.
[0111] Furthermore, as an example of affine inter-frame prediction, motion information at two control points (i.e., two reference points) may be required.
[0112] Figure 6An affine motion model using motion vectors with two control points is illustrated. An affine motion model using two control points can represent three types of motion: translation, scaling, and rotation. An affine motion model representing these three types of motion can be called a similar affine motion model or a simplified affine motion model.
[0113] When the top-left sample position in the current block 600 is (0, 0), the sample positions (0, 0) and (w, 0) can be defined as control points, such as... Figure 6 As shown below, the control point at sample position (0, 0) can be denoted as CP0, and the control point at sample position (w, 0) can be denoted as CP1.
[0114] The formula for the affine motion model can be derived using the aforementioned control points and their corresponding motion vectors. The formula for the affine motion model can be expressed as follows.
[0115] [Formula 3]
[0116]
[0117] Here, w represents the width of the current block 600, and v 0x and v 0y Let x and y represent the motion vector of CP0, respectively, and v 1x and v 1y This represents the x and y components of the motion vector of CP1. Additionally, x represents the x-component of the target sample's position in the current block 600, y represents the y-component of the target sample's position in the current block 600, and v... x Let v represent the x-component of the motion vector of the target sample in the current block 600, and v y This represents the y-component of the motion vector of the target sample in the current block 600.
[0118] Furthermore, the four parameters of Equation 3 can be represented by a, b, c, and d in the following formula, and the formula for the affine motion model represented by the four parameters can be as follows.
[0119] [Formula 4]
[0120]
[0121] Here, w represents the width of the current block 600, and v 0x and v 0y Let x and y represent the motion vector of CP0, respectively, and v 1x and v 1y These represent the x and y components of the motion vector of CP1, respectively. Additionally, x represents the x-component of the target sample's position in the current block 600, y represents the y-component of the target sample's position in the current block 600, and v... xLet v represent the x-component of the motion vector of the target sample in the current block 600, and v y Let y be the y-component of the motion vector of the target sample in the current block 600. An affine motion model using two control points can be represented by four parameters a, b, c, and d as shown in Equation 4, and therefore, an affine motion model or affine inter-frame prediction using four parameters can be called a 4-parameter affine motion model or AF4. That is, based on the affine motion model, the motion vector of each sample in the current block can be derived from the motion vectors of the control points. Furthermore, the set of motion vectors of the samples in the current block derived from the affine motion model can be called the affine motion vector field (MVF).
[0122] Furthermore, as mentioned above, the motion vector of the sample unit can be derived using an affine motion model, thus significantly improving the accuracy of inter-frame prediction. However, this could greatly increase the complexity of motion compensation processing.
[0123] Therefore, restrictions can be imposed to derive the motion vectors of the sub-block units of the current block, rather than the motion vectors of the sample units.
[0124] Figure 7 An example is given of a method for deriving motion vectors based on sub-blocks using an affine motion model. Figure 7 This example illustrates a case where the current block is 16×16 and motion vectors are derived in units of 4×4 sub-blocks. Sub-blocks can be set to various sizes. For example, when the sub-blocks are set to an n×n size (n is a positive integer, e.g., n is 4), motion vectors can be derived in units of n×n sub-blocks within the current block based on an affine motion model, and various methods for deriving the motion vectors representing each sub-block can be applied.
[0125] For example, refer to Figure 7 The motion vector of each sub-block can be derived using the center or lower right sample position of each sub-block as representative coordinates. Here, the lower right center position can indicate the position of the lower right sample among the four samples located at the center of the sub-block. For example, when n is odd, a sample can be located at the center of the sub-block, and in this case, the center sample position can be used to derive the motion vector of the sub-block. However, when n is even, the four samples can be positioned adjacent at the center of the sub-block, and in this case, the lower right sample position can be used to derive the motion vector. For example, refer to... Figure 7 The representative coordinates of each sub-block can be derived as (2, 2), (6, 2), (10, 2), ..., (14, 14), and the encoding / decoding device can derive the motion vector of each sub-block by substituting each representative coordinate of the sub-block into Equation 1 or Equation 3 above. The motion vector of the sub-block in the current block derived by the affine motion model can be called the affine MVF.
[0126] Furthermore, as an example, the size of the sub-blocks in the current block can be derived based on the following formula.
[0127] [Formula 5]
[0128]
[0129] Here, M represents the width of the sub-block, and N represents the height of the sub-block. Additionally, v 0x and v 0y This represents the x and y components of CPMV0 in the current block, v 1x and v 1y This represents the x and y components of the current block's CPMV1, w represents the width of the current block, h represents the height of the current block, and MvPre represents the motion vector fractional precision. For example, the motion vector fractional precision can be set to 1 / 16.
[0130] Furthermore, in inter-frame prediction (i.e., affine motion prediction) using the aforementioned affine motion model, there can be an affine merging mode AF_MERGE and an affine inter-frame mode AF_INTER. Here, the affine inter-frame mode can be referred to as the affine MVP mode AF_MVP.
[0131] The affine merging mode is similar to existing merging modes in that it does not transmit the motion vector MVD of the control points. That is, similar to existing skip / merge modes, the affine merging mode can refer to an encoding / decoding method that performs prediction by deriving the CPMV from the adjacent blocks of the current block for each of two or three control points.
[0132] For example, when the AF_MRG mode is applied to the current block, the motion vectors of CP0 and CP1 (i.e., CPMV0 and CPMV1) can be derived from neighboring blocks in which the affine mode has already been applied. In other words, the CPMV0 and CPMV1 of neighboring blocks in which the affine mode has already been applied can be derived as merge candidates, or can be derived as the CPMV0 and CPMV1 of the current block based on merge candidates. The affine motion model can be derived based on the CPMV0 and CPMV1 of neighboring blocks represented by merge candidates, and the CPMV0 and CPMV1 of the current block can be derived based on the affine motion model.
[0133] Affine inter-frame mode can represent an inter-frame prediction that derives a motion vector predictor (MVP) of the motion vectors of control points, derives the motion vectors of control points based on the received motion vector difference (MVD) and MVP, and derives the affine MVF of the current block based on the motion vectors of the control points; and performs prediction based on the affine MVF. Here, the motion vector of the control point can be referred to as the control point motion vector (CPMV), the MVP of the control point can be referred to as the control point motion vector predictor (CPMVP), and the MVD of the control point can be referred to as the control point motion vector difference (CPMVD). More specifically, for example, the encoding device can derive the control point motion vector predictors (CPMVP) and control point motion vectors (CPMV) of CP0 and CP1 (or CP0, CP1, and CP2 respectively, and transmit or store information about the CPMVP and / or representing the difference CPMVD between the CPMVP and CPMV.
[0134] Here, if the affine inter-frame mode is applied to the current block, the encoding / decoding device can construct an affine MVP candidate list based on the adjacent blocks of the current block, where the affine MVP candidates can be called CPMVP pair candidates, and the affine MVP candidate list can be called the CPMVP candidate list.
[0135] In addition, each affine MVP candidate can represent a combination of CP0 and CP1 CPMVPs in a four-parameter affine motion model, and a combination of CP0, CP1 and CP2 CPMVPs in a six-parameter affine motion model.
[0136] Figure 8 This is a flowchart illustrating an affine motion prediction method according to one embodiment of the present disclosure.
[0137] Reference Figure 8 The affine motion prediction method can be mainly described as follows. Once the affine motion prediction method starts, the CPMV pair is first obtained in S800. Here, if a four-parameter affine model is used, the CPMV pair can include CPMV0 and CPMV1.
[0138] Subsequently, in S810, affine motion compensation can be performed based on CPMV pairs, after which affine motion prediction can be terminated.
[0139] Alternatively, two affine prediction modes can be defined to determine CPMV0 and CPMV1. These two affine prediction modes can include an affine inter-frame mode and an affine merging mode. The affine inter-frame mode can use signaling information about the motion vector difference (MVD) between the two motion vectors of CPMV0 and CPMV1 to clearly determine CPMV0 and CPMV1. On the other hand, the affine merging mode can derive the CPMV pair without needing to signal MVD information.
[0140] In other words, the affine merging pattern can derive the CPMV of the current block by using the CPMV of the adjacent blocks encoded in the affine pattern, and if the motion vector is determined by the sub-block unit, the affine merging pattern can be called the sub-block merging pattern.
[0141] In affine merging mode, the encoding device can signal the decoding device the indices of neighboring blocks encoded in affine mode, used to derive the CPMV of the current block, and can also signal the difference between the CPMV of neighboring blocks and the CPMV of the current block. Here, affine merging mode can construct an affine merging candidate list based on neighboring blocks, where the indices of neighboring blocks indicate the neighboring blocks within the affine merging candidate list to be used to derive the CPMV of the current block. The affine merging candidate list can also be called a sub-block merging candidate list.
[0142] Affine inter-frame mode can also be called affine MVP mode. In affine MVP mode, the CPMV of the current block can be derived based on the Control Point Motion Vector Predictor (CPMVP) and the Control Point Motion Vector Difference (CPMVD). In other words, the encoding device can determine the CPMVP of the current block's CPMV, derive the CPMVD as the difference between the current block's CPMV and CPMVP, and signal the decoding device with information about the CPMVP and CPMVD. Here, affine MVP mode can construct an affine MVP candidate list based on neighboring blocks, and the information about the CPMVP can indicate which neighboring blocks to use to derive the CPMVP of the current block's CPMV from the affine MVP candidate list. The affine MVP candidate list can also be called the Control Point Motion Vector Predictor candidate list.
[0143] For example, when applying the affine inter-frame mode of the six-parameter affine motion model, the current block can be encoded as follows.
[0144] Figure 9 A method for deriving motion vector predictors at control points according to one embodiment of the present disclosure is illustrated.
[0145] Reference Figure 9 The motion vector of CP0 in the current block can be represented by v0, the motion vector of CP1 can be represented by v1, the motion vector of the control point at the lower left sample position can be represented by v2, and the motion vector of CP2 can be represented by v3. In other words, v0 can represent the CPMVP of CP0, v1 can represent the CPMVP of CP1, and v2 can represent the CPMVP of CP2.
[0146] Affine MVP candidates can be a combination of CP0's CPMVP candidates, CP1's CPMVP candidates, and CP2's CPMVP candidates.
[0147] For example, the affine MVP candidate can be derived as follows.
[0148] More specifically, up to 12 CPMVP candidate combinations can be identified, as shown in the following formula.
[0149] [Formula 6]
[0150] {(v0, v1, u2)|v0={v A v B v C}, v1={v D v E}, v2={v F v G}}
[0151] Here, v A v represents the motion vector of adjacent block A. B v represents the motion vector of adjacent block B. C v represents the motion vector of adjacent block C. D v represents the motion vector of adjacent block D. E v represents the motion vector of adjacent block E. F Let v represent the motion vector of the adjacent block F, and v G This represents the motion vector of the adjacent block G.
[0152] Additionally, adjacent block A can represent the adjacent block located to the upper left of the current block's upper left sample position, adjacent block B can represent the adjacent block located above the current block's upper left sample position, and adjacent block C can represent the adjacent block located to the left of the current block's upper left sample position. Additionally, adjacent block D can represent the adjacent block located above the current block's upper right sample position, and adjacent block E can represent the adjacent block located to the upper right of the current block's upper right sample position. Furthermore, adjacent block F can represent the adjacent block located to the left of the current block's lower left sample position, and adjacent block G can represent the adjacent block located to the lower left of the current block's lower left sample position.
[0153] In other words, referring to Equation 6 above, the CPMVP candidates for CP0 can include the motion vector v of the adjacent block A. A The motion vector v of adjacent block B B and / or the motion vector v of adjacent block C C ;CPMVP candidates for CP1 may include the motion vector v of adjacent block D. D and / or the motion vector v of adjacent block E E; CPMVP candidates for CP2 may include the motion vector v of the adjacent block F. F and / or the motion vector v of adjacent block G G .
[0154] In other words, the CPMVP v0 of CP0 can be derived based on at least one motion vector of neighboring blocks A, B, and C relative to the top-left sample position. Here, neighboring block A can represent the block located to the upper left of the current block's top-left sample position, neighboring block B can represent the block located above the current block's top-left sample position, and neighboring block C can represent the block located to the left of the current block's top-left sample position.
[0155] Based on the motion vectors of adjacent blocks, up to 12 CPMVP candidate combinations can be derived, including CPMVP candidates for CP0, CPMVP candidates for CP1, and CPMVP candidates for CP2.
[0156] Then, the derived CPMVP candidate combinations are arranged in ascending order of DV, and the first two CPMVP candidate combinations can be derived as affine MVP candidates.
[0157] The DV of CPMVP candidate combinations is derived using the following formula.
[0158] [Formula 7]
[0159] DV = |(v 1x -v 0x )*h-(v2 y -v0 y )*w|+|(v1 y -v0 y )*h+(v2 x -v0 x )*w|
[0160] The encoding device then determines the CPMV of each affine MVP candidate, compares the rate distortion (RD) costs between CPMVs, and selects the affine MVP candidate with the lowest RD cost as the best affine MVP candidate for the current block. The encoding device can encode and signal the index and CPMVD indicating the best candidate.
[0161] Alternatively, for example, if an affine merging mode is applied, the current block can be encoded as described below.
[0162] Figure 10 A method for deriving motion vector predictors at control points according to one embodiment of the present disclosure is illustrated.
[0163] based on Figure 10The adjacent blocks shown can be used to construct an affine merge candidate list for the current block. Adjacent blocks can include adjacent block A, adjacent block B, adjacent block C, adjacent block D, and adjacent block E. Adjacent block A can represent the left adjacent block of the current block, adjacent block B can represent the top adjacent block of the current block, adjacent block C can represent the top right adjacent block of the current block, adjacent block D can represent the bottom left adjacent block of the current block, and adjacent block E can represent the top left adjacent block of the current block.
[0164] For example, when the size of the current block is W×H, the x-component of the top-left sample position of the current block is 0, and its y-component is 0, the left adjacent block can be the block that includes the sample at coordinate (-1, H-1), the top adjacent block can be the block that includes the sample at coordinate (W-1, -1), the top-right adjacent block can be the block that includes the sample at coordinate (W, -1), the bottom-left adjacent block can be the block that includes the sample at coordinate (-1, H), and the top-left adjacent block can be the block that includes the sample at coordinate (-1, -1).
[0165] More specifically, for example, the encoding device can scan the neighboring blocks A, B, C, D, and E of the current block in a specific scan order; the neighboring block that is first encoded in the affine prediction mode according to the scan order is determined as a candidate block for the affine merging mode, i.e., an affine merging candidate. In other words, a specific scan order can be performed in the order of neighboring blocks A, B, C, D, and E.
[0166] Subsequently, the encoding device can determine the affine motion model of the current block by using the CPMV of the determined candidate blocks, determine the CPMV of the current block based on the affine motion model, and determine the affine MVF of the current block based on the CPMV.
[0167] As an example, if neighboring block A is identified as a candidate block for the current block, encoding can be performed as described below.
[0168] Figure 11 An example of an affine prediction performed when neighboring block A is selected as an affine merge candidate is shown.
[0169] Reference Figure 11 The encoding device can identify neighboring blocks A of the current block as candidate blocks and derive the affine motion model of the current block based on the CPMV (v2 and v3) of the neighboring blocks. Then, the encoding device can determine the CPMV (v0 and v1) of the current block based on the affine motion model. The encoding device can then determine the affine MVF based on the CPMV (v0 and v1) of the current block and perform the encoding process for the current block based on the affine MVF.
[0170] Furthermore, in relation to affine frame prediction, inherited affine candidates and constructed affine candidates are being considered as a means of constructing a list of affine MVP candidates.
[0171] Here, the affine candidate of inheritance can be described as follows.
[0172] For example, if the neighboring blocks of the current block are affine blocks, and the reference image of the current block is the same as that of the neighboring blocks, then the affine MVP pair of the current block can be determined from the affine motion models of the neighboring blocks. Here, an affine block can represent a block that has already been applied with affine inter-frame prediction. Inherited affine candidates can represent CPMVPs (e.g., affine MVP pairs) derived from the affine motion models of the neighboring blocks.
[0173] More specifically, as an example, the affine candidates for inheritance can be derived as follows.
[0174] Figure 12 The example illustrates adjacent blocks used to derive affine candidates for inheritance.
[0175] Reference Figure 12 The adjacent blocks of the current block can include the left adjacent block A0, the lower left adjacent block A1, the upper adjacent block B0, the upper right adjacent block B1, and the upper left adjacent block B2.
[0176] For example, when the size of the current block is W×H, and the x-component of the top-left sample position of the current block is 0 and its y-component is 0, the left adjacent block can be the block that includes the sample at coordinate (-1, H-1), the top adjacent block can be the block that includes the sample at coordinate (W-1, -1), the top-right adjacent block can be the block that includes the sample at coordinate (W, -1), the bottom-left adjacent block can be the block that includes the sample at coordinate (-1, H), and the top-left adjacent block can be the block that includes the sample at coordinate (-1, -1).
[0177] The encoding / decoding device can sequentially examine adjacent blocks A0, A1, B0, B1, and B2. If the adjacent blocks are encoded according to an affine motion model, and the reference image of the current block is the same as that of the adjacent blocks, then two or three CPMVs of the current block can be derived based on the affine motion models of the adjacent blocks. The CPMVs can be derived as affine MVP candidates for the current block. Affine MVP candidates can represent inherited affine candidates.
[0178] As an example, up to two affine candidates for inheritance can be derived based on adjacent blocks.
[0179] For example, an encoding / decoding device can derive a first affine MVP candidate for the current block based on the first block among its neighboring blocks. Here, the first block can be encoded according to an affine motion model, and the reference image of the first block can be the same as the reference image of the current block. In other words, the first block can be the block that is first identified as satisfying a condition from its neighboring blocks, checked according to a specific order. This condition can be encoded according to an affine motion model, and the reference image of this block can be the same as the reference image of the current block.
[0180] Subsequently, the encoding / decoding device can derive a second affine MVP candidate based on the second block among the adjacent blocks. Here, the second block can be encoded according to an affine motion model, and the reference image of the second block can be the same as the reference image of the current block. In other words, the second block can be the second block among the adjacent blocks that is confirmed to meet the conditions according to a specific order. These conditions can be encoded according to an affine motion model, and the reference image of this block can be the same as the reference image of the current block.
[0181] Furthermore, for example, when the number of available affine candidates for inheritance is less than 2 (i.e., when the number of derived affine candidates for inheritance is less than 2), constructed affine candidates can be considered. Constructed affine candidates can be derived as follows.
[0182] Figure 13 The spatial candidate of the constructed affine candidate is illustrated.
[0183] like Figure 13 As shown, the motion vectors of the current block's adjacent blocks can be divided into three groups. (Refer to...) Figure 13 Adjacent blocks can include adjacent block A, adjacent block B, adjacent block C, adjacent block D, adjacent block E, adjacent block F, and adjacent block G.
[0184] Adjacent block A can represent the block to the upper left of the current block's top-left sample position; adjacent block B can represent the block above the current block's top-left sample position; and adjacent block C can represent the block to the left of the current block's top-left sample position. Additionally, adjacent block D can represent the block above the current block's top-right sample position, and adjacent block E can represent the block to the upper right of the current block's top-right sample position. Furthermore, adjacent block F can represent the block to the left of the current block's bottom-left sample position, and adjacent block G can represent the block to the lower left of the current block's bottom-left sample position.
[0185] For example, the three groups may include S0, S1, and S2, which can be derived as shown in the table below.
[0186] [Table 1]
[0187]
[0188] Here, mv A mv represents the motion vector of adjacent block A. B mv represents the motion vector of adjacent block B. C The motion vector of adjacent block C is mv D mv represents the motion vector of adjacent block D. E mv represents the motion vector of adjacent block E. F Let mv represent the motion vector of the adjacent block F, and mv G S represents the motion vector of adjacent block G. S0 can indicate the first group, S1 can indicate the second group, and S2 can indicate the third group.
[0189] The encoding / decoding device can derive mv0 from S0, mv1 from S1, mv2 from S2, and affine MVP candidates including mv0, mv1, and mv2. Affine MVP candidates can indicate the constructed affine candidates. Furthermore, mv0 can be a CPMVP candidate for CP0, mv1 can be a CPMVP candidate for CP1, and mv2 can be a CPMVP candidate for CP2.
[0190] Here, the reference image of mv0 can be the same as the reference image of the current block. In other words, mv0 can be a motion vector that is first identified as satisfying a condition by checking the motion vectors within S0. This condition allows the reference image of the motion vector to be the same as the reference image of the current block. A specific order can be used to check the motion vectors within S0 in the order of neighboring block A, neighboring block B, and neighboring block C. Furthermore, the checking order can be performed differently than described above, and is not limited to the examples above.
[0191] Furthermore, the reference image of mv1 can be the same as the reference image of the current block. In other words, mv1 can be a motion vector that is first identified as satisfying a condition from checking motion vectors within S1. This condition allows the reference image of the motion vector to be the same as the reference image of the current block. A specific order can be used to check motion vectors within S1 in the order of adjacent blocks D and E. Additionally, the checking order can be performed differently than described above, and is not limited to the examples above.
[0192] Furthermore, the reference image of mv2 can be the same as the reference image of the current block. In other words, mv2 can be a motion vector that is first identified as satisfying a condition by checking the motion vectors within S2. This condition allows the reference image of the motion vector to be the same as the reference image of the current block. A specific order can be used to check the motion vectors within S2 in the order of adjacent blocks F and G. However, the checking order can be performed differently than described above, and is not limited to the examples above.
[0193] Furthermore, mv2 can be derived by the following formula only when mv0 and mv1 are available, that is, only when mv0 and mv1 are derived.
[0194] [Formula 8]
[0195]
[0196] Here, This represents the x component of mv2. This represents the y-component of mv2. This represents the x-component of mv0. This represents the y-component of mv0. Let x represent the x component of mv1, and This represents the y-component of mv1. Additionally, w represents the width of the current block, and h represents the height of the current block.
[0197] Furthermore, when only mv0 and mv2 are derived, mv1 can be derived using the following formula.
[0198] [Formula 9]
[0199]
[0200] Here, This represents the x-component of mv1. This represents the y-component of mv1. This represents the x-component of mv0. This represents the y-component of mv0. Let x represent the x component of mv2, and This represents the y-component of mv2. Additionally, w represents the width of the current block, and h represents the height of the current block.
[0201] Furthermore, when the number of available affine candidates for inheritance and / or the number of available affine candidates for construction is less than two, the existing HEVC standard's AMVP processing can be applied to construct the affine MVP list. In other words, when the number of available affine candidates for inheritance and / or the number of available affine candidates for construction is less than two, the processing used to construct MVP candidates specified in the existing HEVC standard can be performed.
[0202] Furthermore, a flowchart for the implementation of constructing an affine MVP list can be described as follows.
[0203] Figure 14 An example of constructing an affine MVP list is shown.
[0204] Reference Figure 14 In S1400, the encoding / decoding device can add inherited candidates to the affine MVP list of the current block. Inherited candidates can represent the affine candidates mentioned above.
[0205] More specifically, in S1405, the encoding / decoding device can derive up to two inherited affine candidates from the neighboring blocks of the current block. Here, the neighboring blocks may include the left neighboring block A0 of the current block, the lower left neighboring block A1 of the current block, the upper neighboring block B0 of the current block, the upper right neighboring block B1 of the current block, and the upper left neighboring block B2 of the current block.
[0206] For example, an encoding / decoding device can derive a first affine MVP candidate for the current block based on the first block among its neighboring blocks. Here, the first block can be encoded according to an affine motion model, and the reference image of the first block can be the same as the reference image of the current block. In other words, the first block can be the block that is first identified as satisfying the conditions by checking neighboring blocks in a specific order. The conditions can be encoded according to an affine motion model, and the reference image of that block can be the same as the reference image of the current block.
[0207] Subsequently, the encoding / decoding device can derive a second affine MVP candidate based on the second block among the adjacent blocks. Here, the second block can be encoded according to an affine motion model, and the reference image of the second block can be the same as the reference image of the current block. In other words, the second block can be the second block confirmed to meet the conditions after checking adjacent blocks in a specific order. The conditions can be encoded according to an affine motion model, and the reference image of this block can be the same as the reference image of the current block.
[0208] Furthermore, a specific order can be used to check adjacent blocks in the following order: left adjacent block A0, bottom left adjacent block A1, top adjacent block B0, top right adjacent block B1, and top left adjacent block B2. Additionally, the checking order can be different from the above, and is not limited to the examples shown.
[0209] In S1410, the encoding / decoding device can add the constructed candidate to the affine MVP list of the current block. The constructed candidate can represent the affine candidate of the above construction. The constructed candidate can also be called the affine MVP candidate of the construction. If the number of available successor candidates is less than 2, the encoding / decoding device can add the constructed candidate to the affine MVP list of the current block. For example, the encoding / decoding device can derive an affine candidate of the construction.
[0210] Furthermore, the method used to derive the affine candidates for the construction can differ depending on whether the affine motion model applied to the current block is a six-parameter affine motion model or a four-parameter affine motion model. A detailed description of how to derive the candidates for the construction will be provided later.
[0211] In S1420, the encoder / decoder can add an HEVC AMVP candidate to the affine MVP list of the current block. If the number of available inheritance candidates and / or construction candidates is less than two, the encoder / decoder can add the HEVC AMVP candidate to the affine MVP list of the current block. In other words, when the number of available inheritance candidates and / or construction candidates is less than two, the encoder / decoder can perform the processing for constructing the MVP candidate specified in the existing HEVC standard.
[0212] In addition, the method for deriving the candidate can be performed as follows.
[0213] For example, if the affine motion model applied to the current block is a six-parameter affine motion model, then it can be like this: Figure 15 The candidate construct is derived as shown in the implementation.
[0214] Figure 15 An example of deriving a candidate construct is shown.
[0215] Reference Figure 15 In S1500, the encoding / decoding device can check mv0, mv1, and mv2 for the current block. In other words, the encoding / decoding device can determine whether mv0, mv1, and mv2 are available in the adjacent blocks of the current block. Here, mv0 can represent the CPMVP candidate of CP0 in the current block, mv1 can represent the CPMVP candidate of CP1 in the current block, and mv2 can represent the CPMVP candidate of CP2 in the current block. In addition, mv0, mv1, and mv2 can represent the candidate motion vectors of each CP.
[0216] For example, an encoding / decoding device can check whether the motion vectors of adjacent blocks in a first group satisfy a specific condition according to a specific order. The encoding / decoding device can derive mv0 from the motion vectors of the adjacent blocks that are first identified as satisfying the condition during the checking process. In other words, mv0 can be the motion vector that is first identified as satisfying the specific condition by checking the motion vectors in the first group according to a specific order. If the motion vectors of adjacent blocks in the first group do not satisfy the specific condition, then mv0 may not exist. Here, for example, the specific order can be performed in the order of adjacent blocks A, B, and C in the first group. Additionally, for example, the specific condition can make the reference image of the motion vectors of adjacent blocks the same as the reference image of the current block.
[0217] Additionally, for example, the encoding / decoding device can check whether the motion vectors of adjacent blocks within the second group satisfy a specific condition according to a specific order. The encoding / decoding device can derive mv1 from the motion vectors of the adjacent blocks that are first identified as satisfying the condition during the checking process. In other words, mv1 can be the motion vector that is first identified as satisfying the specific condition by checking the motion vectors within the second group according to a specific order. If the motion vectors of adjacent blocks within the second group do not satisfy the specific condition, then mv1 may not exist. Here, for example, the specific order can be performed from adjacent block D to adjacent block E in the second group. Additionally, for example, the specific condition can make the reference image of the motion vectors of adjacent blocks the same as the reference image of the current block.
[0218] Additionally, for example, the encoding / decoding device can check whether the motion vectors of adjacent blocks in the third group satisfy a specific condition according to a specific order. The encoding / decoding device can derive mv2 from the motion vectors of the adjacent blocks that are first confirmed to satisfy the condition during the checking process. In other words, mv2 can be the motion vector that is first confirmed to satisfy the specific condition from the motion vectors in the third group according to the specific order. If the motion vectors of adjacent blocks in the third group do not satisfy the specific condition, then mv2 may not exist. Here, for example, the specific order can be performed from adjacent block F to adjacent block G in the third group. Additionally, for example, the specific condition can make the reference image of the motion vectors of adjacent blocks the same as the reference image of the current block.
[0219] Furthermore, the first group may include the motion vectors of adjacent block A, adjacent block B, and adjacent block C; the second group may include the motion vectors of adjacent block D and adjacent block E; and the third group may include the motion vectors of adjacent block F and adjacent block G. Adjacent block A may represent the adjacent block located to the upper left of the current block's upper-left sample position; adjacent block B may represent the adjacent block located above the current block's upper-left sample position; adjacent block C may represent the adjacent block located to the left of the current block's upper-left sample position; adjacent block D may represent the adjacent block located above the current block's upper-right sample position; adjacent block E may represent the adjacent block located to the upper right of the current block's upper-right sample position; adjacent block F may represent the adjacent block located to the left of the current block's lower-left sample position; and adjacent block G may represent the adjacent block located to the lower left of the current block's lower-left sample position.
[0220] When only mv0 and mv1 are available for the current block, that is, when only mv0 and mv1 of the current block are derived, the encoding / decoding device can derive mv2 of the current block based on Equation 8 above in S1510. The encoding / decoding device can derive mv2 by inserting the derived mv0 and mv1 into Equation 8 above.
[0221] When only mv0 and mv2 are available for the current block, that is, when only mv0 and mv2 of the current block are derived, the encoding / decoding device can derive mv1 of the current block based on Equation 9 above in S1520. The encoding / decoding device can derive mv1 by inserting the derived mv0 and mv2 into Equation 9 above.
[0222] In S1530, the encoding / decoding device can provide the derived mv0, mv1, and mv2 as candidates for constructing the current block. When mv0, mv1, and mv2 are available, that is, when mv0, mv1, and mv2 are derived based on the adjacent blocks of the current block, the encoding / decoding device can provide the derived mv0, mv1, and mv2 as candidates for constructing the current block.
[0223] Additionally, when only mv0 and mv1 are available for the current block, i.e., when only mv0 and mv1 of the current block are derived, the encoding / decoding device can provide the derived mv0, mv1, and mv2 derived based on Equation 8 above as candidates for constructing the current block.
[0224] Additionally, when only mv0 and mv2 are available for the current block, i.e., when only mv0 and mv2 of the current block are derived, the encoding / decoding device can provide the derived mv0, mv2, and mv1 derived based on Equation 9 above as candidates for constructing the current block.
[0225] Additionally, for example, if the affine motion model applied to the current block is a four-parameter affine motion model, then it can be like this: Figure 15 The candidate construct is derived as shown in the implementation.
[0226] Figure 16 An example of a candidate for the derivation construction is shown.
[0227] Reference Figure 16 In S1600, the encoding / decoding device can examine mv0, mv1, and mv2. In other words, the encoding / decoding device can determine whether mv0, mv1, and mv2 are available in the adjacent blocks of the current block. Here, mv0 can represent a CPMVP candidate for CP0 of the current block, mv1 can represent a CPMVP candidate for CP1 of the current block, and mv2 can represent a CPMVP candidate for CP2 of the current block.
[0228] For example, an encoding / decoding device can check whether the motion vectors of adjacent blocks in a first group satisfy a specific condition according to a specific order. The encoding / decoding device can derive mv0 from the motion vectors of adjacent blocks that are first identified as satisfying the condition during the checking process. In other words, mv0 can be the motion vector that is first identified as satisfying the specific condition by checking the motion vectors in the first group according to a specific order. If the motion vectors of adjacent blocks in the first group do not satisfy the specific condition, then mv0 may not exist. Here, for example, the specific order can be performed in the order of adjacent blocks A, B, and C in the first group. Additionally, for example, the specific condition may make the reference image of the motion vectors of adjacent blocks the same as the reference image of the current block.
[0229] Additionally, for example, the encoding / decoding device can check whether the motion vectors of adjacent blocks within the second group satisfy a specific condition according to a specific order. The encoding / decoding device can derive mv1 from the motion vector of the adjacent block that is first confirmed to satisfy the condition during the checking process. In other words, mv1 can be the motion vector that is first confirmed to satisfy the specific condition by checking the motion vectors within the second group according to a specific order. If the motion vectors of adjacent blocks within the second group do not satisfy the specific condition, then mv1 may not exist. Here, for example, the specific order can be performed from adjacent block D to adjacent block E within the second group. Additionally, for example, the specific condition can make the reference image of the motion vector of the adjacent block the same as the reference image of the current block.
[0230] Additionally, for example, the encoding / decoding device can check whether the motion vectors of adjacent blocks within a third group satisfy a specific condition according to a specific order. The encoding / decoding device can derive mv2 from the motion vectors of the adjacent blocks that are first identified as satisfying the condition during the checking process. In other words, mv2 can be the motion vector that is first identified as satisfying the specific condition from the motion vectors within the third group checked according to a specific order. If the motion vectors of adjacent blocks within the third group do not satisfy the specific condition, then mv2 may not exist. Here, for example, the specific order can be performed from adjacent block F to adjacent block G within the third group. Additionally, for example, the specific condition can make the reference image of the motion vectors of adjacent blocks the same as the reference image of the current block.
[0231] Furthermore, the first group may include the motion vectors of adjacent block A, adjacent block B, and adjacent block C; the second group may include the motion vectors of adjacent block D and adjacent block E; and the third group may include the motion vectors of adjacent block F and adjacent block G. Adjacent block A may represent the adjacent block located to the upper left of the current block's upper-left sample position; adjacent block B may represent the adjacent block located above the current block's upper-left sample position; adjacent block C may represent the adjacent block located to the left of the current block's upper-left sample position; adjacent block D may represent the adjacent block located above the current block's upper-right sample position; adjacent block E may represent the adjacent block located to the upper right of the current block's upper-right sample position; adjacent block F may represent the adjacent block located to the left of the current block's lower-left sample position; and adjacent block G may represent the adjacent block located to the lower left of the current block's lower-left sample position.
[0232] In S1610, the encoding / decoding device may provide the derived mv0 and mv1 as candidates for constructing the current block when only mv0 and mv1 are available for the current block, or when mv0, mv1 and mv2 are available for the current block. That is, when only mv0 and mv1 are derived for the current block, or when mv0, mv1 and mv2 are derived for the current block.
[0233] Furthermore, when only mv0 and mv2 are available for the current block, i.e., when only mv0 and mv2 are derived for the current block, in S1620, the encoding / decoding device can derive mv1 for the current block based on Equation 9 above. The encoding / decoding device can derive mv1 by inserting the derived mv0 and mv2 into Equation 9 above.
[0234] Then, in S1610, the encoding / decoding device can provide the derived mv0 and mv1 as candidates for constructing the current block.
[0235] Furthermore, another implementation of the affine candidate for inheritance according to this disclosure will be proposed. The proposed implementation can reduce computational complexity and thus improve coding performance when deriving the affine candidate for inheritance.
[0236] Figure 17 This example illustrates the positions of adjacent blocks scanned to derive the affine candidate of inheritance.
[0237] Encoding / decoding devices can derive up to two inherited affine candidates from adjacent blocks of the current block. Figure 17 The example illustrates adjacent blocks of inherited affine candidates. For example, adjacent blocks may include... Figure 17 The adjacent blocks A and B are shown. Adjacent block A can represent the left adjacent block A0, and adjacent block B can represent the upper adjacent block B0.
[0238] For example, an encoding / decoding device can check the availability of neighboring blocks in a specific order and derive affine candidates for the current block's inheritance based on the first neighboring block confirmed as available. In other words, the encoding / decoding device can check neighboring blocks in a specific order to see if they satisfy a specific condition and derive affine candidates for the current block's inheritance based on the first neighboring block confirmed as available. Furthermore, the encoding / decoding device can derive affine candidates for the current block's inheritance based on a second neighboring block confirmed as satisfying the specific condition. In other words, the encoding / decoding device can derive affine candidates for the current block's inheritance based on a second neighboring block confirmed as satisfying the specific condition. Here, availability can mean that the block is encoded based on an affine motion model and that the block's reference image is the same as the current block's reference image. In other words, a specific condition can indicate that the block is encoded based on an affine motion model and that the block's reference image is the same as the current block's reference image. Additionally, for example, a specific order can be executed from neighboring block A to neighboring block B. Furthermore, pruning checks can be omitted between two inherited affine candidates (i.e., the derived inherited affine candidates). Pruning checks can represent a process that checks whether candidates are identical to each other and removes the later derived candidate if identical candidates are found.
[0239] The above implementation proposes a method for examining only two adjacent blocks (i.e., adjacent block A and adjacent block B) and deriving affine candidates for inheritance, instead of examining all existing adjacent blocks (i.e., adjacent block A, adjacent block B, adjacent block C, adjacent block D, and adjacent block E) and deriving affine candidates for inheritance. Here, adjacent block C can represent the upper right adjacent block B1, adjacent block D can represent the lower left adjacent block A1, and adjacent block E can represent the upper left adjacent block B2.
[0240] When affine prediction is applied to each neighboring block to analyze the spatial correlation between neighboring blocks and the current block, the probability of applying the affine prediction to the current block can be utilized. The probability of applying the affine prediction to the current block can be derived as shown in the table below when applying the affine prediction to each neighboring block.
[0241] [Table 2]
[0242] Reference block A B C D E probability 65% 41% 5% 3% 1%
[0243] Referring to Table 2 above, it can be observed that neighboring blocks A and B within adjacent blocks have a high spatial correlation with the current block. Therefore, by using only neighboring blocks A and B, which exhibit high spatial correlation, to derive the inherited affine candidates, processing time can be reduced and high decoding performance can be achieved.
[0244] Furthermore, pruning checks can be performed to prevent duplicate candidates from existing in the candidate list. While pruning checks can remove redundancy, thus improving coding efficiency, computational complexity can increase. Specifically, the computational complexity is very high because pruning checks must consider the affine type (e.g., whether the affine motion model is four-parameter or six-parameter), the reference image (or reference image index), and the MV of CP0, CP1, and CP2. Therefore, this embodiment proposes a method that does not perform pruning checks between inherited affine candidates derived from neighboring block A (e.g., inherited_A) and inherited affine candidates derived from neighboring block B (e.g., inherited_B). In the case of neighboring blocks A and B, they are far apart and exhibit low spatial correlation. Therefore, the probability of inherited_A and inherited_B being identical is low. Thus, it is desirable not to perform pruning checks between inherited affine candidates.
[0245] Furthermore, based on the above reasons, a method for performing pruning check processing with minimal intervention can be proposed. For example, the encoding / decoding device can perform pruning check processing by comparing only the MVs of CP0 in the inherited affine candidates with each other.
[0246] Furthermore, this disclosure proposes a method for deriving a construct candidate that differs from the construct candidates obtained through the above embodiments. Compared to the above embodiments for deriving construct candidates, the proposed embodiment can improve coding performance by reducing complexity. The proposed embodiment can be described as follows. Additionally, when the number of available affine inheritance candidates is less than 2 (i.e., when the number of derived affine inheritance candidates is less than 2), constructed affine candidates can be considered.
[0247] For example, the encoding / decoding device can examine mv0, mv1, and mv2 for the current block. In other words, the encoding / decoding device can determine whether mv0, mv1, and mv2 are available in the adjacent blocks of the current block. Here, mv0 can represent a CPMVP candidate for CP0 of the current block, mv1 can represent a CPMVP candidate for CP1 of the current block, and mv2 can represent a CPMVP candidate for CP2 of the current block.
[0248] Specifically, the adjacent blocks of the current block can be divided into three groups, and the adjacent blocks can include adjacent block A, adjacent block B, adjacent block C, adjacent block D, adjacent block E, adjacent block F, and adjacent block G. The first group can include the motion vectors of adjacent block A, adjacent block B, and adjacent block C; the second group can include the motion vectors of adjacent block D and adjacent block E; and the third group can include the motion vectors of adjacent block F and adjacent block G. Adjacent block A can represent the adjacent block located to the upper left of the upper left sample position of the current block; adjacent block B can represent the adjacent block located above the upper left sample position of the current block; adjacent block C can represent the adjacent block located to the left of the upper left sample position of the current block; adjacent block D can represent the adjacent block located above the upper right sample position of the current block; adjacent block E can represent the adjacent block located to the upper right of the upper right sample position of the current block; adjacent block F can represent the adjacent block located to the left of the lower left sample position of the current block; and adjacent block G can represent the adjacent block located to the lower left of the lower left sample position of the current block.
[0249] The encoding / decoding device can determine the availability of mv0 in the first group, the availability of mv1 in the second group, and the availability of mv2 in the third group.
[0250] More specifically, for example, the encoding / decoding device can check whether the motion vectors of adjacent blocks in the first group satisfy a specific condition according to a specific order. The encoding / decoding device can derive mv0 from the motion vectors of the adjacent blocks that are first identified as satisfying the condition during the checking process. In other words, mv0 can be the motion vector that is first identified as satisfying the specific condition by checking the motion vectors in the first group according to a specific order. If the motion vectors of adjacent blocks in the first group do not satisfy the specific condition, then mv0 may not exist. Here, for example, the specific order can be performed in the order of adjacent blocks A, adjacent blocks B, and adjacent blocks C in the first group. In addition, for example, the specific condition can make the reference image of the motion vectors of the adjacent blocks the same as the reference image of the current block.
[0251] Furthermore, the encoding / decoding device can check whether the motion vectors of adjacent blocks within the second group satisfy specific conditions according to a specific order. The encoding / decoding device can derive mv1 from the motion vectors of the adjacent blocks that are first identified as satisfying the conditions during the checking process. In other words, mv1 can be the motion vector that is first identified as satisfying the specific conditions by checking the motion vectors within the second group according to a specific order. If the motion vectors of adjacent blocks within the second group do not satisfy the specific conditions, then mv1 may not exist. Here, for example, the specific order can be executed from adjacent block D to adjacent block E within the second group. Additionally, for example, the specific conditions may make the reference image of the motion vectors of adjacent blocks the same as the reference image of the current block.
[0252] Furthermore, the encoding / decoding device can check whether the motion vectors of adjacent blocks within the third group satisfy specific conditions according to a specific order. The encoding / decoding device can derive mv2 from the motion vectors of the adjacent blocks that are first identified as satisfying the conditions during the checking process. In other words, mv2 can be the motion vector that is first identified as satisfying the specific conditions by checking the motion vectors within the third group according to a specific order. If the motion vectors of adjacent blocks within the third group do not satisfy the specific conditions, then mv2 may not exist. Here, for example, the specific order can be executed from adjacent block F to adjacent block G within the third group. Additionally, for example, the specific conditions may make the reference image of the motion vectors of adjacent blocks the same as the reference image of the current block.
[0253] Then, if the affine motion model applied to the current block is a 4-parameter affine motion model, and mv0 and mv1 of the current block are available, the encoding / decoding device can provide the derived mv0 and mv1 as candidates for the construction of the current block. Furthermore, if mv0 and / or mv1 of the current block are not available, i.e., if at least one of mv0 and mv1 is not derived from the adjacent blocks of the current block, the encoding / decoding device may not add candidates for construction to the affine MVP list of the current block.
[0254] Furthermore, if the affine motion model applied to the current block is a 6-parameter affine motion model, and mv0, mv1, and mv2 of the current block are available, the encoding / decoding device can provide derived mv0, mv1, and mv2 as construction candidates for the current block. Conversely, if mv0, mv1, and / or mv2 of the current block are not available, i.e., if at least one of mv0, mv1, and mv2 is not derived from adjacent blocks of the current block, the encoding / decoding device may not add construction candidates to the affine MVP list of the current block.
[0255] The proposed implementation describes a method for considering motion vectors of the CP used to generate the affine motion model of the current block as candidates for construction only when all motion vectors are available. Here, availability can mean that the reference image of the adjacent block is the same as the reference image of the current block. In other words, candidates for construction can only be derived if there are motion vectors among the motion vectors of the adjacent blocks of each CP of the current block that satisfy the conditions. Therefore, if the affine motion model applied to the current block is a 4-parameter affine motion model, candidates for construction can only be considered if the motion vectors of CP0 and CP1 of the current block (i.e., mv0 and mv1) are available. Furthermore, if the affine motion model applied to the current block is a 6-parameter affine motion model, candidates for construction can only be considered if the motion vectors of CP0, CP1, and CP2 of the current block (i.e., mv0, mv1, and mv2) are available. Therefore, according to the proposed implementation, additional construction for deriving the motion vectors of the CP based on Equation 8 or Equation 9 may not be necessary. The proposed implementation reduces the computational complexity for deriving candidates for construction. In addition, since the constructed candidate is determined only when a CPMVP candidate with the same reference image is available, the overall coding performance can be improved.
[0256] Furthermore, pruning checks may not be performed between derived affine candidates and constructed affine candidates. Pruning checks can represent a process that checks whether candidates are identical to each other and removes the subsequently derived candidate if identical candidates are found.
[0257] The above implementation methods can be as follows: Figure 18 and Figure 19 As shown.
[0258] Figure 18 An example is shown of deriving a candidate construct when a four-parameter affine motion model is applied to the current block.
[0259] Reference Figure 18 In S1800, the encoding / decoding device can determine whether mv0 and mv1 of the current block are available. In other words, the encoding / decoding device can determine whether there are available mv0 and mv1 in the adjacent blocks of the current block. Here, mv0 can be a CPMVP candidate for CP0 of the current block, and mv1 can be a CPMVP candidate for CP1.
[0260] The encoding / decoding device can determine whether mv0 is available in the first group and whether mv1 is available in the second group.
[0261] Specifically, the adjacent blocks of the current block can be divided into three groups, and the adjacent blocks can include adjacent block A, adjacent block B, adjacent block C, adjacent block D, adjacent block E, adjacent block F, and adjacent block G. The first group can include the motion vectors of adjacent block A, adjacent block B, and adjacent block C; the second group can include the motion vectors of adjacent block D and adjacent block E; and the third group can include the motion vectors of adjacent block F and adjacent block G. Adjacent block A can represent the adjacent block located to the upper left of the upper left sample position of the current block; adjacent block B can represent the adjacent block located above the upper left sample position of the current block; adjacent block C can represent the adjacent block located to the left of the upper left sample position of the current block; adjacent block D can represent the adjacent block located above the upper right sample position of the current block; adjacent block E can represent the adjacent block located to the upper right of the upper right sample position of the current block; adjacent block F can represent the adjacent block located to the left of the lower left sample position of the current block; and adjacent block G can represent the adjacent block located to the lower left of the lower left sample position of the current block.
[0262] The encoding / decoding device can check whether the motion vectors of adjacent blocks in the first group satisfy a specific condition according to a specific order. The encoding / decoding device can derive mv0 from the motion vectors of the adjacent blocks that are first identified as satisfying the condition during the checking process. In other words, mv0 can be the motion vector that is first identified as satisfying the specific condition by checking the motion vectors in the first group according to a specific order. If the motion vectors of adjacent blocks in the first group do not satisfy the specific condition, then mv0 may not exist. Here, for example, the specific order can be performed in the order of adjacent blocks A, B, and C in the first group. Additionally, for example, the specific condition may make the reference image of the motion vectors of adjacent blocks the same as the reference image of the current block.
[0263] Furthermore, the encoding / decoding device can check whether the motion vectors of adjacent blocks within the second group satisfy specific conditions according to a specific order. The encoding / decoding device can deduce mv1 from the motion vectors of the adjacent blocks that are first confirmed to satisfy the conditions during the checking process. In other words, mv1 can be the motion vector first confirmed to satisfy the specific conditions by checking the motion vectors within the second group according to a specific order. If the motion vectors of adjacent blocks within the second group do not satisfy the specific conditions, then mv1 may not exist. Here, for example, the specific order can be executed from adjacent block D to adjacent block E within the second group. Additionally, for example, the specific conditions may make the reference image of the motion vectors of adjacent blocks the same as the reference image of the current block.
[0264] If mv0 and mv1 of the current block are available, i.e., if mv0 and mv1 of the current block are derived, then in S1810, the encoding / decoding device can provide the derived mv0 and mv1 as candidates for constructing the current block. Furthermore, if mv0 and mv1 of the current block are not available, i.e., if at least one of mv0 and mv1 is not derived from the adjacent blocks of the current block, then the encoding / decoding device may not add candidates for constructing the affine MVP list of the current block.
[0265] Furthermore, pruning checks may not be performed between derived affine candidates and constructed affine candidates. Pruning checks can represent a process that checks whether candidates are identical to each other and removes the subsequently derived candidate if identical candidates are found.
[0266] Figure 19 An example is shown of a candidate construct derived when a six-parameter affine motion model is applied to the current block.
[0267] Reference Figure 19 In S1900, the encoding / decoding device can determine whether mv0, mv1, and mv2 are available in the adjacent blocks of the current block. In other words, the encoding / decoding device can determine whether there are available mv0, mv1, and mv2 in the adjacent blocks of the current block. Here, mv0 can represent a CPMVP candidate for CP0 of the current block, mv1 can represent a CPMVP candidate for CP1 of the current block, and mv2 can represent a CPMVP candidate for CP2 of the current block.
[0268] The encoding / decoding device can determine whether mv0 is available in the first group, whether mv1 is available in the second group, and whether mv2 is available in the third group.
[0269] Specifically, the adjacent blocks of the current block can be divided into three groups, and the adjacent blocks can include adjacent block A, adjacent block B, adjacent block C, adjacent block D, adjacent block E, adjacent block F, and adjacent block G. The first group can include the motion vectors of adjacent block A, adjacent block B, and adjacent block C; the second group can include the motion vectors of adjacent block D and adjacent block E; and the third group can include the motion vectors of adjacent block F and adjacent block G. Adjacent block A can represent the adjacent block located to the upper left of the upper left sample position of the current block; adjacent block B can represent the adjacent block located above the upper left sample position of the current block; adjacent block C can represent the adjacent block located to the left of the upper left sample position of the current block; adjacent block D can represent the adjacent block located above the upper right sample position of the current block; adjacent block E can represent the adjacent block located to the upper right of the upper right sample position of the current block; adjacent block F can represent the adjacent block located to the left of the lower left sample position of the current block; and adjacent block G can represent the adjacent block located to the lower left of the lower left sample position of the current block.
[0270] The encoding / decoding device can check whether the motion vectors of adjacent blocks in the first group satisfy a specific condition according to a specific order. The encoding / decoding device can derive mv0 from the motion vectors of the adjacent blocks that are first identified as satisfying the condition during the checking process. In other words, mv0 can be the motion vector that is first identified as satisfying the specific condition by checking the motion vectors in the first group according to a specific order. If the motion vectors of adjacent blocks in the first group do not satisfy the specific condition, then mv0 may not exist. Here, for example, the specific order can be performed in the order of adjacent blocks A, B, and C in the first group. Additionally, for example, the specific condition may make the reference image of the motion vectors of adjacent blocks the same as the reference image of the current block.
[0271] Furthermore, the encoding / decoding device can check whether the motion vectors of adjacent blocks within the second group satisfy specific conditions according to a specific order. The encoding / decoding device can derive mv1 from the motion vectors of the adjacent blocks that are first identified as satisfying the conditions during the checking process. In other words, mv1 can be the motion vector that is first identified as satisfying the specific conditions by checking the motion vectors within the second group according to a specific order. If the motion vectors of adjacent blocks within the second group do not satisfy the specific conditions, then mv1 may not exist. Here, for example, the specific order can be executed from adjacent block D to adjacent block E within the second group. Additionally, for example, the specific conditions may make the reference image of the motion vectors of adjacent blocks the same as the reference image of the current block.
[0272] Furthermore, the encoding / decoding device can check whether the motion vectors of adjacent blocks within the third group satisfy specific conditions according to a specific order. The encoding / decoding device can derive mv2 from the motion vectors of the adjacent blocks that are first identified as satisfying the conditions during the checking process. In other words, mv2 can be the motion vector that is first identified as satisfying the specific conditions by checking the motion vectors within the third group according to a specific order. If the motion vectors of adjacent blocks within the third group do not satisfy the specific conditions, then mv2 may not exist. Here, for example, the specific order can be executed from adjacent block F to adjacent block G within the third group. Additionally, for example, the specific conditions may make the reference image of the motion vectors of adjacent blocks the same as the reference image of the current block.
[0273] If mv0, mv1, and mv2 of the current block are available, i.e., if mv0, mv1, and mv2 of the current block are derived, then in S1910, the encoding / decoding device can provide the derived mv0, mv1, and mv2 as candidates for constructing the current block. Furthermore, if mv0, mv1, and / or mv2 of the current block are not available, i.e., if at least one of mv0, mv1, and mv2 is not derived from the adjacent blocks of the current block, then the encoding / decoding device may not add candidates for constructing the affine MVP list of the current block.
[0274] Furthermore, pruning checks may not be performed between derived affine candidates and constructed affine candidates.
[0275] In addition, when the number of derived affine candidates is less than 2 (i.e., when the number of inherited affine candidates and / or the number of constructed affine candidates is less than 2), HEVC AMVP candidates can be added to the affine MVP list of the current block.
[0276] For example, HEVC AMVP candidates can be derived in the following order.
[0277] More specifically, CPMV0 can be used as an affine MVP candidate when the number of derived affine candidates is less than 2 and the constructed affine candidate CPMV0 is available. In other words, when the number of derived affine candidates is less than 2 and the constructed affine candidate CPMV0 is available (i.e., when the number of derived affine candidates is less than 2 and the constructed affine candidate CPMV0 is derived), the constructed affine candidate CPMV0 can be derived as a first affine MVP candidate including CPMV0, CPMV1, and CPMV2.
[0278] Furthermore, when the number of derived affine candidates is less than 2 and the constructed affine candidate CPMV1 is available, CPMV1 can be used as an affine MVP candidate. In other words, when the number of derived affine candidates is less than 2 and the constructed affine candidate CPMV1 is available (i.e., when the number of derived affine candidates is less than 2 and the constructed affine candidate CPMV1 is derived), the constructed affine candidate CPMV1 can be derived as a second affine MVP candidate including CPMV0, CPMV1, and CPMV2.
[0279] Furthermore, when the number of derived affine candidates is less than 2, and the constructed affine candidate CPMV2 is available, CPMV2 can be used as an affine MVP candidate. In other words, when the number of derived affine candidates is less than 2, and the constructed affine candidate CPMV2 is available (i.e., when the number of derived affine candidates is less than 2, and the constructed affine candidate CPMV2 is derived), the constructed affine candidate CPMV2 can be derived as a third affine MVP candidate including CPMV0, CPMV1, and CPMV2.
[0280] Furthermore, when the number of derived affine candidates is less than two, the HEVC Temporal Motion Vector Predictor (TMVP) can be used as an affine MVP candidate. The HEVC TMVP can be derived based on the motion information of the temporally neighboring blocks of the current block. In other words, when the number of derived affine candidates is less than two, the motion vectors of the temporally neighboring blocks of the current block can be derived as a third affine MVP candidate including CPMV0, CPMV1, and CPMV2. Temporally neighboring blocks can indicate juxtaposed blocks within the juxtaposed image corresponding to the current block.
[0281] Furthermore, when the number of derived affine candidates is less than 2, the zero motion vector (MV) can be used as an affine MVP candidate. In other words, when the number of derived affine candidates is less than 2, the zero motion vector can be derived as a third affine MVP candidate including CPMV0, CPMV1, and CPMV2. The zero motion vector can be represented as a motion vector whose elements are all zero.
[0282] The processing steps of the CPMV using the constructed affine candidate are reused, which already considers the generation of the constructed affine candidate MV, thereby reducing the processing complexity compared to existing methods used to derive HEVC AMVP candidates.
[0283] Furthermore, this disclosure proposes another implementation for deriving affine candidates for inheritance.
[0284] To derive the inherited affine candidate, affine prediction information of adjacent blocks is needed, and more specifically, the following affine prediction information is needed:
[0285] 1) Affine flag (affine_flag), which indicates whether affine prediction-based coding has been applied to neighboring blocks, and
[0286] 2) Motion information of adjacent blocks.
[0287] If a four-parameter affine motion model is applied to adjacent blocks, the motion information of adjacent blocks can include L0 and L1 motion information for CP0, and L0 and L1 motion information for CP1. Alternatively, if a six-parameter affine motion model is applied to adjacent blocks, the motion information of adjacent blocks can include L0 and L1 motion information for CP0, and L0 and L1 motion information for CP2. Here, L0 motion information can represent the motion information for list 0 (L0), and L1 motion information can represent the motion information for list 1 (L1). L0 motion information can include an L0 reference image index and an L0 motion vector, and L1 motion information can include an L1 reference image index and an L1 motion vector.
[0288] As mentioned above, in the case of affine prediction, a large amount of information must be stored, which is likely a major reason for the increased hardware cost in the actual implementation of encoding / decoding devices. Specifically, if an adjacent block is above the current block and belongs to the CTU boundary, a line buffer is needed to store information related to the affine prediction of the adjacent block, which can further increase the implementation cost. This problem can be referred to as the line buffer problem below. In this regard, this disclosure proposes an implementation for deriving affine candidates of inheritance that minimizes hardware cost by not storing affine prediction-related information in the line buffer or by reducing the amount of affine prediction-related information in the line buffer. The proposed implementation can improve coding performance by reducing the computational complexity in deriving affine candidates of inheritance. Furthermore, it should be noted that the line buffer already stores motion information about 4×4 size blocks, and if affine prediction-related information is additionally stored, the amount of information stored can be increased to three times the existing storage amount.
[0289] In this embodiment, additional information about affine prediction may not be stored in the row buffer, and the generation of inherited affine candidates may be restricted when the information in the row buffer must be used to generate inherited affine candidates.
[0290] Figure 20a and Figure 20b An implementation method for deriving affine candidates for inheritance is illustrated.
[0291] Reference Figure 20aWhen the adjacent block B (i.e., the block above the current block) does not belong to the same CTU as the current block, the adjacent block B may not be used to generate affine candidates for inheritance. Furthermore, although the adjacent block A also does not belong to the same CTU as the current block, information about the adjacent block A is not stored in the line buffer, therefore the adjacent block A can be used to generate affine candidates for inheritance. Therefore, according to this embodiment, an adjacent block can only be used to derive affine candidates for inheritance when the adjacent block above the current block belongs to the same CTU as the current block. Additionally, when the adjacent block above the current block does not belong to the same CTU as the current block, the upper adjacent block may not be used to derive affine candidates for inheritance.
[0292] Reference Figure 20b The neighboring block B of the current block (i.e., the block above the current block) can belong to the same CTU as the current block. In this case, the encoding / decoding device can generate inherited affine candidates by referring to the neighboring block B.
[0293] Figure 21 An example of a video coding method performed by an encoding device according to the present disclosure is illustrated. Figure 21 The method disclosed in the article can be derived from Figure 2 The encoding device disclosed herein shall be used to perform the following: More specifically, for example, steps S2100 to S2120 may be performed by the predictor of the encoding device, step S2130 may be performed by the subtractor of the encoding device, and step S2140 may be performed by the entropy encoder of the encoding device. Additionally, although not shown in the figure, the predictor of the encoding device may perform the processing for deriving the predicted sample of the current block based on the CPMV, the subtractor of the encoding device may perform the processing for deriving the residual sample of the current block based on the original sample and the predicted sample of the current block, the transformer of the encoding device may perform the processing for generating information about the residual of the current block based on the residual sample, and the encoder of the encoding device may perform the processing for encoding the information about the residual.
[0294] In S2100, the encoding device constructs an affine motion vector predictor (MVP) candidate list for the current block. The encoding device can construct an affine MVP candidate list for the current block that includes affine MVP candidates. The maximum number of affine MVP candidates in the affine MVP candidate list can be 2.
[0295] Additionally, as an example, the affine MVP candidate list can include inherited affine MVP candidates. The encoding device can check if inherited affine MVP candidates for the current block are available, and if so, can deduce them. For example, inherited affine MVP candidates can be derived based on the current block's neighboring blocks, and the maximum number of inherited affine MVP candidates can be two. The availability of neighboring blocks can be checked in a specific order, and inherited affine MVP candidates can be derived based on the checked available neighboring blocks. In other words, the availability of neighboring blocks can be checked in a specific order, the first inherited affine MVP candidate can be derived based on the first checked available neighboring block, and the second inherited affine MVP candidate can be derived based on the second checked available neighboring block. Availability can mean that the neighboring block is encoded based on an affine motion model, and that the neighboring block's reference image is the same as the current block's reference image. In other words, an available neighboring block can be a neighboring block encoded according to an affine motion model (i.e., a neighboring block to which affine prediction has been applied) whose reference image is the same as the current block's reference image. More specifically, the encoding device can derive the motion vector of the current block's CP based on the affine motion model of the first identified available neighboring blocks, and derive the first inherited affine MVP candidate including this motion vector as a CPMVP candidate. Alternatively, the encoding device can derive the motion vector of the current block's CP based on the affine motion model of the second identified available neighboring blocks, and derive the second inherited affine MVP candidate including this motion vector as a CPMVP candidate. The affine motion model can be derived according to Equation 1 or Equation 3 above.
[0296] In other words, neighboring blocks can be checked in a specific order to see if they meet specific conditions, and inherited affine MVP candidates can be derived based on neighboring blocks that meet the specific conditions. Specifically, the encoding device can derive the motion vector of the current block's CP based on the affine motion model of the neighboring blocks that first meet the specific conditions, and derive the first inherited affine MVP candidate based on the second inherited affine MVP candidate. The affine motion model can be derived according to Equation 1 or Equation 3 above. Furthermore, specific conditions can indicate that neighboring blocks are encoded according to an affine motion model, and that the reference image of the neighboring block is the same as the reference image of the current block. In other words, neighboring blocks that satisfy specific conditions can be encoded according to an affine motion model (i.e., affine prediction is applied to neighboring blocks), and their reference images are the same as the reference image of the current block.
[0297] Here, for example, adjacent blocks can include the current block's left adjacent block, top adjacent block, top right adjacent block, bottom left adjacent block, and top left adjacent block. In this case, the specific order could be from left adjacent block to bottom left adjacent block to top adjacent block to top right adjacent block to top left adjacent block.
[0298] Alternatively, for example, adjacent blocks may consist only of the left adjacent block and the top adjacent block. In this case, the specific order could be from the left adjacent block to the top adjacent block.
[0299] Alternatively, for example, a neighboring block may include its left neighboring block, and if its upper neighboring block belongs to the current CTU that includes the current block, then the neighboring block may further include its upper neighboring block. In this case, the specific order could be from the left neighboring block to the upper neighboring block. Additionally, if the upper neighboring block does not belong to the current CTU, the neighboring block may not include its upper neighboring block. In this case, only the left neighboring block can be checked.
[0300] Furthermore, when the current block size is W×H, and the x-component of the top-left sample position of the current block is 0, and its y-component is also 0, the bottom-left adjacent block can be the block containing the sample at coordinates (-1, H), the left adjacent block can be the block containing the sample at coordinates (-1, H-1), the top-right adjacent block can be the block containing the sample at coordinates (W, -1), the top adjacent block can be the block containing the sample at coordinates (W-1, -1), and the top-left adjacent block can be the block containing the sample at coordinates (-1, -1). In other words, the left adjacent block can be the left adjacent block at the lowest position among the left adjacent blocks of the current block, and the top adjacent block can be the top adjacent block at the leftmost position among the top adjacent blocks of the current block.
[0301] Additionally, as an example, if constructed affine MVP candidates are available, the affine MVP candidate list can include constructed affine MVP candidates. The encoding device can check if constructed affine MVP candidates for the current block are available, and if so, can derive them. Alternatively, for example, constructed affine MVP candidates can be derived after deriving inherited affine MVP candidates. If the number of derived affine MVP candidates (i.e., the number of inherited affine MVPs) is less than 2, and constructed affine MVP candidates are available, the affine MVP candidate list can include constructed affine MVP candidates. Here, constructed affine MVP candidates can include candidate motion vectors of the CP. Constructed affine MVP candidates can be available when all candidate motion vectors are available.
[0302] For example, if a four-parameter affine motion model is applied to the current block, the CP of the current block can include CP0 and CP1. If candidate motion vectors for CP0 and CP1 are available, the constructed affine MVP candidates can be available, and the list of affine MVP candidates can include the constructed affine MVP candidates. Here, CP0 can represent the top-left position of the current block, and CP1 can represent the top-right position of the current block.
[0303] The constructed affine MVP candidate can include candidate motion vectors for CP0 and candidate motion vectors for CP1. The candidate motion vector for CP0 can be the motion vector of the first block, and the candidate motion vector for CP1 can be the motion vector of the second block.
[0304] Additionally, the first block can be a block whose reference image is the same as the reference image of the current block, which is first confirmed to be the same as the reference image of the current block while checking neighboring blocks in the first group according to a first specific order. In other words, the candidate motion vector of CP1 can be the motion vector of the block whose reference image is the same as the reference image of the current block, which is first confirmed to be the same as the reference image of the current block by checking neighboring blocks in the first group according to a first order. Availability can indicate the existence of neighboring blocks, and neighboring blocks are encoded by inter-frame prediction. Here, if the reference image of the first block in the first group is the same as the reference image of the current block, then the candidate motion vector of CP0 can be available. In addition, for example, the first group can include neighboring blocks A, B, and C, and the first specific order can be from neighboring block A to neighboring block B, and then to neighboring block C.
[0305] Additionally, the second block can be a block whose reference image is the same as the reference image of the current block, while checking adjacent blocks in the second group in a second specific order. Here, if the reference image of the second block in the second group is the same as the reference image of the current block, then the candidate motion vector of CP1 can be available. Furthermore, for example, the second group can include adjacent blocks D and E, and the second specific order can be from adjacent block D to adjacent block E.
[0306] Furthermore, when the current block size is W×H, and the x-component of the top-left sample position of the current block is 0, and its y-component is 0, the adjacent block A can be the block containing the sample at coordinates (-1, -1), the adjacent block B can be the block containing the sample at coordinates (0, -1), the adjacent block C can be the block containing the sample at coordinates (-1, 0), the adjacent block D can be the block containing the sample at coordinates (W-1, -1), and the adjacent block E can be the block containing the sample at coordinates (W, -1). In other words, adjacent block A can be the top-left adjacent block of the current block, adjacent block B can be the leftmost adjacent block among the top adjacent blocks of the current block, adjacent block C can be the leftmost adjacent block among the left adjacent blocks of the current block, adjacent block D can be the rightmost adjacent block among the top adjacent blocks of the current block, and adjacent block E can be the top-right adjacent block of the current block.
[0307] Furthermore, if at least one of the candidate motion vectors of CP0 and CP1 is unavailable, the constructed affine MVP candidate may be unavailable.
[0308] Alternatively, for example, if a six-parameter affine motion model is applied to the current block, then the CP of the current block can include CP0, CP1, and CP2. If candidate motion vectors for CP0, CP1, and CP2 are available, then the constructed affine MVP candidates can be available, and the list of affine MVP candidates can include the constructed affine MVP candidates. Here, CP0 can represent the top-left position of the current block, CP1 can represent the top-right position of the current block, and CP2 can represent the bottom-left position of the current block.
[0309] The constructed affine MVP candidates can include candidate motion vectors for CP0, CP1, and CP2. The candidate motion vector for CP0 can be the motion vector of the first block, the candidate motion vector for CP1 can be the motion vector of the second block, and the candidate motion vector for CP2 can be the motion vector of the third block.
[0310] Additionally, the first block can be a block whose reference image is the same as the reference image of the current block, while checking adjacent blocks in the first group in a first specific order. Here, if the reference image of the first block in the first group is the same as the reference image of the current block, then the candidate motion vector of CP0 can be available. Furthermore, for example, the first group can include adjacent block A, adjacent block B, and adjacent block C, and the first specific order can be from adjacent block A to adjacent block B, and then to adjacent block C.
[0311] Additionally, the second block can be a block whose reference image is the same as the reference image of the current block, while checking adjacent blocks in the second group in a second specific order. Here, if the reference image of the second block in the second group is the same as the reference image of the current block, then the candidate motion vector of CP1 can be available. Furthermore, for example, the second group can include adjacent blocks D and E, and the second specific order can be from adjacent block D to adjacent block E.
[0312] Additionally, the third block can be a block whose reference image is the same as the reference image of the current block, which has already been identified while checking adjacent blocks in the third group in a third specific order. Here, if the reference image of the third block in the third group is the same as the reference image of the current block, then the candidate motion vector of CP2 can be available. Furthermore, for example, the third group can include adjacent blocks F and G, and the third specific order can be from adjacent block F to adjacent block G.
[0313] Furthermore, when the current block size is W×H, the x-component of the upper left sample position of the current block is 0, and its y-component is 0, the adjacent block A can be a block that includes a sample at coordinates (-1, -1), the adjacent block B can be a block that includes a sample at coordinates (0, -1), the adjacent block C can be a block that includes a sample at coordinates (-1, 0), the adjacent block D can be a block that includes a sample at coordinates (W-1, -1), the adjacent block E can be a block that includes a sample at coordinates (W, -1), the adjacent block F can be a block that includes a sample at coordinates (-1, H-1), and the adjacent block G can be a block that includes a sample at coordinates (-1, H). In other words, adjacent block A can be the top-left adjacent block of the current block, adjacent block B can be the leftmost adjacent block of the top adjacent blocks of the current block, adjacent block C can be the highest left adjacent block of the left adjacent blocks of the current block, adjacent block D can be the rightmost adjacent block of the top adjacent blocks of the current block, adjacent block E can be the top-right adjacent block of the current block, adjacent block F can be the lowest left adjacent block of the left adjacent blocks of the current block, and adjacent block G can be the bottom-left adjacent block of the current block.
[0314] Furthermore, if at least one of the candidate motion vectors of CP0, CP1, and CP2 is unavailable, the constructed affine MVP candidate may be unavailable.
[0315] Then, the affine MVP candidate list can be derived based on the steps described below.
[0316] For example, when the number of derived affine MVP candidates is less than 2 and the motion vector of CP0 is available, the encoding device can derive a first affine MVP candidate. Here, the first affine MVP candidate can be an affine MVP candidate that includes the motion vector of CP0, which is a candidate motion vector of CP.
[0317] Additionally, for example, when the number of derived affine MVP candidates is less than 2 and the motion vector of CP1 is available, the encoding device can derive a second affine MVP candidate. Here, the second affine MVP candidate can be an affine MVP candidate that includes the motion vector of CP1, which is a candidate motion vector of CP.
[0318] Additionally, for example, when the number of derived affine MVP candidates is less than 2 and the motion vector of CP2 is available, the encoding device can derive a third affine MVP candidate. Here, the third affine MVP candidate can be an affine MVP candidate that includes the motion vector of CP2, which is a candidate motion vector of CP.
[0319] Additionally, for example, when the number of derived affine MVP candidates is less than two, the encoding device can derive a fourth affine MVP candidate, which includes a temporal MVP derived based on temporally adjacent blocks of the current block, serving as candidate motion vectors for the CP. A temporally adjacent block can refer to an identical juxtaposed block within the same juxtaposed image corresponding to the current block. The temporal MVP can be derived based on the motion vectors of the temporally adjacent blocks.
[0320] Additionally, for example, when the number of derived affine MVP candidates is less than 2, the encoding device can derive a fifth affine MVP candidate, which includes a zero motion vector as a candidate motion vector for CP. A zero motion vector can represent a motion vector whose elements are all zero.
[0321] In S2110, the encoding device derives the control point motion vector predictor (CPMVP) of the control point (CP) of the current block based on the affine MVP candidate list. The encoding device can derive the CPMV of the current block's CP exhibiting the optimal RD cost, and can select the affine MVP candidate most similar to the CPMV from the affine MVP candidates as the affine MVP candidate for the current block. The encoding device can derive the CPMVP of the current block's CP based on the affine MVP candidate selected from the affine MVP candidates. More specifically, if the affine MVP candidates include candidate motion vectors of CP0 and candidate motion vectors of CP1, then the candidate motion vector of CP0 can be derived as the CPMVP of CP0, and the candidate motion vector of CP1 can be derived as the CPMVP of CP1. Furthermore, if the affine MVP candidate includes the candidate motion vectors of CP0, CP1, and CP2, then the candidate motion vector of CP0 can be derived as the CPMVP of CP0, the candidate motion vector of CP1 can be derived as the CPMVP of CP1, and the candidate motion vector of CP2 can be derived as the CPMVP of CP2.
[0322] The encoding device can encode the affine MVP candidate index, which indicates the selected affine MVP candidate among the affine MVP candidates. The affine MVP candidate index can indicate one of the affine MVP candidates included in the affine motion vector predictor (MVP) candidate list of the current block.
[0323] In S2120, the encoding device derives the CPMV of the CP of the current block. The encoding device can derive the CPMV of each CP of the current block.
[0324] In S2130, the encoding device derives the control point motion vector difference (CPMVD) of the current block's CP based on CPMVP and CPMV. The encoding device can derive the CPMVD of the current block's CP based on the CPMVP and CPMV of each CP.
[0325] In S2140, the encoding device encodes motion prediction information including information about the CPMVD. The encoding device can output the motion prediction information including information about the CPMVD in the form of a bitstream. In other words, the encoding device can output image information including motion prediction information in the form of a bitstream. The encoding device can encode information about the CPMVD for each CP, wherein the motion prediction information may include information about the CPMVD.
[0326] Additionally, motion prediction can include an affine MVP candidate index. The affine MVP candidate index can indicate the selected affine MVP candidate among the affine MVP candidates included in the affine motion vector predictor (MVP) candidate list of the current block.
[0327] Furthermore, as an example, the encoding device can derive the predicted sample of the current block based on the CPMV, derive the residual sample of the current block based on the original sample and the predicted sample of the current block, generate information about the residual of the current block based on the residual sample, and encode the information about the residual. Image information may include information about the residual.
[0328] Furthermore, the bitstream can be transmitted to the decoding device via a network or (digital) storage medium. Here, the network can include broadcast networks and / or communication networks, and the digital storage medium can include various types of storage media, including USB, SD, CD, DVD, Blu-ray, HDD, and SSD.
[0329] Figure 22 An encoding device for performing a video encoding method according to the present disclosure is illustrated. Figure 21 The method disclosed in the article can be derived from Figure 22 The encoding device disclosed herein is used to perform this action. More specifically, for example, the predictor of the encoding device can perform this action. Figure 21 The steps from S2100 to S2130, and Figure 22 The entropy encoder of the encoding device can perform Figure 21 Step S2140. Additionally, although not shown in the figure, the processing for deriving the predicted samples for the current block based on CPMV can be performed by... Figure 22The predictor of the encoding device performs the processing to derive the residual samples of the current block based on the original samples and predicted samples of the current block. Figure 22 The subtractor of the encoding device performs the processing for generating residual information about the current block based on residual samples; the transformer of the encoding device performs the processing for encoding the information about the residuals; and the transformer of the encoding device performs the processing for encoding the residual information. Figure 22 The entropy encoder of the encoding device is executed.
[0330] Figure 23 An example of a video decoding method performed by a decoding device according to the present disclosure is illustrated. Figure 23 The method disclosed in the article can be derived from Figure 3 The decoding is performed by the publicly disclosed decoding device. More specifically, for example, it can be performed by the entropy decoder of the decoding device. Figure 23 Step S2300 can be performed by the predictor of the decoding device, steps S2310 to S2350 can be performed by the adder of the decoding device, and step S2360 can be performed by the adder of the decoding device. Additionally, although not shown in the figure, the processing for obtaining residual information about the current block from the bitstream can be performed by the entropy decoder of the decoding device, and the processing for deriving residual samples of the current block based on the residual information can be performed by the inverse transformer of the decoding device.
[0331] In S2300, the decoding device obtains motion prediction information for the current block from the bitstream. The decoding device can obtain image information, including motion prediction information, from the bitstream.
[0332] Additionally, for example, motion prediction information may include information about the control point motion vector difference (CPMVD) of the control points (CP) of the current block. In other words, motion prediction information may include information about the CPVD of each CP of the current block.
[0333] Additionally, for example, motion prediction information may include an affine motion vector predictor (MVP) candidate index for the current block. The affine MVP candidate index can indicate one of the affine MVP candidates included in the list of affine MVP candidates for the current block.
[0334] In S2310, the decoding device constructs an affine MVP candidate list for the current block. The decoding device can construct an affine MVP candidate list for the current block that includes affine MVP candidates. The maximum number of affine MVP candidates in the affine MVP candidate list can be 2.
[0335] Furthermore, as an example, the affine MVP candidate list can include inherited affine MVP candidates. The decoding device can check if inherited affine MVP candidates for the current block are available, and if so, can deduce them. For example, inherited affine MVP candidates can be derived based on the neighboring blocks of the current block, and the maximum number of inherited affine MVP candidates can be two. The availability of neighboring blocks can be checked in a specific order, and inherited affine MVP candidates can be derived based on the checked available neighboring blocks. In other words, the availability of neighboring blocks can be checked in a specific order, the first inherited affine MVP candidate can be derived based on the first checked available neighboring block, and the second inherited affine MVP candidate can be derived based on the second checked available neighboring block. Availability can mean that the neighboring block is encoded based on an affine motion model, and that the reference image of the neighboring block is the same as the reference image of the current block. In other words, an available neighboring block can be a neighboring block encoded according to an affine motion model (i.e., a neighboring block to which affine prediction has been applied) and whose reference image is the same as the reference image of the current block. More specifically, the decoding device can derive the motion vector of the current block's CP based on the affine motion model of the first identified available neighboring blocks, and derive the first inherited affine MVP candidate including this motion vector as a CPMVP candidate. Additionally, the decoding device can derive the motion vector of the current block's CP based on the affine motion model of the second identified available neighboring blocks, and derive the second inherited affine MVP candidate including this motion vector as a CPMVP candidate. The affine motion model can be derived according to Equation 1 or Equation 3 above.
[0336] In other words, neighboring blocks can be checked in a specific order to see if they meet specific conditions, and affine MVP candidates can be derived based on neighboring blocks that meet the specific conditions. Specifically, the decoding device can derive the motion vector of the current block's CP based on the affine motion model of the neighboring blocks that first meet the specific conditions, and derive the first inherited affine MVP candidate based on the second inherited affine MVP candidate. The affine motion model can be derived according to Equation 1 or Equation 3 above. Furthermore, specific conditions can indicate that neighboring blocks are encoded according to an affine motion model, and that the reference image of the neighboring block is the same as the reference image of the current block. In other words, neighboring blocks that satisfy specific conditions can be encoded according to an affine motion model (i.e., affine prediction is applied to neighboring blocks) and have the same reference image as the current block.
[0337] Here, for example, adjacent blocks can include the current block's left adjacent block, top adjacent block, top right adjacent block, bottom left adjacent block, and top left adjacent block. In this case, the specific order could be from left adjacent block to bottom left adjacent block to top adjacent block to top right adjacent block to top left adjacent block.
[0338] Alternatively, for example, adjacent blocks may consist only of the left adjacent block and the top adjacent block. In this case, the specific order could be from the left adjacent block to the top adjacent block.
[0339] Alternatively, for example, a neighboring block may include its left neighboring block, and if its upper neighboring block belongs to the current coding tree unit (CTU) that includes the current block, then the neighboring block may further include its upper neighboring block. In this case, the specific order could be from the left neighboring block to the upper neighboring block. Additionally, if the upper neighboring block does not belong to the current CTU, the neighboring block may not include its upper neighboring block. In this case, only the left neighboring block may be examined. In other words, if the upper neighboring block of the current block belongs to the current coding tree unit (CTU) that includes the current block, then the upper neighboring block can be used to derive the inherited affine MVP candidate, and if the upper neighboring block of the current block does not belong to the current CTU, then the upper neighboring block may not be used to derive the inherited affine MVP candidate.
[0340] Furthermore, when the current block size is W×H, and the x-component of the top-left sample position of the current block is 0, and its y-component is also 0, the bottom-left adjacent block can be the block containing the sample at coordinates (-1, H), the left adjacent block can be the block containing the sample at coordinates (-1, H-1), the top-right adjacent block can be the block containing the sample at coordinates (W, -1), the top adjacent block can be the block containing the sample at coordinates (W-1, -1), and the top-left adjacent block can be the block containing the sample at coordinates (-1, -1). In other words, the left adjacent block can be the left adjacent block at the lowest position among the left adjacent blocks of the current block, and the top adjacent block can be the top adjacent block at the leftmost position among the top adjacent blocks of the current block.
[0341] Additionally, as an example, if constructed affine MVP candidates are available, the affine MVP candidate list can include constructed affine MVP candidates. The decoding device can check if constructed affine MVP candidates for the current block are available, and if so, can derive them. Alternatively, for example, constructed affine MVP candidates can be derived after deriving inherited affine MVP candidates. If the number of derived affine MVP candidates (i.e., the number of inherited affine MVPs) is less than 2, and constructed affine MVP candidates are available, the affine MVP candidate list can include constructed affine MVP candidates. Here, constructed affine MVP candidates can include candidate motion vectors of the CP. Constructed affine MVP candidates can be available when all candidate motion vectors are available.
[0342] For example, if a four-parameter affine motion model is applied to the current block, the CP of the current block can include CP0 and CP1. If candidate motion vectors for CP0 and CP1 are available, the constructed affine MVP candidates can be available, and the list of affine MVP candidates can include the constructed affine MVP candidates. Here, CP0 can represent the top-left position of the current block, and CP1 can represent the top-right position of the current block.
[0343] The constructed affine MVP candidate can include candidate motion vectors for CP0 and candidate motion vectors for CP1. The candidate motion vector for CP0 can be the motion vector of the first block, and the candidate motion vector for CP1 can be the motion vector of the second block.
[0344] Additionally, the first block can be a block whose reference image is the same as the reference image of the current block, which is first confirmed to be the same as the reference image of the current block while checking neighboring blocks in the first group according to a first specific order. In other words, the candidate motion vector of CP1 can be the motion vector of the block whose reference image is the same as the reference image of the current block, which is first confirmed to be the same as the reference image of the current block by checking neighboring blocks in the first group according to a first order. Availability can indicate the existence of neighboring blocks, and neighboring blocks are encoded by inter-frame prediction. Here, if the reference image of the first block in the first group is the same as the reference image of the current block, then the candidate motion vector of CP0 can be available. In addition, for example, the first group can include neighboring blocks A, B, and C, and the first specific order can be from neighboring block A to neighboring block B, and then to neighboring block C.
[0345] Additionally, the second block can be a block whose reference image is the same as the reference image of the current block, while checking adjacent blocks in the second group in a second specific order. Here, if the reference image of the second block in the second group is the same as the reference image of the current block, then the candidate motion vector of CP1 can be available. Furthermore, for example, the second group can include adjacent blocks D and E, and the second specific order can be from adjacent block D to adjacent block E.
[0346] Furthermore, when the current block size is W×H, and the x-component of the top-left sample position of the current block is 0, and its y-component is 0, the adjacent block A can be the block containing the sample at coordinates (-1, -1), the adjacent block B can be the block containing the sample at coordinates (0, -1), the adjacent block C can be the block containing the sample at coordinates (-1, 0), the adjacent block D can be the block containing the sample at coordinates (W-1, -1), and the adjacent block E can be the block containing the sample at coordinates (W, -1). In other words, adjacent block A can be the top-left adjacent block of the current block, adjacent block B can be the leftmost adjacent block among the top adjacent blocks of the current block, adjacent block C can be the leftmost adjacent block among the left adjacent blocks of the current block, adjacent block D can be the rightmost adjacent block among the top adjacent blocks of the current block, and adjacent block E can be the top-right adjacent block of the current block.
[0347] Furthermore, if at least one of the candidate motion vectors of CP0 and CP1 is unavailable, the constructed affine MVP candidate may be unavailable.
[0348] Alternatively, for example, if a six-parameter affine motion model is applied to the current block, then the CP of the current block can include CP0, CP1, and CP2. If candidate motion vectors for CP0, CP1, and CP2 are available, then the constructed affine MVP candidates can be available, and the list of affine MVP candidates can include the constructed affine MVP candidates. Here, CP0 can represent the top-left position of the current block, CP1 can represent the top-right position of the current block, and CP2 can represent the bottom-left position of the current block.
[0349] The constructed affine MVP candidates can include candidate motion vectors for CP0, CP1, and CP2. The candidate motion vector for CP0 can be the motion vector of the first block, the candidate motion vector for CP1 can be the motion vector of the second block, and the candidate motion vector for CP2 can be the motion vector of the third block.
[0350] Additionally, the first block can be a block whose reference image is the same as the reference image of the current block, while checking adjacent blocks in the first group in a first specific order. Here, if the reference image of the first block in the first group is the same as the reference image of the current block, then the candidate motion vector of CP0 can be available. Furthermore, for example, the first group can include adjacent block A, adjacent block B, and adjacent block C, and the first specific order can be from adjacent block A to adjacent block B, and then to adjacent block C.
[0351] Additionally, the second block can be a block whose reference image is the same as the reference image of the current block, while checking adjacent blocks in the second group in a second specific order. Here, if the reference image of the second block in the second group is the same as the reference image of the current block, then the candidate motion vector of CP1 can be available. Furthermore, for example, the second group can include adjacent blocks D and E, and the second specific order can be from adjacent block D to adjacent block E.
[0352] Additionally, the third block can be a block whose reference image is the same as the reference image of the current block, which has already been identified while checking adjacent blocks in the third group in a third specific order. Here, if the reference image of the third block in the third group is the same as the reference image of the current block, then the candidate motion vector of CP2 can be available. Furthermore, for example, the third group can include adjacent blocks F and G, and the third specific order can be from adjacent block F to adjacent block G.
[0353] Furthermore, when the current block size is W×H, the x-component of the upper left sample position of the current block is 0, and its y-component is 0, the adjacent block A can be a block that includes a sample at coordinates (-1, -1), the adjacent block B can be a block that includes a sample at coordinates (0, -1), the adjacent block C can be a block that includes a sample at coordinates (-1, 0), the adjacent block D can be a block that includes a sample at coordinates (W-1, -1), the adjacent block E can be a block that includes a sample at coordinates (W, -1), the adjacent block F can be a block that includes a sample at coordinates (-1, H-1), and the adjacent block G can be a block that includes a sample at coordinates (-1, H). In other words, adjacent block A can be the top-left adjacent block of the current block, adjacent block B can be the leftmost adjacent block of the top adjacent blocks of the current block, adjacent block C can be the highest left adjacent block of the left adjacent blocks of the current block, adjacent block D can be the rightmost adjacent block of the top adjacent blocks of the current block, adjacent block E can be the top-right adjacent block of the current block, adjacent block F can be the lowest left adjacent block of the left adjacent blocks of the current block, and adjacent block G can be the bottom-left adjacent block of the current block.
[0354] Furthermore, if at least one of the candidate motion vectors of CP0, CP1, and CP2 is unavailable, the constructed affine MVP candidate may be unavailable.
[0355] Furthermore, pruning checks may not be performed between inherited affine MVP candidates and constructed affine MVP candidates. Pruning checks can mean checking whether the constructed affine MVP candidate is the same as the inherited affine MVP candidate, and if they are found to be the same, then no processing is performed for the constructed affine MVP candidate.
[0356] Then, the affine MVP candidate list can be derived based on the steps described below.
[0357] For example, when the number of derived affine MVP candidates is less than 2 and the motion vector of CP0 is available, the decoding device can derive a first affine MVP candidate. Here, the first affine MVP candidate can be an affine MVP candidate that includes the motion vector of CP0, which is a candidate motion vector of CP.
[0358] Additionally, for example, when the number of derived affine MVP candidates is less than 2 and the motion vector of CP1 is available, the decoding device can derive a second affine MVP candidate. Here, the second affine MVP candidate can be an affine MVP candidate that includes the motion vector of CP1, which is a candidate motion vector of CP.
[0359] Additionally, for example, when the number of derived affine MVP candidates is less than 2 and the motion vector of CP2 is available, the decoding device can derive a third affine MVP candidate. Here, the third affine MVP candidate can be an affine MVP candidate that includes the motion vector of CP2, which is a candidate motion vector of CP.
[0360] Additionally, for example, when the number of derived affine MVP candidates is less than two, the decoding device can derive a fourth affine MVP candidate, which includes a temporal MVP derived based on temporally adjacent blocks of the current block, serving as candidate motion vectors for the CP. A temporally adjacent block can refer to an identical juxtaposed block within the same juxtaposed image corresponding to the current block. The temporal MVP can be derived based on the motion vectors of the temporally adjacent blocks.
[0361] Additionally, for example, when the number of derived affine MVP candidates is less than 2, the decoding device can derive a fifth affine MVP candidate, which includes a zero motion vector as a candidate motion vector for the CP. A zero motion vector can represent a motion vector whose elements are all zero.
[0362] In S2320, the decoding device derives the control point motion vector predictor (CPMVP) of the control point (CP) of the current block based on the affine MVP candidate list.
[0363] The decoding device can select a specific affine MVP candidate from the affine MVP candidate list and derive the selected affine MVP candidate as the CPMVP of the current block. For example, the decoding device can obtain the affine MVP candidate index of the current block from the bitstream, and derive the affine MVP candidate indicated by the affine MVP candidate index as the CPMVP of the current block from the affine MVP candidate list. More specifically, if the affine MVP candidates include the candidate motion vector of CP0 and the candidate motion vector of CP1, then the candidate motion vector of CP0 can be derived as the CPMVP of CP0, and the candidate motion vector of CP1 can be derived as the CPMVP of CP1. Furthermore, if the affine MVP candidate includes the candidate motion vectors of CP0, CP1, and CP2, then the candidate motion vector of CP0 can be derived as the CPMVP of CP0, the candidate motion vector of CP1 can be derived as the CPMVP of CP1, and the candidate motion vector of CP2 can be derived as the CPMVP of CP2.
[0364] In S2330, the decoding device derives the control point motion vector difference (CPMVD) of the current block's CP based on motion prediction information. The motion prediction information may include information about the CPVD of each CP, and the decoding device can derive the CPVD of each CP in the current block based on the information about the CPVD of each CP.
[0365] In S2340, the decoding device derives the control point motion vector (CPMV) of the current block's CP based on CPMVP and CPVD. The decoding device can derive the CPMV of each CP based on the CPMVP and CPVD of each CP. For example, the decoding device can derive the CPMV of each CP by adding the CPMVP and CPVD of the CP.
[0366] In the S2350, the decoding device derives the predicted samples of the current block based on the CPMV. The decoding device can derive the motion vector of the current block on a sub-block or sample basis based on the CPMV. In other words, the decoding device can derive the motion vector of each sub-block or each sample of the current block based on the CPMV. The motion vector on a sub-block or sample basis can be derived according to Equation 1 or Equation 3 above. The motion vector can be called an affine motion vector field (MVF) or a motion vector array.
[0367] The decoding device can derive the predicted sample of the current block based on motion vectors at the sub-block or sample level. The decoding device can also derive a reference region within a reference image based on motion vectors at the sub-block or sample level, and generate the predicted sample of the current block based on the reconstructed samples within the reference region.
[0368] In S2360, the decoding device generates a reconstructed image of the current block based on the derived prediction samples. The decoding device can generate the reconstructed image of the current block based on the derived prediction samples. Depending on the prediction mode, the decoding device can directly use the prediction samples as reconstructed samples, or it can generate reconstructed samples by adding the residual samples to the prediction samples. If residual samples of the current block exist, the decoding device can obtain information about the residuals of the current block from the bitstream. Information about the residuals may include the transform coefficients of the residual samples. The decoding device can derive the residual samples (or residual sample array) of the current block based on the residual information. The decoding device can generate reconstructed samples based on the prediction samples and residual samples, and derive the reconstructed block or reconstructed image based on the reconstructed samples. Then, as described above, the decoding device can apply loop filtering processes such as deblocking filtering and / or SAO processing to the reconstructed image as needed to improve subjective / objective image quality.
[0369] Figure 24 An example of a decoding device for performing a video decoding method according to the present disclosure is shown. Figure 23 The method disclosed in the article can be derived from Figure 24 The decoding device disclosed in the document performs the operation. More specifically, for example, Figure 24 The entropy decoder of the decoding device can perform Figure 23 The S2300 steps, Figure 24 The predictor of the decoding device can execute steps S2310 to S2350, and Figure 24 The adder of the decoding device can perform Figure 23 Step S2360. Additionally, although not shown in the figure, the processing for obtaining image information, including information about the residuals of the current block, via the bitstream can be performed by... Figure 24 The entropy decoder of the decoding device is executed, and the processing used to derive the residual samples of the current block based on the residual information can be performed by... Figure 24 The inverse converter of the decoding device is executed.
[0370] According to this disclosure, the efficiency of video coding based on affine motion prediction can be improved.
[0371] Furthermore, according to this disclosure, in deriving the affine MVP candidate list, a constructed affine MVP candidate can only be added when all candidate motion vectors of the CP of the constructed affine MVP candidate are available. This reduces the complexity of the processing for deriving the constructed affine MVP candidate and the processing for constructing the affine MVP candidate list, and improves coding efficiency.
[0372] Furthermore, according to this disclosure, in deriving the affine MVP candidate list, additional affine MVP candidates can be derived based on the candidate motion vectors of the CP derived through the processing used to derive the constructed affine MVP candidates. This reduces the complexity of the processing used to construct the affine MVP candidate list and improves coding efficiency.
[0373] Furthermore, according to this disclosure, in deriving inherited affine MVP candidates, the inherited affine MVP candidates can only be derived by using the upper adjacent block if the upper adjacent block is included in the current CTU. This reduces the storage of the row buffer used for affine prediction and minimizes hardware costs.
[0374] In the above embodiments, although the method is described based on a flowchart using a series of steps or blocks, this disclosure does not limit the specific order of the steps, and some steps may be performed in a different order than the remaining steps or simultaneously with the remaining steps. Furthermore, those skilled in the art will understand that the steps shown in the flowchart are not exclusive and may include other steps, or one or more steps may be deleted from the flowchart without affecting the technical scope of this disclosure.
[0375] The embodiments according to this disclosure can be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional units shown in each figure can be implemented and executed on a computer, processor, microprocessor, controller, or chip. In this case, the information for implementation (e.g., information about instructions) or algorithms can be stored in a digital storage medium.
[0376] Furthermore, the decoding and encoding devices applying the embodiments of this disclosure may include multimedia broadcasting transmitting and receiving devices, mobile communication terminals, home theater video devices, digital cinema video devices, surveillance cameras, video communication devices, real-time communication devices for video communication, mobile streaming devices, storage media, portable video cameras, video-on-demand (VoD) service providers, over-the-top (OTT) video devices, internet streaming service providers, 3D video devices, video telephony devices, transportation terminals (e.g., vehicle terminals, aircraft terminals, and ship terminals), and medical video devices; and may be used to process video signals or data signals. For example, OTT video devices may include game consoles, Blu-ray players, internet-connected televisions, home theater systems, smartphones, tablet computers, digital video recorders (DVRs), etc.
[0377] Furthermore, the processing methods applying the embodiments of this disclosure can be generated in the form of a computer-executable program and can be stored in a computer-readable recording medium. Multimedia data with data structures according to this disclosure can also be stored in a computer-readable recording medium. Computer-readable recording media include all types of storage devices and distributed storage devices in which computer-readable data is stored. Computer-readable recording media can include, for example, Blu-ray discs (BD), Universal Serial Bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disks, and optical data storage devices. Additionally, computer-readable recording media include media implemented in the form of a carrier wave (e.g., transmission via the Internet). Furthermore, bitstreams generated according to the encoding method can be stored in a computer-readable recording medium or transmitted via wired / wireless communication networks.
[0378] Furthermore, embodiments of this disclosure can be implemented as computer program products by forming program code, and according to embodiments of this disclosure, the program code can be executed by a computer. The program code can be stored on a computer-readable medium.
[0379] Figure 25 An example of a content streaming system architecture applying an embodiment of the present disclosure is illustrated.
[0380] The content streaming system using the embodiments of this disclosure may mainly include an encoding server, a streaming server, a network server, a media storage device, a user device, and a multimedia input device.
[0381] An encoding server compresses content input from multimedia input devices such as smartphones, cameras, or camcorders into digital data to generate a bitstream, which is then sent to a streaming server. As another example, if the multimedia input device, such as a smartphone, camera, or camcorder, generates the bitstream directly, the encoding server can be omitted.
[0382] A bit stream can be generated by an encoding method that applies the embodiments of this disclosure or a method for generating bit streams, and the streaming server can temporarily store the bit stream while sending or receiving it.
[0383] The streaming server sends multimedia data to the user's device via a web server based on user requests, and the web server acts as a notification to the user which services are available. If a user requests a service from the web server, the web server sends a request to the streaming server, and the streaming server then sends the multimedia data to the user. In this scenario, the content streaming system may include a separate control server. In this case, the control server is used to control the commands / responses between devices within the content streaming system.
[0384] A streaming server can receive content from media storage and / or encoding servers. For example, when receiving content from an encoding server, the content can be received in real time. In this case, to provide a smooth streaming service, the streaming server can store the bitstream for a predetermined period of time.
[0385] Examples of user devices may include mobile phones, smartphones, laptops, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation terminals, touchscreen PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, and head-mounted displays (HMDs)), digital TVs, desktop computers, and digital signage. Each individual server within the content streaming system can operate as a distributed server, in which case data received from each server can be processed in a distributed manner.
Claims
1. A method of video decoding performed by a decoding device, the method comprising: obtaining, from a bitstream, motion prediction information for a current block; constructing an affine motion vector predictor (MVP) candidate list for the current block; deriving, based on the affine MVP candidate list, a control point motion vector predictor (CPMVP) for a control point (CP) of the current block; deriving, based on the motion prediction information, a control point motion vector difference (CPMVD) for the CP of the current block; deriving, based on the CPMVP and the CPMVD, a control point motion vector (CPMV) for the CP of the current block; deriving, based on the CPMV, a prediction sample for the current block; and generating, based on the derived prediction sample, a reconstructed picture for the current block, wherein the step of constructing the affine MVP candidate list comprises the steps of: checking whether an inherited affine MVP candidate is available, wherein the inherited affine MVP candidate is derived when the inherited affine MVP candidate is available; checking whether a constructed affine MVP candidate is available, wherein the constructed affine MVP candidate is derived when the constructed affine MVP candidate is available, and the constructed affine MVP candidate comprises a candidate motion vector for a CP0 of the current block, a candidate motion vector for a CP1 of the current block, and a candidate motion vector for a CP2 of the current block; deriving a first affine MVP candidate when a number of the derived affine MVP candidates including the inherited affine MVP candidate and the constructed affine MVP candidate is less than 2, wherein the first affine MVP candidate is an affine MVP candidate comprising a particular motion vector as a candidate motion vector for the CP, and the particular motion vector is an available motion vector among the candidate motion vector for the CP0, the candidate motion vector for the CP1, and the candidate motion vector for the CP2. the CP0 represents a top-left position of the current block, the CP1 represents a top-right position of the current block, and the CP2 represents a bottom-left position of the current block; and 2. The video decoding method of claim 1, wherein, the constructed affine MVP candidate is available when the candidate motion vector is available. the candidate motion vector for the CP0 is available when a reference picture of a first block in a first group is the same as a reference picture of the current block, 3. The video decoding method of claim 2, wherein, the candidate motion vector for the CP1 is available when a reference picture of a second block in a second group is the same as the reference picture of the current block, the candidate motion vector for the CP2 is available when a reference picture of a third block in a third group is the same as the reference picture of the current block; and the affine MVP candidate list comprises the constructed affine MVP candidate when the candidate motion vector for the CP0 is available, a candidate motion vector for the CP1 is available, and the candidate motion vector for the CP2 is available. 4. The video decoding method of claim 3, wherein, The first group includes a neighboring block A, a neighboring block B and a neighboring block C; the second group includes a neighboring block D and a neighboring block E; and the third group includes a neighboring block F and a neighboring block G; And When the size of the current block is WxH, and the x component and the y component of the top-left sample position of the current block are 0, the neighboring block A is a block including a sample at coordinate (-1, -1), the neighboring block B is a block including a sample at coordinate (0, -1), the neighboring block C is a block including a sample at coordinate (-1, 0), the neighboring block D is a block including a sample at coordinate (W-1, -1), the neighboring block E is a block including a sample at coordinate (W, -1), the neighboring block F is a block including a sample at coordinate (-1, H-1), and the neighboring block G is a block including a sample at coordinate (-1, H).
5. The video decoding method of claim 4, wherein, The first block is a block that has been first confirmed as a reference picture being identical to a reference picture of the current block while checking neighboring blocks in the first group in a first specific order, The second block is a block that has been first confirmed as a reference picture being identical to a reference picture of the current block while checking neighboring blocks in the second group in a second specific order, and The third block is a block that has been first confirmed as a reference picture being identical to a reference picture of the current block while checking neighboring blocks in the third group in a third specific order.
6. The video decoding method of claim 5, wherein, The first specific order is an order from the neighboring block A to the neighboring block B, then to the neighboring block C, The second specific order is an order from the neighboring block D to the neighboring block E, and The third specific order is an order from the neighboring block F to the neighboring block G.
7. A video encoding method performed by an encoding device, the video encoding method comprising the steps of: constructing an affine motion vector predictor (MVP) candidate list for a current block; deriving a control point motion vector predictor (CPMVP) for a control point (CP) of the current block based on the affine MVP candidate list; deriving a control point motion vector (CPMV) for the CP of the current block; deriving a control point motion vector difference (CPMVD) for the CP of the current block based on the CPMVP and the CPMV; and encoding motion prediction information including information about the CPMVD, wherein the step of constructing the affine MVP candidate list comprises the steps of: checking whether an inherited affine MVP candidate of the current block is available, wherein the inherited affine MVP candidate is derived when the inherited affine MVP candidate is available; checking whether a constructed affine MVP candidate of the current block is available, wherein the constructed affine MVP candidate is derived when the constructed affine MVP candidate is available, and the constructed affine MVP candidate includes a candidate motion vector for CP0 of the current block, a candidate motion vector for CP1 of the current block, and a candidate motion vector for CP2 of the current block; deriving a first affine MVP candidate when the number of derived affine MVP candidates including the inherited affine MVP candidate and the constructed affine MVP candidate is less than 2, wherein the first affine MVP candidate is an affine MVP candidate including a particular motion vector as a candidate motion vector for the CP, and the particular motion vector is an available motion vector among the candidate motion vector for the CP0, the candidate motion vector for the CP1 and the candidate motion vector for the CP2.
8. A non-transitory computer-readable storage medium storing instructions which, when executed by a processor, cause a method to be performed, the method comprising the steps of: constructing an affine motion vector predictor (MVP) candidate list for a current block; deriving a control point motion vector predictor (CPMVP) for a control point (CP) of the current block based on the affine MVP candidate list; deriving a control point motion vector (CPMV) for the CP of the current block; deriving a control point motion vector difference (CPMVD) for the CP of the current block based on the CPMVP and the CPMV; and encoding motion prediction information including information on the CPMVD, wherein the step of constructing the affine MVP candidate list comprises the steps of: checking whether an inherited affine MVP candidate for the current block is available, wherein the inherited affine MVP candidate is derived when the inherited affine MVP candidate is available; checking whether a constructed affine MVP candidate for the current block is available, wherein the constructed affine MVP candidate is derived when the constructed affine MVP candidate is available, and the constructed affine MVP candidate includes a candidate motion vector for a CP0 of the current block, a candidate motion vector for a CP1 of the current block and a candidate motion vector for a CP2 of the current block; deriving a first affine MVP candidate when the number of derived affine MVP candidates including the inherited affine MVP candidate and the constructed affine MVP candidate is less than 2, wherein the first affine MVP candidate is an affine MVP candidate including a particular motion vector as a candidate motion vector for the CP, and the particular motion vector is an available motion vector among the candidate motion vector for the CP0, the candidate motion vector for the CP1 and the candidate motion vector for the CP2.
9. A transmission method for data of an image, the transmission method comprising the steps of: obtaining a bitstream of image information, wherein the bitstream is generated based on the following steps: constructing an affine motion vector predictor (MVP) candidate list for a current block; deriving a control point motion vector predictor (CPMVP) for a control point (CP) of the current block based on the affine MVP candidate list; deriving a control point motion vector (CPMV) for the CP of the current block; deriving a control point motion vector difference (CPMVD) for the CP of the current block based on the CPMVP and the CPMV; and encoding motion prediction information including information about the CPMVD; and transmitting the data including the bitstream, wherein the step of constructing the affine MVP candidate list comprises the following steps: checking whether an inherited affine MVP candidate of the current block is available, wherein the inherited affine MVP candidate is derived when the inherited affine MVP candidate is available; checking whether a constructed affine MVP candidate of the current block is available, wherein the constructed affine MVP candidate is derived when the constructed affine MVP candidate is available, and the constructed affine MVP candidate comprises a candidate motion vector for a CP0 of the current block, a candidate motion vector for a CP1 of the current block, and a candidate motion vector for a CP2 of the current block; deriving a first affine MVP candidate when a number of the derived affine MVP candidates including the inherited affine MVP candidate and the constructed affine MVP candidate is less than 2, wherein the first affine MVP candidate is an affine MVP candidate including a specific motion vector as a candidate motion vector for the CP, and the specific motion vector is an available motion vector among the candidate motion vector for the CP0, the candidate motion vector for the CP1, and the candidate motion vector for the CP2.
Citation Information
Patent Citations
Inter prediction method and apparatus in video coding system
CN108141582A
Image prediction method and related device
CN108271023A