Image encoding / decoding method and device, and recording medium for storing bitstream

Abstract

The present invention provides an image decoding method comprising the steps of: determining a prediction mode of a current block as an affine intra prediction mode; determining a control point block vector of the current block; determining block vectors of sub-blocks of the current block on the basis of the control point block vector of the current block; and predicting the current block by using reference blocks of a current picture indicated by the block vectors of the sub-blocks.

Description

Video encoding / decoding method, device, and recording medium storing bitstream

[0001] The present invention relates to a video encoding / decoding method, a device, and a recording medium storing a bitstream. Specifically, the present invention relates to a video encoding / decoding method and device using an intra-screen prediction method, and a recording medium storing a bitstream.

[0002] Recently, the demand for high-resolution, high-quality images, such as UHD (Ultra High Definition) images, is increasing across various application fields. As image data becomes higher in resolution and quality, the relative amount of data increases compared to conventional image data. Therefore, transmitting image data using existing media such as wired or wireless broadband lines or storing it using existing storage media leads to increased transmission and storage costs. To address these issues arising from the increasing resolution and quality of image data, high-efficiency image encoding / decoding technologies for higher-resolution and higher-quality images are required.

[0003] In video encoding and decoding, intra-prediction is a technique for predicting the current block using previously reconstructed reference pixels from the current picture. Intra-prediction generates a predicted block from the reference pixels of the current picture based on a predefined non-directional mode, directional mode, or intra-block copy mode. Intra-prediction can have lower prediction accuracy than inter-prediction, which can limit encoding efficiency. Therefore, various methods for improving the prediction accuracy of intra-prediction are being discussed.

[0004] The purpose of the present invention is to provide a video encoding / decoding method and device with improved encoding / decoding efficiency.

[0005] In addition, the present invention aims to provide a recording medium storing a bitstream generated by the image decoding method or device provided in the present invention.

[0006] A video decoding method according to one embodiment of the present invention may include a step of determining a prediction mode of a current block as a prediction mode within an affine picture, a step of determining a control point block vector of the current block, a step of determining block vectors of sub-blocks of the current block based on the control point block vector of the current block, and a step of predicting the current block using reference blocks of the current picture indicated by the block vectors of the sub-blocks.

[0007] According to one embodiment, when the determined affine screen prediction mode is an affine screen prediction mode using block vector prediction, the image decoding method may further include a step of generating a list of affine control point block vector predictor candidates of the current block, and the control point block vector of the current block may be determined from a control point block vector predictor selected from the list of affine control point block vector candidates of the current block.

[0008] According to one embodiment, the affine control point block vector predictor candidate list may be characterized by including an inherited predictor candidate derived from a surrounding block to which an affine screen prediction is applied among the surrounding blocks of the current block.

[0009] According to one embodiment, the affine control point block vector predictor candidate list may be characterized by including a combined predictor candidate derived from a combination of block vectors of a plurality of surrounding blocks of the current block.

[0010] According to one embodiment, the affine control point block vector predictor candidate list may be characterized by including a translational motion vector candidate derived from a block vector of a neighboring block of the current block, when the number of inherited predictor candidates and combined predictor candidates is less than the maximum number of the affine control point block vector predictor candidate list.

[0011] According to one embodiment, the image decoding method may further include a step of determining a control point block vector difference value, wherein the control point block vector of the current block is determined based on the selected control point block vector predictor and the control point block vector difference value.

[0012] According to one embodiment, when the determined affine screen prediction mode is an affine screen prediction mode using a merge, the image decoding method may further include a step of generating a merge list within the affine screen of the current block, and the control point block vector of the current block may be determined from a control point block vector selected from the merge list within the affine screen of the current block.

[0013] According to one embodiment, the merge list within the affine screen may be characterized by including an inherited predictor candidate derived from a surrounding block of the current block to which an affine screen prediction has been applied.

[0014] According to one embodiment, the merge list within the affine screen may be characterized by including a combination predictor candidate derived from a combination of block vectors of a plurality of surrounding blocks of the current block.

[0015] According to one embodiment, the merge list within the affine screen may be characterized by including a translational motion vector candidate derived from a block vector of a neighboring block of the current block, when the number of inherited predictor candidates and combined predictor candidates is less than the maximum number of the merge list within the affine screen.

[0016] According to one embodiment, when the determined affine screen prediction mode is a simple affine screen prediction mode, the method may be characterized by deriving a control point block vector of the current block from a plurality of adjacent blocks of the current block.

[0017] According to one embodiment, the control point block vector of the current block is derived from an adjacent block having a block vector among adjacent blocks adjacent to a control point of the control point block vector of the current block, and when the number of adjacent blocks having the block vector is greater than 1, the control point block vector of the current block is derived according to a block vector of an adjacent block having a high priority according to a predetermined search order.

[0018] According to one embodiment, the control point block vector of the current block is derived from an adjacent block to which an affine screen prediction mode is applied among adjacent blocks adjacent to a control point of the control point block vector of the current block, and when the number of adjacent blocks to which the affine screen prediction mode is applied is greater than 1, the control point block vector of the current block is derived according to a block vector of a sub-block of an adjacent block with a high priority according to a predetermined search order.

[0019] According to one embodiment, if there is an adjacent block to which an affine screen prediction mode is applied or an adjacent block to which a block vector exists among adjacent blocks adjacent to a control point of a control point block vector of the current block, the control point block vector of the current block is determined to be available, and if there is neither an adjacent block to which an affine screen prediction mode is applied nor an adjacent block to which a block vector exists among adjacent blocks adjacent to a control point of a control point block vector of the current block, the control point block vector of the current block is determined to be unavailable.

[0020] According to one embodiment, when there are two available control point block vectors, the current block is predicted according to a four-parameter affine model based on the two control point block vectors, and when there are three available control point block vectors, the current block is predicted according to a six-parameter affine model based on the three control point block vectors.

[0021] According to one embodiment, it may be characterized in that if there is less than one available control point block vector, it is determined that the affine screen prediction mode is not applied to the current block.

[0022] A video encoding method according to one embodiment of the present invention may include a step of determining a prediction mode of a current block as a prediction mode within an affine picture, a step of determining a control point block vector of the current block, a step of determining block vectors of sub-blocks of the current block based on the control point block vector of the current block, and a step of predicting the current block using reference blocks of the current picture indicated by the block vectors of the sub-blocks.

[0023] A non-transitory computer-readable recording medium according to one embodiment of the present invention stores a bitstream generated by the image encoding method.

[0024] A transmission method according to one embodiment of the present invention transmits a bitstream generated by the image encoding method.

[0025] The features briefly summarized above of the present invention are merely exemplary aspects of the detailed description of the present invention described below and do not limit the scope of the present invention.

[0026] The present invention proposes various embodiments of an affine intra-screen prediction mode using block vector prediction or merge.

[0027] Furthermore, the present invention proposes various embodiments of a simple affine screen prediction mode.

[0028] According to the above various embodiments, as the prediction accuracy of the prediction mode within the screen is improved, the overall encoding efficiency can be improved.

[0029] Figure 1 is a block diagram showing the configuration according to one embodiment of an encoding device to which the present invention is applied.

[0030] Figure 2 is a block diagram showing the configuration according to one embodiment of a decryption device to which the present invention is applied.

[0031] FIG. 3 is a diagram schematically showing a video coding system to which the present invention can be applied.

[0032] Figure 4 shows a CPMV-based affine motion model for affine inter-screen prediction.

[0033] Figure 5 shows a CPBV-based affine motion model for prediction within an affine screen.

[0034] Figure 6 illustrates the locations of surrounding blocks of the current block referenced in deriving CPBVP candidates according to the inheritance method.

[0035] Figure 7 is a diagram for explaining a method of deriving a CPBVP candidate of a current block from a CPBV of a surrounding block according to an inheritance method.

[0036] Figure 8 illustrates the locations of surrounding blocks of the current block that are referenced when deriving CPBVP candidates according to a combination method.

[0037] Figure 9 shows a CPBV determination method for simple affine screen prediction.

[0038] Fig. 10 illustrates an embodiment of an intra-screen prediction method to which intra-screen prediction mode improvement is applied.

[0039] Figure 11 illustrates an example of a content streaming system to which an embodiment according to the present invention can be applied.

[0040] A video decoding method according to one embodiment of the present invention may include a step of determining a prediction mode of a current block as a prediction mode within an affine picture, a step of determining a control point block vector of the current block, a step of determining block vectors of sub-blocks of the current block based on the control point block vector of the current block, and a step of predicting the current block using reference blocks of the current picture indicated by the block vectors of the sub-blocks.

[0041] The present invention is susceptible to various modifications and embodiments. Therefore, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, but rather to encompass all modifications, equivalents, and substitutes falling within the spirit and scope of the present invention. In the drawings, similar reference numerals designate the same or similar functions throughout. The shape and size of elements in the drawings may be provided by way of example only for clarity. The detailed description of the exemplary embodiments described below refers to the accompanying drawings, which illustrate specific embodiments. These embodiments are described in sufficient detail to enable those skilled in the art to practice the embodiments. It should be understood that the various embodiments, while different, are not necessarily mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the present invention. Furthermore, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be modified without departing from the spirit and scope of the embodiment. Accordingly, the detailed description set forth below is not intended to be taken in a limiting sense, and the scope of the exemplary embodiments, if properly described, is defined only by the appended claims, along with the full scope equivalents to which such claims are entitled.

[0042] In the present invention, terms such as first, second, etc. may be used to describe various components, but the components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term "and / or" includes a combination of multiple related described items or any of multiple related described items.

[0043] The components shown in the embodiments of the present invention are independently depicted to represent different characteristic functions, and do not mean that each component is composed of separate hardware or a single software component. That is, each component is listed and included as a separate component for convenience of explanation, and at least two components among each component may be combined to form a single component, or a single component may be divided into multiple components to perform a function, and such integrated and separate embodiments of each component are also included in the scope of the present invention as long as they do not deviate from the essence of the present invention.

[0044] The terminology used herein is merely used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In addition, some components of the present invention are not essential components that perform essential functions in the present invention and may be optional components merely for performance enhancement. The present invention may be implemented by including only components essential to realizing the essence of the present invention, excluding components used only for performance enhancement, and a structure including only essential components, excluding optional components used only for performance enhancement, is also within the scope of the present invention.

[0045] In an embodiment, the term "at least one" may mean one of a number greater than or equal to 1, such as 1, 2, 3, and 4. In an embodiment, the term "a plurality of" may mean one of a number greater than or equal to 2, such as 2, 3, and 4.

[0046] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In describing the embodiments of this specification, if it is determined that a detailed description of a related known configuration or function may obscure the gist of this specification, the detailed description will be omitted. The same reference numerals will be used for identical components in the drawings, and duplicate descriptions of identical components will be omitted.

[0047] Glossary of Terms

[0048] Hereinafter, “video” may mean a single picture constituting a video, or may refer to the video itself. For example, “encoding and / or decoding of a video” may mean “encoding and / or decoding of a video,” or may mean “encoding and / or decoding of one of the videos constituting the video.”

[0049] Hereinafter, the terms "video" and "movie" may be used interchangeably and have the same meaning. Furthermore, the target image may be an encoding target image, which is the target of encoding, and / or a decoding target image, which is the target of decoding. Furthermore, the target image may be an input image input to an encoding device, or an input image input to a decoding device. Here, the target image may have the same meaning as the current image.

[0050] Hereinafter, “image”, “picture”, “frame” and “screen” may be used with the same meaning and may be used interchangeably.

[0051] Hereinafter, the term "target block" may refer to an encoding target block, which is the target of encoding, and / or a decoding target block, which is the target of decoding. Furthermore, the target block may refer to a current block, which is the target of current encoding and / or decoding. For example, the terms "target block" and "current block" may be used interchangeably and have the same meaning.

[0052] Hereinafter, "block" and "unit" may be used with the same meaning and may be used interchangeably. In addition, "unit" may mean including a luminance component block and a corresponding chroma component block to distinguish it from a block. For example, a coding tree unit (CTU) may be composed of one luma component (Y) coding tree block (CTB) and two chroma component (Cb, Cr) coding tree blocks associated with it.

[0053] Hereinafter, “sample,” “pixel,” and “pixel” may be used interchangeably and have the same meaning. Here, a sample may represent a basic unit that constitutes a block.

[0054] Hereinafter, “inter” and “between screens” may be used interchangeably and have the same meaning.

[0055] Hereinafter, “intra” and “within screen” may be used interchangeably and have the same meaning.

[0056]

[0057] Figure 1 is a block diagram showing the configuration according to one embodiment of an encoding device to which the present invention is applied.

[0058] The encoding device (100) may be an encoder, a video encoding device, or an image encoding device. A video may include one or more images. The encoding device (100) may sequentially encode one or more images.

[0059] Referring to FIG. 1, the encoding device (100) may include an image segmentation unit (110), an intra prediction unit (120), a motion prediction unit (121), a motion compensation unit (122), a switch (115), a subtractor (113), a transformation unit (130), a quantization unit (140), an entropy encoding unit (150), an inverse quantization unit (160), an inverse transformation unit (170), an adder (117), a filter unit (180), and a reference picture buffer (190).

[0060] Additionally, the encoding device (100) can generate a bitstream including encoded information through encoding an input image and output the generated bitstream. The generated bitstream can be stored in a computer-readable recording medium or can be streamed via a wired / wireless transmission medium.

[0061] The video segmentation unit (110) can segment the input video into various forms to increase the efficiency of video encoding / decoding. That is, the input video is composed of multiple pictures, and one picture can be hierarchically segmented and processed for compression efficiency, parallel processing, etc. For example, one picture can be segmented into one or more tiles or slices, which can then be segmented into multiple Coding Tree Units (CTUs). Alternatively, one picture can first be segmented into multiple sub-pictures defined as groups of rectangular slices, and each sub-picture can then be segmented into the tiles / slices. Here, the sub-pictures can be utilized to support the function of partially independently encoding / decoding and transmitting the picture. Since multiple sub-pictures can each be individually restored, there is an advantage of easy editing in applications that configure multi-channel input into a single picture. In addition, tiles can be segmented horizontally to generate bricks. Here, a brick can be utilized as the basic unit of parallel processing within a picture. In addition, one CTU can be recursively split into a quadtree (QT), and the terminal node of the split can be defined as a coding unit (CU). The CU can be split into a prediction unit (PU) and a transformation unit (TU), and prediction and splitting can be performed. Meanwhile, the CU can be utilized as a prediction unit and / or a transformation unit itself. Here, for flexible splitting, each CTU can be recursively split into a multi-type tree (MTT) as well as a quadtree (QT). Splitting of a CTU into a multi-type tree can start from the terminal node of a QT, and the MTT can be composed of a binary tree (BT) and a triple tree (TT).For example, the MTT structure can be divided into vertical binary split mode (SPLIT_BT_VER), horizontal binary split mode (SPLIT_BT_HOR), vertical ternary split mode (SPLIT_TT_VER), and horizontal ternary split mode (SPLIT_TT_HOR). In addition, the minimum block size (MinQTSize) of the quad tree of the luminance block during splitting can be set to 16x16, the maximum block size (MaxBtSize) of the binary tree can be set to 128x128, and the maximum block size (MaxTtSize) of the triple tree can be set to 64x64. In addition, the minimum block size (MinBtSize) of the binary tree and the minimum block size (MinTtSize) of the triple tree can be set to 4x4, and the maximum depth (MaxMttDepth) of the multi-type tree can be set to 4. Additionally, to improve the encoding efficiency of the I slice, a dual tree can be applied that uses different CTU partition structures for the luminance and chrominance components. On the other hand, in the P and B slices, the luminance and chrominance CTBs (Coding Tree Blocks) within the CTU can be partitioned into a single tree that shares the coding tree structure.

[0062] The encoding device (100) may perform encoding on the input image in intra mode and / or inter mode. Alternatively, the encoding device (100) may perform encoding on the input image in a third mode (e.g., IBC mode, Palette mode, etc.) other than the intra mode and inter mode. However, if the third mode has functional characteristics similar to the intra mode or inter mode, it may be classified as intra mode or inter mode for convenience of explanation. In the present invention, the third mode will be classified and described separately only when a specific description is required.

[0063] When the intra mode is used as the prediction mode, the switch (115) can be switched to intra, and when the inter mode is used as the prediction mode, the switch (115) can be switched to inter. Here, the intra mode can mean the intra prediction mode, and the inter mode can mean the inter-screen prediction mode. The encoding device (100) can generate a prediction block for an input block of an input image. In addition, after the prediction block is generated, the encoding device (100) can encode a residual block using the residual of the input block and the prediction block. The input image can be referred to as a current image that is currently a target of encoding. The input block can be referred to as a current block that is currently a target of encoding or an encoding target block.

[0064] When the prediction mode is intra mode, the intra prediction unit (120) can use samples of blocks already encoded / decoded around the current block as reference samples. The intra prediction unit (120) can perform spatial prediction on the current block using the reference samples, and can generate prediction samples for the input block through spatial prediction. Here, intra prediction can mean prediction within the screen.

[0065] As an intra prediction method, non-directional prediction modes such as DC mode and Planar mode, as well as directional prediction modes (e.g., 65 directions) can be applied. Here, the intra prediction method can be expressed as an intra prediction mode or an intra prediction mode.

[0066] When the prediction mode is inter mode, the motion prediction unit (121) can search for an area that best matches the input block from the reference image during the motion prediction process and derive a motion vector using the searched area. At this time, the area can be used as a search area. The reference image can be stored in the reference picture buffer (190). Here, when encoding / decoding for the reference image is processed, it can be stored in the reference picture buffer (190).

[0067] The motion compensation unit (122) can generate a prediction block for the current block by performing motion compensation using a motion vector. Here, inter prediction may mean inter-screen prediction or motion compensation.

[0068] The above motion prediction unit (121) and motion compensation unit (122) can generate a prediction block by applying an interpolation filter to a portion of the reference image when the value of the motion vector does not have an integer value. In order to perform inter-screen prediction or motion compensation, it is possible to determine whether the motion prediction and motion compensation method of the prediction unit included in the corresponding encoding unit is one of Skip Mode, Merge Mode, Advanced Motion Vector Prediction (AMVP) mode, and Intra Block Copy (IBC) mode based on the encoding unit, and inter-screen prediction or motion compensation can be performed according to each mode.

[0069] In addition, based on the above inter-screen prediction method, the AFFINE mode of sub-PU based prediction, the SbTMVP (Subblock-based Temporal Motion Vector Prediction) mode, and the MMVD (Merge with MVD) mode and the GPM (Geometric Partitioning Mode) mode of PU based prediction can be applied. In addition, in order to improve the performance of each mode, the HMVP (History based MVP), the PAMVP (Pairwise Average MVP), the CIIP (Combined Intra / Inter Prediction), the AMVR (Adaptive Motion Vector Resolution), the BDOF (Bi-Directional Optical-Flow), the BCW (Bi-predictive with CU Weights), the LIC (Local Illumination Compensation), the TM (Template Matching), and the OBMC (Overlapped Block Motion Compensation) can be applied.

[0070] The subtractor (113) can generate a residual block using the difference between the input block and the predicted block. The residual block may also be referred to as a residual signal. The residual signal may refer to the difference between the original signal and the predicted signal. Alternatively, the residual signal may be a signal generated by transforming, quantizing, or transforming and quantizing the difference between the original signal and the predicted signal. The residual block may be a residual signal in block units.

[0071] The transform unit (130) can perform a transform on the residual block to generate a transform coefficient and output the generated transform coefficient. Here, the transform coefficient may be a coefficient value generated by performing a transform on the residual block. When the transform skip mode is applied, the transform unit (130) may also skip the transform on the residual block.

[0072] Quantized levels can be generated by applying quantization to transform coefficients or residual signals. In the following embodiments, quantized levels may also be referred to as transform coefficients.

[0073] For example, a 4x4 luminance residual block generated through within-screen prediction can be transformed using a basis vector based on DST (Discrete Sine Transform), and the remaining residual blocks can be transformed using a basis vector based on DCT (Discrete Cosine Transform). In addition, through RQT (Residual Quad Tree) technology, the transform block is divided into a quad tree shape for one block, and after performing transformation and quantization on each transform block divided through RQT, a coded block flag (cbf) can be transmitted to increase encoding efficiency when all coefficients become 0.

[0074] Another alternative is to apply Multiple Transform Selection (MTS) technology, which selectively performs transformation using multiple transformation bases. That is, instead of dividing CUs into TUs via RQT, a Sub-block Transform (SBT) technology can perform a function similar to TU division. Specifically, SBT is applied only to inter-screen prediction blocks, and unlike RQT, it can divide the current block into ½ or ¼ blocks vertically or horizontally, and then perform transformation on only one of the blocks. For example, in a vertically divided block, the transformation can be performed on the leftmost or rightmost block, and in a horizontally divided block, the transformation can be performed on the topmost or bottommost block.

[0075] Additionally, LFNST (Low Frequency Non-Separable Transform), a secondary transform technique that further transforms the residual signal converted to the frequency domain through DCT or DST, can be applied. LFNST additionally performs a transform on the low-frequency region of 4x4 or 8x8 in the upper left, which allows the residual coefficients to be concentrated in the upper left.

[0076] The quantization unit (140) can generate a quantized level by quantizing a transform coefficient or residual signal according to a quantization parameter (QP), and can output the generated quantized level. At this time, the quantization unit (140) can quantize the transform coefficient using a quantization matrix.

[0077] For example, a quantizer with QP values ​​of 0 to 51 can be used. Alternatively, if the image size is larger and high encoding efficiency is required, a QP of 0 to 63 can be used. In addition, a Dependent Quantization (DQ) method that uses two quantizers instead of a single quantizer can be applied. DQ performs quantization using two quantizers (e.g., Q0 and Q1), but even without signaling information about the use of a specific quantizer, the quantizer to be used for the next transform coefficient can be selected based on the current state through a state transition model.

[0078] The entropy encoding unit (150) can generate a bitstream by performing entropy encoding according to a probability distribution on values ​​produced by the quantization unit (140) or coding parameter values ​​produced during the encoding process, and can output the bitstream. The entropy encoding unit (150) can perform entropy encoding on information about image samples and information for decoding the image. For example, the information for decoding the image can include syntax elements, etc.

[0079] When entropy encoding is applied, a small number of bits are allocated to symbols with a high occurrence probability, and a large number of bits are allocated to symbols with a low occurrence probability, thereby representing the symbols, whereby the size of the bit string for the symbols to be encoded can be reduced. The entropy encoding unit (150) can use an encoding method such as exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), or Context-Adaptive Binary Arithmetic Coding (CABAC) for entropy encoding. For example, the entropy encoding unit (150) can perform entropy encoding using a Variable Length Coding / Code (VLC) table. In addition, the entropy encoding unit (150) may perform arithmetic encoding using the binarization method, probability model, and context model derived from the binarization method of the target symbol and the probability model of the target symbol / bin.

[0080] In this regard, when applying CABAC, the table probability update method can be changed to a simple formula-based table update method to reduce the size of the probability table stored in the decryption device. Furthermore, two different probability models can be used to obtain more accurate symbol probability values.

[0081] The entropy encoding unit (150) can change a two-dimensional block form coefficient into a one-dimensional vector form through a transform coefficient scanning method to encode a transform coefficient level (quantized level).

[0082] Coding parameters may include not only information (flags, indexes, etc.) encoded in an encoding device (100) and signaled to a decoding device (200), such as syntax elements, but also information derived during an encoding process or a decoding process, and may mean information necessary when encoding or decoding an image.

[0083] Here, signaling a flag or index may mean that the encoder entropy encodes the flag or index and includes it in the bitstream, and that the decoder entropy decodes the flag or index from the bitstream.

[0084] The encoded current image can be used as a reference image for other images to be processed later. Accordingly, the encoding device (100) can reconstruct or decode the encoded current image again and store the reconstructed or decoded image as a reference image in the reference picture buffer (190).

[0085] The quantized level can be dequantized in the dequantization unit (160) and inversely transformed in the inverse transformation unit (170). The dequantized and / or inversely transformed coefficients can be combined with a prediction block through an adder (117), and a reconstructed block can be generated by combining the dequantized and / or inversely transformed coefficients and the prediction block. Here, the dequantized and / or inversely transformed coefficients refer to coefficients on which at least one of dequantization and inverse transformation has been performed, and may refer to a reconstructed residual block. The dequantization unit (160) and the inverse transformation unit (170) can be performed in the reverse process of the quantization unit (140) and the transformation unit (130).

[0086] The restoration block may pass through a filter unit (180). The filter unit (180) may apply a deblocking filter, a sample adaptive offset (SAO), an adaptive loop filter (ALF), a bilateral filter (BIF), a Luma Mapping with Chroma Scaling (LMCS), etc. as a filtering technique, in whole or in part, to the restoration sample, restoration block, or restoration image. The filter unit (180) may also be referred to as an in-loop filter. In this case, the in-loop filter is also used as a name excluding LMCS.

[0087] A deblocking filter can remove block distortion that occurs at the boundaries between blocks. Whether to apply a deblocking filter to the current block can be determined based on the samples contained in several columns or rows within the block. When applying a deblocking filter to a block, different filters can be applied depending on the required deblocking filtering strength.

[0088] Sample adaptive offset can be used to compensate for encoding errors by adding an appropriate offset value to sample values. Sample adaptive offset can compensate for the offset from the original image on a sample-by-sample basis for deblocked images. This can be done by dividing the samples contained in the image into a fixed number of regions, determining the regions to be offset, and applying the offset to those regions. Alternatively, the offset can be applied by considering the edge information of each sample.

[0089] Bilateral filter (BIF) can also compensate for the offset from the original image on a sample-by-sample basis for the deblocked image.

[0090] An adaptive loop filter can perform filtering based on a comparison between a reconstructed image and the original image. By dividing the samples contained in the image into predetermined groups and determining the filter to be applied to each group, filtering can be performed differentially for each group. Information regarding whether to apply an adaptive loop filter can be signaled for each coding unit (CU), and the shape and filter coefficients of the adaptive loop filter applied to each block can vary.

[0091] In LMCS (Luma Mapping with Chroma Scaling), luma mapping (LM) refers to remapping luminance values ​​through a piece-wise linear model, and chroma scaling (CS) refers to a technique that scales the residual values ​​of chrominance components according to the average luminance value of the prediction signal. In particular, LMCS can be utilized as an HDR correction technique that reflects the characteristics of HDR (High Dynamic Range) images.

[0092] The restored block or restored image that has passed through the filter unit (180) may be stored in the reference picture buffer (190). The restored block that has passed through the filter unit (180) may be a part of the reference image. In other words, the reference image may be a restored image composed of restored blocks that have passed through the filter unit (180). The stored reference image may be used for inter-screen prediction or motion compensation thereafter.

[0093] Figure 2 is a block diagram showing the configuration according to one embodiment of a decryption device to which the present invention is applied.

[0094] The decoding device (200) may be a decoder, a video decoding device, or an image decoding device.

[0095] Referring to FIG. 2, the decoding device (200) may include an entropy decoding unit (210), an inverse quantization unit (220), an inverse transformation unit (230), an intra prediction unit (240), a motion compensation unit (250), an adder (201), a switch (203), a filter unit (260), and a reference picture buffer (270).

[0096] The decoding device (200) can receive a bitstream output from the encoding device (100). The decoding device (200) can receive a bitstream stored in a computer-readable recording medium, or a bitstream streamed through a wired / wireless transmission medium. The decoding device (200) can perform decoding on the bitstream in intra mode or inter mode. In addition, the decoding device (200) can generate a restored image or a decoded image through decoding, and can output the restored image or the decoded image.

[0097] If the prediction mode used for decryption is intra mode, the switch (203) can be switched to intra. If the prediction mode used for decryption is inter mode, the switch (203) can be switched to inter.

[0098] The decoding device (200) can decode the input bitstream to obtain a reconstructed residual block and generate a prediction block. Once the reconstructed residual block and the prediction block are obtained, the decoding device (200) can generate a reconstructed block to be decoded by adding the reconstructed residual block and the prediction block. The block to be decoded may be referred to as a current block.

[0099] The entropy decoding unit (210) can generate symbols by performing entropy decoding according to a probability distribution for the bitstream. The generated symbols may include symbols in the form of quantized levels. Here, the entropy decoding method may be the reverse process of the entropy encoding method described above.

[0100] The entropy decoding unit (210) can change a one-dimensional vector-shaped coefficient into a two-dimensional block-shaped coefficient through a transform coefficient scanning method to decode a transform coefficient level (quantized level).

[0101] The quantized level can be inversely quantized in the inverse quantization unit (220) and inversely transformed in the inverse transformation unit (230). The quantized level can be generated as a restored residual block as a result of performing inverse quantization and / or inverse transformation. At this time, the inverse quantization unit (220) can apply a quantization matrix to the quantized level. The inverse quantization unit (220) and inverse transformation unit (230) applied to the decoding device can apply the same technology as the inverse quantization unit (160) and inverse transformation unit (170) applied to the encoding device described above.

[0102] When intra mode is used, the intra prediction unit (240) can generate a predicted block by performing spatial prediction on the current block using sample values ​​of already decoded blocks surrounding the block to be decoded. The intra prediction unit (240) applied to the decoding device can apply the same technology as the intra prediction unit (120) applied to the encoding device described above.

[0103] When the inter mode is used, the motion compensation unit (250) can generate a prediction block by performing motion compensation using a motion vector and a reference image stored in the reference picture buffer (270) on the current block. The motion compensation unit (250) can generate a prediction block by applying an interpolation filter to a portion of the reference image when the value of the motion vector does not have an integer value. In order to perform motion compensation, it is possible to determine whether the motion compensation method of the prediction unit included in the corresponding encoding unit is skip mode, merge mode, AMVP mode, or current picture reference mode based on the encoding unit, and motion compensation can be performed according to each mode. The motion compensation unit (250) applied to the decoding device can apply the same technology as the motion compensation unit (122) applied to the encoding device described above.

[0104] The adder (201) can add the restored residual block and the predicted block to generate a restored block. The filter unit (260) can apply at least one of an Inverse-LMCS, a deblocking filter, a sample adaptive offset, and an adaptive loop filter to the restored block or restored image. The filter unit (260) applied to the decoding device can apply the same filtering technology as that applied to the filter unit (180) applied to the encoding device described above.

[0105] The filter unit (260) can output a restored image. The restored block or restored image can be stored in the reference picture buffer (270) and used for inter prediction. The restored block that has passed through the filter unit (260) can be a part of the reference image. In other words, the reference image can be a restored image composed of restored blocks that have passed through the filter unit (260). The stored reference image can be used for inter-screen prediction or motion compensation thereafter.

[0106] FIG. 3 is a diagram schematically showing a video coding system to which the present invention can be applied.

[0107] A video coding system according to one embodiment may include an encoding device (10) and a decoding device (20). The encoding device (10) may transmit encoded video and / or image information or data to the decoding device (20) in the form of a file or streaming through a digital storage medium or a network.

[0108] An encoding device (10) according to one embodiment may include a video source generation unit (11), an encoding unit (12), and a transmission unit (13). A decoding device (20) according to one embodiment may include a reception unit (21), a decoding unit (22), and a rendering unit (23). The encoding unit (12) may be referred to as a video / image encoding unit, and the decoding unit (22) may be referred to as a video / image decoding unit. The transmission unit (13) may be included in the encoding unit (12). The reception unit (21) may be included in the decoding unit (22). The rendering unit (23) may include a display unit, and the display unit may be configured as a separate device or an external component.

[0109] The video source generation unit (11) can obtain video / images through a process of capturing, synthesizing, or generating video / images. The video source generation unit (11) can include a video / image capture device and / or a video / image generation device. The video / image capture device can include, for example, one or more cameras, a video / image archive including previously captured video / images, etc. The video / image generation device can include, for example, a computer, a tablet, a smartphone, etc., and can (electronically) generate video / images. For example, a virtual video / image can be generated through a computer, etc., in which case the video / image capture process can be replaced with a process of generating related data.

[0110] The encoding unit (12) can encode the input video / image. The encoding unit (12) can perform a series of procedures such as prediction, transformation, and quantization for compression and encoding efficiency. The encoding unit (12) can output encoded data (encoded video / image information) in the form of a bitstream. The detailed configuration of the encoding unit (12) can also be configured in the same manner as the encoding device (100) of FIG. 1 described above.

[0111] The transmission unit (13) can transmit encoded video / image information or data output in the form of a bitstream to the reception unit (21) of the decoding device (20) via a digital storage medium or a network in the form of a file or streaming. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmission unit (13) can include an element for generating a media file through a predetermined file format and can include an element for transmission via a broadcasting / communication network. The reception unit (21) can extract / receive the bitstream from the storage medium or network and transmit it to the decoding unit (22).

[0112] The decoding unit (22) can decode video / image by performing a series of procedures such as inverse quantization, inverse transformation, and prediction corresponding to the operation of the encoding unit (12). The detailed configuration of the decoding unit (22) can also be configured in the same manner as the decoding device (200) of FIG. 2 described above.

[0113] The rendering unit (23) can render the decrypted video / image. The rendered video / image can be displayed through the display unit.

[0114]

[0115] The present disclosure describes an affine intra-prediction method that extends the prediction technique based on an affine model used in inter-screen prediction to intra-screen prediction. By combining the intra-block copy (IBC) mode or intra-template matching prediction (IntraTMP) mode of intra-screen prediction with the prediction technique based on an affine model, the accuracy of intra-screen prediction can be improved.

[0116] Below, various embodiments of prediction methods within the affine screen are described.

[0117] Affine motion compensation prediction is used to efficiently encode video signals that include motions such as zoom, rotation, and shear. When motions such as zoom and rotation are included, affine motion compensation prediction can be performed according to a four-parameter affine motion model using two control point motion vectors (CPMV). When motions such as zoom, rotation, and shear are included, affine motion compensation prediction can be performed according to a six-parameter affine motion model using three CPMVs.

[0118] Figure 4 illustrates a CPMV-based affine motion model for affine inter-frame prediction. Figure 4(a) illustrates a four-parameter affine motion model using two CPMVs (V0, V1). Figure 4(b) illustrates a six-parameter affine motion model using three CPMVs (V0, V1, V2).

[0119] According to the 4-parameter affine motion model, the motion vector (mv) of the (x, y) pixel position of the current block is based on mathematical expression 1. x , mv y ) can be derived. In mathematical expression 1, W represents the horizontal length of the current block.

[0120] [Mathematical Formula 1]

[0121]

[0122] According to the 6-parameter affine motion model, the motion vector (mv) of the (x, y) pixel position of the current block is based on Equation 2. x , mv y ) can be derived. In mathematical expression 2, W and H represent the horizontal and vertical lengths of the current block, respectively.

[0123] [Equation 2]

[0124]

[0125] As described above, an affine motion model is derived from multiple CPMVs at predetermined locations associated with the current block. Based on the derived affine motion model, a motion vector (MV) can be calculated for each pixel within the current block.

[0126] Motion prediction and motion compensation using pixel-level motion vectors have high complexity. Therefore, motion vectors can be calculated for each sub-block of the current block. At this time, the same motion vector is applied to all pixels of the sub-block. When motion vectors are calculated for each sub-block, the current block can be divided into sub-blocks of size MxN, where M and N are each any positive integer greater than or equal to 1. The size of the sub-block can be fixed to a specific size, but can also be adaptively determined based on the block size, etc. The motion vector of each sub-block can be derived according to an affine motion model based on a predetermined position of each sub-block. The predetermined position can be the upper left position, the middle position, the lower right position, etc. of each sub-block. Motion prediction of each sub-block can be performed based on the derived motion vector.

[0127] According to one embodiment, affine intra prediction can be performed based on an affine motion model derived based on a control point block vector (CPBV).

[0128] Figure 5 illustrates a CPBV-based affine motion model for prediction within an affine scene. Figure 5(a) illustrates a four-parameter affine motion model using two CPBVs (BV0, BV1). Figure 5(b) illustrates a six-parameter affine motion model using three CPBVs (BV0, BV1, BV2).

[0129] According to the 4-parameter affine motion model, the block vector (bv) of the (x, y) pixel position of the current block is based on Equation 3. x , bv y ) can be derived. In mathematical expression 3, W represents the horizontal length of the current block.

[0130] [Equation 3]

[0131]

[0132] According to the 6-parameter affine motion model, the block vector (bv) of the (x, y) pixel position of the current block is based on Equation 4. x , bv y ) can be derived. In mathematical expression 4, W and H represent the horizontal and vertical lengths of the current block, respectively.

[0133] [Equation 4]

[0134]

[0135] As described above, an affine motion model is derived from multiple CPBVs at predetermined locations associated with the current block. Based on the derived affine motion model, a block vector (BV) can be calculated for each pixel within the current block.

[0136] Motion prediction and motion compensation using pixel-level block vectors have high complexity. Therefore, block vectors are calculated in units of sub-blocks of MxN size, and motion prediction and motion compensation of each sub-block can be performed according to the block vector of the calculated sub-block. That is, the current block is divided into sub-blocks of MxN size, and a block vector is derived based on an affine motion model at a predetermined position of each sub-block, so that motion prediction and compensation can be performed for each sub-block. Here, M and N are each any positive integer greater than or equal to 1. The size of the sub-block can be fixed to a specific size, but can also be adaptively determined based on the block size, etc. The predetermined position can be the upper left position, the middle position, the lower right position, etc. of each sub-block. Motion prediction of each sub-block can be performed based on the derived motion vector.

[0137] According to one embodiment, a CPBV representing an affine motion model of a current block can be derived based on a block vector in an intra block copy (IBC) mode or a block vector in an intra template matching prediction (IntraTMP) mode.

[0138] According to one embodiment, a method similar to the CPMV determination method applied in the affine inter-screen mode or the affine merge mode may be applied to determine the CPBV used for prediction within the affine screen.

[0139] Hereinafter, an affine intra-screen prediction method using block vector prediction according to an embodiment is described. Hereinafter, the prediction method is referred to as an affine block vector prediction mode.

[0140] When the affine block vector prediction mode is applied to the current block, an affine model flag indicating whether a 4-parameter affine motion model or a 6-parameter affine motion model is additionally used may be encoded / decoded. In the affine block vector prediction mode, a CPBVP candidate list including at most S CPBV prediction (Control point block vector predictor, CPBVP) candidates may be constructed using neighboring blocks of the current block. Here, S is any positive integer. The CPBVP candidates may be constructed according to an inheritance method and / or a combination method. In addition, the CPBVP candidates may include two or three CPBVs depending on the affine motion model.

[0141] Hereinafter, a method of constructing a CPBVP candidate list according to an inheritance method according to an embodiment is described based on FIGS. 6 and 7.

[0142] According to one embodiment, neighboring blocks of a current block can be searched in a predetermined order. Then, CPBVP candidates of the current block can be derived based on the CPBV of the neighboring blocks, which are prediction modes within the affine screen, among the searched neighboring blocks. Up to S CPBVP candidates can be derived using an inheritance method. Then, the derived CPBVP candidates can be included in a CPBVP candidate list according to the predetermined order. Hereinafter, neighboring blocks referenced in the CPBVP candidate list will be described with reference to FIG. 6.

[0143] FIG. 6 illustrates the locations of surrounding blocks (602, 604, 606, 608, 610) of the current block (600) that are referenced in deriving CPBVP candidates according to the inheritance method.

[0144] According to one embodiment, the surrounding blocks of the current block (600) may be searched in the order of the A0 block (602) and the A1 block (604) of FIG. 6. Then, a CPBVP candidate may be derived from a block on which affine screen prediction is performed among the A0 block (602) and the A1 block (604). Depending on the embodiment, unlike FIG. 6, the A1 block (604) may be located directly below the B2 block (610) or in the middle of the A0 block (602) and the B2 block (610). In addition, depending on the embodiment, the surrounding blocks of the current block (600) may be searched in the order of the A1 block (604) and the A0 block (602).

[0145] According to one embodiment, the surrounding blocks of the current block (600) may be searched in the order of B0 block (606), B1 block (608), and B2 block (610) of FIG. 6. Then, a CPBVP candidate may be derived from a block on which affine screen prediction is performed among the B0 block (606), B1 block (608), and B2 block (610). Depending on the embodiment, unlike FIG. 6, the B1 block (608) may be located immediately to the right of the B2 block (610) or in the middle of the B0 block (606) and the B2 block (610). In addition, depending on the embodiment, the surrounding blocks of the current block (600) may be searched in the order of B2 block (610), B1 block (608), and B0 block (606).

[0146] According to one embodiment, the conditions of the surrounding blocks for deriving CPBVP candidates may be set in various ways depending on the number of candidates in the CPBVP candidate list. For example, if the number of candidates in the CPBVP candidate list is 2, one left CPBVP candidate may be derived from the A0 block (602) and the A1 block (604) to the left of the current block (600), and one upper CPBVP candidate may be derived from the B0 block (606), the B1 block (608), and the B2 block (610) above the current block (600). In this case, if prediction is performed within the affine picture for both the A0 block (602) and the A1 block (604), the left CPBVP candidate may be derived from the block searched first in a predetermined order among the A0 block (602) and the A1 block (604). And if affine screen prediction is performed on two or more blocks among the B0 block (606), the B1 block (608) and the B2 block (610), the upper CPBVP candidate can be derived from the block searched first in a predetermined order among the B0 block (606), the B1 block (608) and the B2 block (610).

[0147] According to another embodiment, if the number of candidates in the CPBVP candidate list is 3 or more, the CPBVP candidate list can be constructed in a predetermined order using the CPBVs of blocks for which prediction is performed within the affine screen among the A0 block (602), the A1 block (604), the B0 block (606), the B1 block (608), and the B2 block (610). At this time, the number, location, and search order of the surrounding blocks of the current block used can be arbitrarily determined.

[0148] Figure 7 is a diagram for explaining a method of deriving a CPBVP candidate of a current block from a CPBV of a surrounding block according to an inheritance method.

[0149] If the A1 block (710) of the current block (700) is predicted within the affine screen as shown in Fig. 7, the CPBVP candidate of the current block (700) can be derived from the CPBV of the A1 block (710) according to Equation 5 or Equation 6. If a 4-parameter affine motion model is applied to the current block (700), the CPBVP(v) of the upper left position according to Equation 5 0x , v 0y ) and CPBVP(v) in the upper right position 1x , v 1y ) can be derived. When a 6-parameter affine motion model is applied to the current block (700), CPBVP(v) at the upper left position according to Equation 6 0x , v 0y ), CPBVP(v) in the upper right position 1x , v 1y ), and CPBVP(v) at the lower left position 2x , v 2y ) can be derived.

[0150] [Equation 5]

[0151]

[0152] [Equation 6]

[0153]

[0154] In mathematical expressions 5 and 6, cW and cH represent the horizontal and vertical lengths of the current block (700), respectively. In addition, nW and nH represent the horizontal and vertical lengths of the surrounding block (710) to which the prediction within the affine screen of the current block (700) is applied.

[0155] Hereinafter, a method of constructing a CPBVP candidate list according to a combination method according to one embodiment is described based on FIG. 8.

[0156] In one embodiment, a CPBVP candidate can be generated by combining block vectors of neighboring blocks of a current block. The combination method can be performed when the CPBVP candidate list generated by the inheritance method cannot be filled with at most S CPBVP candidate candidates.

[0157] FIG. 8 illustrates the locations of surrounding blocks (802, 804, 806, 808, 810, 812, 814) of the current block (800) that are referenced in deriving CPBVP candidates according to the combination method.

[0158] According to one embodiment, in order to determine CPBV1, the presence of block vectors may be searched in the order of B2 block (812), B3 block (814), and A2 block (806) from the upper left of the current block (800). Then, as a result of the search, the block vector of the first block in the order among the blocks in which the block vector exists may be determined as CPBV1. At this time, the order of the blocks searched may be determined differently depending on the embodiment.

[0159] To determine CPBV2, the presence of block vectors may be searched in the order of B1 block (810) and B0 block (808) from the upper right of the current block (800). As a result of the search, the block vector of the first block in the order among the blocks in which a block vector exists may be determined as CPBV2. The order of the blocks searched at this time may vary depending on the embodiment.

[0160] To determine CPBV3, the presence of block vectors may be searched for in the order of A1 block (804) and A0 block (802) from the lower left of the current block (800). As a result of the search, the block vector of the first block in the order among the blocks in which a block vector exists may be determined as CPBV3. The order of the blocks searched for may vary depending on the embodiment.

[0161] If the current block is predicted based on a 4-parameter affine motion model, a CPBVP candidate can be generated by combining {CPBV1, CPBV2}. If the current block is predicted based on a 6-parameter affine motion model, a CPBVP candidate can be generated by combining {CPBV1, CPBV2, CPBV3}.

[0162] If the number of prediction candidates in the CPBVP candidate list generated using the above inheritance method and combination method is less than S, the missing translational motion vector candidates may be generated using CPBV1, CPBV2, and / or CPBV3 to fill the CPBVP candidate list. For example, if a 4-parameter affine motion model is used in the current block, {CPBV1, CPBV1}, {CPBV2, CPBV2}, {CPBV3, CPBV3} may be sequentially added to the CPBVP candidate list. And, if a 6-parameter affine motion model is used in the current block, {CPBV1, CPBV1, CPBV1}, {CPBV2, CPBV2, CPBV2}, {CPBV3, CPBV3, CPBV3} may be sequentially added to the CPBVP candidate list. The order of the added translational motion vector candidates may be arbitrarily changed.

[0163] If the number of prediction candidates in the CPBVP candidate list is less than S even though a translational motion vector candidate is added to the CPBVP candidate list, one or more default block vectors may be added to the CPBVP candidate list. The default block vector may be a zero block vector. Alternatively, the default block vector may be a block vector determined for an upper data unit such as a slice or a picture.

[0164] The encoder can encode CPBVP candidate index information indicating a CPBVP used in the current block among CPBVP candidates in the CPBVP candidate list of the current block. In addition, the encoder can encode CPBV difference information indicating the difference between the actually derived CPBV and the CPBVP.

[0165] The decoder can determine the CPBVP used for the current block among the CPBVP candidates in the CPBVP candidate list of the current block based on the CPBVP candidate index information. In addition, the decoder can restore the CPBV from the CPBVP and the CPBV difference based on the CPBV difference information.

[0166] Hereinafter, an affine intra-prediction using merge method according to one embodiment is described. Hereinafter, the prediction method is referred to as the affine intra-merge mode.

[0167] In the merge mode within the affine screen of the current block, a list of up to T CPBVP candidates can be constructed using the surrounding blocks of the current block, where T is any positive integer greater than or equal to 1. The CPBVP candidates can be constructed according to an inheritance method and / or a combination method. In addition, the CPBVP candidates can include two or three CPBVs according to the affine motion model. The CPBVP candidate list can be called a merge (candidate) list within the affine screen.

[0168] In an inheritance method for constructing an affine screen merge list of an affine screen merge mode, embodiments of an inheritance method for constructing a CPBVP candidate list of an affine block vector prediction mode may be applied. According to an embodiment, the inheritance method of the affine screen merge mode may be determined in the same manner as the inheritance method of the affine block vector prediction mode. Alternatively, according to an embodiment, considering that the number of candidates of the CPBVP candidate list of the affine block vector prediction mode is S and the number of candidates of the affine screen merge list of the affine screen merge mode may be T and may be different from each other, the inheritance method of the affine screen merge mode and the inheritance method of the affine block vector prediction mode may be determined differently from each other.

[0169] In addition, in the combination method for constructing the merge list within the affine screen of the merge mode within the affine screen, embodiments of the combination method for constructing the CPBVP candidate list of the affine block vector prediction mode described based on FIG. 8 may be applied. According to an embodiment, the combination method of the merge mode within the affine screen may be determined in the same manner as the combination method of the affine block vector prediction mode. Alternatively, according to an embodiment, considering that the number of candidates of the CPBVP candidate list of the affine block vector prediction mode is S and the number of candidates of the affine screen merge list of the affine screen merge mode may be T and may be different from each other, the combination method of the merge mode within the affine screen and the combination method of the affine block vector prediction mode may be determined differently from each other.

[0170] And in the merge mode within the affine picture, if the number of prediction candidates of the merge list within the affine picture generated using the inheritance method and the combination method is less than T, the translational motion vector candidates determined from CPBV1, CPBV2 and / or CPBV3 can be filled into the merge list within the affine picture, similarly to the affine block vector prediction mode. And if the number of prediction candidates of the merge list within the affine picture is less than T even though the translational motion vector candidates are added to the merge list within the affine picture, one or more default block vectors can be added to the merge list within the affine picture. The default block vector may be a zero block vector. Or, the default block vector may be a block vector determined for an upper data unit such as a slice or a picture.

[0171] The encoder may encode merge index information indicating a merge candidate used for the current block among the merge candidates in the merge list of the affine screen of the current block in the merge mode within the affine screen. Unlike the affine block vector prediction mode, CPBV difference information may not be encoded.

[0172] The decoder can determine a merge candidate to be used for the current block among the merge candidates in the merge list within the affine screen of the current block based on the merge index information. In addition, the CPBV of the current block can be derived from the CPBV of the merge candidate without decrypting the CPBV difference information.

[0173] The encoder can encode prediction information within the affine picture, which indicates whether a prediction mode within the affine picture is applied. In addition, the encoder can encode prediction method information within the affine picture, which indicates which method among block vector prediction and block merge is used by the prediction mode within the affine picture.

[0174] The decoder can determine whether to apply an affine screen prediction mode and the type of the affine screen prediction mode based on the affine screen prediction information and the affine screen prediction method information.

[0175] Hereinafter, a simple affine intra-prediction method is described. Simple affine intra-prediction has lower complexity than the above-described affine intra-prediction method. Therefore, by applying simple affine intra-prediction instead of affine intra-prediction to the encoder and decoder, the computational burden on the encoder and decoder can be reduced. Furthermore, efficient predictive encoding or predictive decoding can be implemented by adaptively applying either affine intra-prediction or simple affine intra-prediction on a picture or slice basis.

[0176] Figure 9 illustrates a CPBV determination method for simple affine intra-frame prediction. The simple affine intra-frame prediction is performed based on a single CPBV derived from neighboring blocks of the current block, rather than generating a list of CPBVP candidates.

[0177] According to one embodiment, in order to determine CPBV1, the presence of a block vector of each block may be searched in the order of B2 block (912), B3 block (914), and A2 block (906) from the upper left of the current block (900). The block vector may be derived from a neighboring block encoded in an within-screen block copy mode or an within-screen template matching prediction mode. Then, the first block vector available in the order may be set to CPBV1. Whether the block vector is available is determined based on whether the block vector was used to restore the B2 sample (932), B3 sample (934), and A2 sample (926) of the current block. For example, if the block vector was used to restore the B3 sample (934) and the A2 sample (926), the block vector used to restore the B3 sample (934), which is earlier in the order, is determined to be CPBV1.

[0178] According to another embodiment, in order to determine CPBV1, whether the prediction mode within the affine picture is applied to the corresponding blocks in the order of B2 block (912), B3 block (914), and A2 block (906) from the upper left of the current block (900) can be searched. Then, when the first block to which the prediction mode within the affine picture is applied is searched, CPBV1 can be derived from the block vector of a preset region within the first block. The preset region within the B2 block (912), B3 block (914), and A2 block (906) at the upper left can be set as a sub-block of size KxL (B2 sub-block (952), B3 sub-block (954), and A2 sub-block (946) indicated in gray in FIG. 9). Here, K and L are arbitrary positive integers. And the block vector of the sub-block can be determined based on the center position of the sub-block. Therefore, in deriving CPBV1 of the current block from the block vector of the sub-block, CPBV1 of the current block can be determined by applying an affine model of the sub-block to the block vector of the sub-block based on the difference between the center position of the sub-block and the position of the control point of the current block.

[0179] According to another embodiment, in order to determine CPBV1, whether the prediction mode within the affine picture is applied to the corresponding blocks may first be searched in the order of B2 block (912), B3 block (914), and A2 block (906) from the upper left of the current block (900). Then, when the first block to which the prediction mode within the affine picture is applied is searched, the block vector of the preset area within the first block may be set to CPBV1. If there is no block to which the prediction mode within the affine picture is applied among B2 block (912), B3 block (914), and A2 block (906), the presence of block vectors may be searched in the order of B2 block (912), B3 block (914), and A2 block (906). Then, the first block vector available in the order may be set to CPBV1.

[0180] According to one embodiment, the object to be searched for deriving CPBV1 may be determined according to CPBV search information. For example, if the CPBV search information indicates that the presence of block vectors of adjacent blocks is searched for deriving CPBV, the presence of block vectors for B2 block (912), B3 block (914), and A2 block (906) may be searched for in a predetermined search order for deriving CPBV1. If the CPBV search information indicates that the application of prediction mode within an affine picture of adjacent blocks is searched for deriving CPBV, the application of prediction mode within an affine picture may be searched for B2 block (912), B3 block (914), and A2 block (906) for deriving CPBV1.

[0181] According to one embodiment, the CPBV search information may indicate that, in order to derive CPBV, whether the prediction mode is applied to an affine picture of an adjacent block is first searched, and if there is no adjacent block to which the prediction mode is applied within the affine picture, the presence of a block vector of the adjacent block is searched. At this time, in order to derive CPBV1, whether the prediction mode is applied within the affine picture to the B2 block (912), the B3 block (914), and the A2 block (906) may be searched according to a predetermined search order, and if there is no adjacent block to which the prediction mode is applied within the affine picture, the presence of block vectors for the B2 block (912), the B3 block (914), and the A2 block (906) may be searched according to a predetermined search order.

[0182] If there is no block vector that can be derived from the B2 block (912), the B3 block (914), and the A2 block (906), CPBV1 is not defined, and the information indicating whether CPBV1 is available may be set to unavailable. In the above case, CPBV1 may not be used. Conversely, if there is a block vector that can be derived from the B2 block (912), the B3 block (914), and the A2 block (906), CPBV1 is determined from the block vector, and the information indicating whether CPBV1 is available may be set to available.

[0183] Alternatively, if there is no block vector that can be derived from the B2 block (912), the B3 block (914), or the A2 block (906), CPBV1 may be set to have a default value.

[0184] In order to determine CPBV2 of the current block (900), it may first be searched whether there is a block vector applied to the corresponding sample in the order of B1 block (910) and B0 block (908) from the upper right of the current block (900) and / or whether an affine picture prediction mode applied to the corresponding sub-block is applied. Then, the first block vector available according to the order may be set as CPBV2. Alternatively, if the first block to which the affine picture prediction mode is applied is searched, the block vector of a preset area within the first block may be set as CPBV2. The embodiment applied to CPBV1 may also be applied to CPBV2. According to the embodiment, the search order may be B0 block (908) - B1 block (910).

[0185] In order to determine CPBV3 of the current block (900), it may first be searched whether there is a block vector applied to the corresponding sample in the order of A1 block (904) and A0 block (902) at the lower left of the current block (900) and / or whether an affine picture prediction mode applied to the corresponding sub-block is applied. Then, the first block vector available according to the order may be set to CPBV3. Alternatively, if the first block to which the affine picture prediction mode is applied is searched, the block vector of the preset area within the first block may be set to CPBV3. The embodiment applied to CPBV1 may also be applied to CPBV3. According to the embodiment, the search order may be A0 block (902) - A1 block (904).

[0186] Affine prediction within a scene can be performed based on the above-determined multiple CPBVs. If two CPMVs are available from three control points (upper left, upper right, lower left), affine prediction within a scene can be performed using a four-parameter affine motion model. For example, if CPBV1 and CPBV2 are available, affine prediction within a scene can be performed according to an affine motion model based on the two CPMVs. If all three CPMVs (CPBV1, CPBV2, CPBV3) of three control points are available, affine prediction within a scene can be performed using a six-parameter affine motion model.

[0187] If only one CPMV is available from three control points, a general intra-picture block copy prediction can be performed. Alternatively, if zero or one CPMV is available from three control points, it can be decided not to perform simple affine intra-picture prediction without encoding information indicating whether simple affine intra-picture prediction is applied.

[0188] The encoder can encode simple affine prediction information indicating whether simple affine prediction is applied within the picture. The decoder can determine whether simple affine prediction is applied based on the simple affine prediction information. As explained above, when there are 0 or 1 CPMVs available, it can be determined that simple affine prediction is not performed without simple affine prediction information.

[0189] Simple affine intra-frame prediction not only reduces the complexity of the encoder and decoder, but also reduces the amount of information required for the prediction method. Therefore, the introduction of simple affine intra-frame prediction can improve the performance of the encoder and decoder.

[0190] Fig. 10 illustrates an embodiment of an intra-screen prediction method to which intra-screen prediction mode improvement is applied.

[0191] In step 1002, the prediction mode of the current block is determined as the prediction mode within the affine screen.

[0192] According to one embodiment, the affine screen prediction mode may be an affine screen prediction mode using block vector prediction, in which case a list of affine control point block vector predictor candidates of the current block may be generated.

[0193] The above affine control point block vector predictor candidate list may include inherited predictor candidates derived from neighboring blocks of the current block to which affine screen prediction is applied. If the number of derived inherited predictor candidates is less than the maximum number of the affine control point block vector predictor candidate list, the affine control point block vector predictor candidate list may include a combined predictor candidate derived from a combination of block vectors of a plurality of neighboring blocks of the current block. If the number of derived inherited predictor candidates and combined predictor candidates is less than the maximum number of the affine control point block vector predictor candidate list, the affine control point block vector predictor candidate list may include a translational motion vector candidate derived from a block vector of a neighboring block of the current block.

[0194] According to one embodiment, the prediction mode within the affine screen may be a prediction mode within the affine screen using merge, in which case a merge list within the affine screen of the current block may be generated.

[0195] The merge list within the affine picture may include inherited predictor candidates derived from neighboring blocks of the current block to which prediction within the affine picture has been applied. If the number of derived inherited predictor candidates is less than the maximum number of the merge list within the affine picture, the merge list within the affine picture may include combined predictor candidates derived from combinations of block vectors of a plurality of neighboring blocks of the current block. If the number of derived inherited predictor candidates and combined predictor candidates is less than the maximum number of the merge list within the affine picture, the merge list within the affine picture may include translational motion vector candidates derived from block vectors of neighboring blocks of the current block.

[0196] In one embodiment, the affine-in-picture prediction mode may be a simple affine-in-picture prediction mode, in which case a list containing control point block vectors for predicting the current block is not generated. Instead, a single set of control point block vectors generated by combining the block vectors of adjacent blocks of the current block may be used.

[0197] In step 1004, the control point block vector of the current block is determined.

[0198] According to one embodiment, when the prediction mode within the affine screen is an affine prediction mode using block vector prediction, a control point block vector predictor may be selected from a list of affine control point block vector predictor candidates of the current block. Then, the control point block vector of the current block may be determined based on a difference value between the control point block vector predictor and the control point block vector.

[0199] According to one embodiment, when the prediction mode within the affine screen is an affine screen prediction mode using merge, a control point block vector may be selected from the merge list within the affine screen of the current block.

[0200] According to one embodiment, when the prediction mode within the affine screen is a simple prediction mode within the affine screen, the control point block vector of the current block may be derived from an adjacent block having a block vector among adjacent blocks adjacent to a control point of the control point block vector of the current block. In addition, when the number of adjacent blocks having the block vector is greater than 1, the control point block vector of the current block may be derived according to a block vector of an adjacent block having a high priority according to a predetermined search order.

[0201] According to one embodiment, when the affine screen prediction mode is a simple affine screen prediction mode, the control point block vector of the current block may be derived from an adjacent block to which the affine screen prediction mode is applied among adjacent blocks adjacent to a control point of the control point block vector of the current block. In addition, when the number of adjacent blocks to which the affine screen prediction mode is applied is greater than 1, the control point block vector of the current block may be derived according to a block vector of a sub-block of an adjacent block with a high priority according to a predetermined search order.

[0202] According to one embodiment, if there is an adjacent block to which an affine screen prediction mode is applied or an adjacent block to which a block vector exists among the adjacent blocks adjacent to the control point of the control point block vector of the current block, the control point block vector of the current block may be determined to be available. Conversely, if there is neither an adjacent block to which an affine screen prediction mode is applied nor an adjacent block to which a block vector exists among the adjacent blocks adjacent to the control point of the control point block vector of the current block, the control point block vector of the current block may be determined to be unavailable.

[0203] If there are two available control point block vectors, the current block is predicted according to a four-parameter affine model based on the two control point block vectors. And if there are three available control point block vectors, the current block is predicted according to a six-parameter affine model based on the three control point block vectors. In addition, if there is one or less available control point block vectors, it may be determined that the affine screen prediction mode is not applied to the current block.

[0204] In step 1006, block vectors of sub-blocks of the current block are determined based on the control point block vector of the current block.

[0205] In step 1008, the current block is predicted using reference blocks of the current picture indicated by the block vectors of the sub blocks.

[0206] Depending on the prediction method performed in steps 1002 to 1008, the current block can be encoded or decoded. In some embodiments, prediction information within the affine picture, prediction method information within the affine picture, affine model flag, CPBVP candidate index information, CPBV difference information, merge index information, and simple affine picture prediction information are encoded in the encoder according to the prediction result. The information is then decoded in the decoder and can be used to predict the block.

[0207] And the bitstream generated by the encoder according to the prediction method performed in steps 1002 to 1008 can be stored in a recording medium or transmitted outside the encoder.

[0208] FIG. 11 is a drawing exemplifying a content streaming system to which an embodiment according to the present invention can be applied.

[0209] As illustrated in FIG. 11, a content streaming system to which an embodiment of the present invention is applied may largely include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0210] The encoding server compresses content input from multimedia input devices such as smartphones, cameras, and CCTVs into digital data, generates a bitstream, and transmits it to the streaming server. Alternatively, if multimedia input devices such as smartphones, cameras, and CCTVs directly generate bitstreams, the encoding server may be omitted.

[0211] The above bitstream can be generated by an image encoding method and / or an image encoding device to which an embodiment of the present invention is applied, and the streaming server can temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[0212] The streaming server transmits multimedia data to a user device based on a user request via a web server, and the web server can act as an intermediary to inform the user of available services. When a user requests a desired service from the web server, the web server transmits the request to the streaming server, and the streaming server can transmit multimedia data to the user. At this time, the content streaming system may include a separate control server, and in this case, the control server may control commands / responses between each device within the content streaming system.

[0213] The streaming server can receive content from a media repository and / or encoding server. For example, when receiving content from the encoding server, the content can be received in real time. In this case, to provide a smooth streaming service, the streaming server can store the bitstream for a certain period of time.

[0214] Examples of the user devices may include mobile phones, smart phones, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, HMDs), digital TVs, desktop computers, digital signage, etc.

[0215] Each server within the above content streaming system can be operated as a distributed server, in which case data received from each server can be processed in a distributed manner.

[0216] The above embodiments can be performed in the same or corresponding manner in an encoding device and a decoding device. In addition, an image can be encoded / decoded using at least one or a combination of at least one of the above embodiments.

[0217] The order in which the above embodiments are applied may be different in the encoding device and the decoding device. Alternatively, the order in which the above embodiments are applied may be the same in the encoding device and the decoding device.

[0218] The above embodiments can be performed for each of the luminance and chrominance signals. Alternatively, the above embodiments can be performed identically for the luminance and chrominance signals.

[0219] In the above embodiments, the methods are described based on a flowchart as a series of steps or units. However, the present invention is not limited to the order of the steps, and some steps may occur in a different order or simultaneously with other steps described above. Furthermore, those skilled in the art will understand that the steps depicted in the flowchart are not exclusive, and other steps may be included, or one or more steps in the flowchart may be deleted without affecting the scope of the present invention.

[0220] The above embodiments may be implemented in the form of program commands that can be executed by various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, etc., either singly or in combination. The program commands recorded on the computer-readable recording medium may be those specifically designed and constructed for the present invention, or may be known and usable by those skilled in the art of computer software.

[0221] The bitstream generated by the encoding method according to the above embodiment can be stored in a non-transitory computer-readable recording medium. In addition, the bitstream stored in the non-transitory computer-readable recording medium can be decoded by the decoding method according to the above embodiment.

[0222] Here, examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program instructions such as ROMs, RAMs, and flash memories. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. The hardware devices may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

[0223] Although the present invention has been described above with specific details such as specific components and limited examples and drawings, these are provided only to help a more general understanding of the present invention, and the present invention is not limited to the above examples, and those with ordinary knowledge in the technical field to which the present invention pertains can make various modifications and variations from this description.

[0224] Therefore, the idea of ​​the present invention should not be limited to the embodiments described above, and all things that are modified equally or equivalently to the following claims as well as the claims are considered to fall within the scope of the idea of ​​the present invention.

[0225] The present invention can be used in a device for encoding / decoding an image and a recording medium storing a bitstream.

Claims

1. In the video decryption method, A step of determining the prediction mode of the current block as the prediction mode within the affine screen; A step of determining a control point block vector of the current block; A step of determining block vectors of sub-blocks of the current block based on the control point block vector of the current block; An image decoding method comprising a step of predicting the current block using reference blocks of the current picture indicated by the block vectors of the above sub-blocks.

2. In paragraph 1, When the above-determined affine screen prediction mode is an affine screen prediction mode using block vector prediction, the image decoding method further includes a step of generating a list of affine control point block vector predictor candidates of the current block, An image decoding method, characterized in that the control point block vector of the current block is determined from a control point block vector predictor selected from a list of affine control point block vector candidates of the current block.

3. In paragraph 2, The above affine control point block vector predictor candidate list is, An image decoding method characterized in that it includes an inherited predictor candidate derived from a surrounding block to which an affine screen prediction is applied among the surrounding blocks of the current block.

4. In paragraph 2, The above affine control point block vector predictor candidate list is, An image decoding method, characterized in that it includes a combined predictor candidate derived from a combination of block vectors of a plurality of surrounding blocks of the current block.

5. In paragraph 2, The above affine control point block vector predictor candidate list is, An image decoding method characterized in that the method includes a translational motion vector candidate derived from a block vector of a neighboring block of the current block, when the number of inheritance predictor candidates and combination predictor candidates is less than the maximum number of the affine control point block vector predictor candidate list.

6. In paragraph 2, The above video decryption method is, Further comprising a step of determining a control point block vector difference value, An image decoding method, characterized in that the control point block vector of the current block is determined based on a difference value between the selected control point block vector predictor and the control point block vector.

7. In paragraph 1, When the above-determined affine screen prediction mode is an affine screen prediction mode using merge, the image decoding method further includes a step of generating a merge list within the affine screen of the current block, An image decoding method, characterized in that the control point block vector of the current block is determined from a control point block vector selected from a merge list in an affine screen of the current block.

8. In paragraph 7, The merge list in the above affine screen is, An image decoding method characterized in that it includes an inherited predictor candidate derived from a surrounding block to which an affine screen prediction is applied among the surrounding blocks of the current block.

9. In paragraph 7, The merge list in the above affine screen is, An image decoding method, characterized in that it includes a combined predictor candidate derived from a combination of block vectors of a plurality of surrounding blocks of the current block.

10. In paragraph 7, The merge list in the above affine screen is, An image decoding method characterized in that it includes a translational motion vector candidate derived from a block vector of a neighboring block of the current block when the number of inheritance predictor candidates and combination predictor candidates is less than the maximum number of merge lists in the affine screen.

11. In paragraph 1, An image decoding method characterized in that, when the above-determined affine screen prediction mode is a simple affine screen prediction mode, a control point block vector of the current block is derived from a plurality of adjacent blocks of the current block.

12. In paragraph 11, The control point block vector of the current block above is, The control point of the current block is derived from an adjacent block in which a block vector exists among adjacent blocks adjacent to the control point of the control point block vector of the current block, An image decoding method, characterized in that when the number of adjacent blocks in which the above block vector exists is greater than 1, the control point block vector of the current block is derived according to the block vector of the adjacent block with a high priority according to a predetermined search order.

13. In paragraph 11, The control point block vector of the current block above is, The control point of the control point block vector of the current block is derived from an adjacent block to which an affine screen prediction mode is applied among the adjacent blocks. An image decoding method, characterized in that when the number of adjacent blocks to which the prediction mode is applied within the above affine screen is greater than 1, the control point block vector of the current block is derived according to the block vector of the sub-block of the adjacent block with a high priority according to a predetermined search order.

14. In paragraph 11, If there is an adjacent block to which an affine screen prediction mode is applied or an adjacent block with a block vector exists among the adjacent blocks adjacent to the control point of the control point block vector of the current block, the control point block vector of the current block is determined to be available, An image decoding method, characterized in that if there is no adjacent block to which an affine screen prediction mode is applied and no adjacent block having a block vector among the adjacent blocks adjacent to the control point of the control point block vector of the current block, the control point block vector of the current block is determined to be unavailable.

15. In paragraph 14, When there are two available control point block vectors, the current block is predicted according to a four-parameter affine model based on the two control point block vectors. An image decoding method, characterized in that when there are three available control point block vectors, the current block is predicted according to a six-parameter affine model according to three control point block vectors.

16. In paragraph 14, An image decoding method, characterized in that when there is less than or equal to one available control point block vector, it is determined that no affine screen prediction mode is applied to the current block.

17. In a video encoding method, A step of determining the prediction mode of the current block as the prediction mode within the affine screen; A step of determining a control point block vector of the current block; A step of determining block vectors of sub-blocks of the current block based on the control point block vector of the current block; An image encoding method comprising a step of predicting the current block using reference blocks of the current picture indicated by the block vectors of the sub-blocks.

18. In a computer-readable recording medium storing a bitstream generated by a video encoding method, In a video encoding method, A step of determining the prediction mode of the current block as the prediction mode within the affine screen; A step of determining a control point block vector of the current block; A step of determining block vectors of sub-blocks of the current block based on the control point block vector of the current block; A recording medium including a step of predicting the current block using reference blocks of the current picture indicated by the block vectors of the above sub-blocks.

19. A bitstream transmission method for transmitting a bitstream generated by a video encoding method, A step of encoding an image based on the above image encoding method; and Comprising a step of transmitting a bitstream including the encoded image, The above image encoding method is, A step of determining the prediction mode of the current block as the prediction mode within the affine screen; A step of determining a control point block vector of the current block; A step of determining block vectors of sub-blocks of the current block based on the control point block vector of the current block; A bitstream transmission method comprising a step of predicting the current block using reference blocks of the current picture indicated by the block vectors of the above sub-blocks.

Images