Video encoding / decoding method and device for affine-based intra prediction

Abstract

Disclosed are a method and a device for video encoding / decoding in which affine-based intra prediction is performed. The video decoding method of the present disclosure comprises the steps of: determining motion vectors of a control point position of a current block; determining a motion vector of a predetermined data unit in the current block by using the motion vectors of the control point position; and performing intra prediction regarding the predetermined data unit by using the motion vector of the data unit, wherein the motion vector of the control point position is a block vector of a neighboring block including a neighboring pixel of the control point position.

Description

Video encoding / decoding method and device for affine-based intra prediction

[0001] The present disclosure relates to the field of video encoding and decoding. More specifically, the present disclosure relates to a video encoding and decoding method and device that performs intra-prediction, i.e., prediction within a current block based on an affine transform.

[0002] With the development and widespread adoption of hardware capable of playing and storing high-resolution or high-definition video content, the need for codecs that effectively encode or decode this content is increasing. Recently, methods for effectively compressing such high-definition or high-definition video content have been implemented. Representative examples of codecs include High Efficiency Video Coding (HEVC) and Versatile Video Coding (VVC).

[0003] In HEVC, a picture is divided into one or more tiles / slices, which are then divided into multiple CTUs (Coding Tree Units). VVC can first divide a picture into multiple sub-pictures. A sub-picture is defined as a group of rectangular slices and was added to VVC to support the function of partially independently encoding / decoding and transmitting a picture. A sub-picture can be divided into tiles / slices, similar to HEVC. VVC also adds a new picture division structure called brick. Bricks are created by horizontally dividing tiles and are the basic unit of parallel processing. In order to process higher resolution images than HEVC, VVC uses a CTU (Coding Tree Unit) with a maximum size of 256x256, which is 16 times larger than HEVC, as the basic unit of encoding / decoding.

[0004] In intra-block prediction, to remove redundancy within the screen, a prediction block is generated using reconstructed pixels adjacent to the current coding block, and a difference value is generated from the current coding block. Unlike intra-block prediction, inter-block prediction generates a prediction block by searching for the block most similar to the current coding block in the previous or subsequent frame. Similar to inter-block prediction, a technique that searches for and uses the most similar block in the current frame for intra-block prediction is currently being used in standard codecs.

[0005] The present disclosure provides a video encoding / decoding method and device that perform intra prediction based on affine transform.

[0006] According to embodiments of the present disclosure, a motion vector (hereinafter referred to as a "Control Point Motion Vector" or "CPMV") of a control point position (hereinafter referred to as a "CP") of a current block is determined using a block vector of a neighboring block, an affine block vector of a sub-block or pixel located within a current block is obtained using the CP position and the CPMV, and intra-prediction is performed by performing intra-screen prediction on a sub-block or pixel basis using the obtained affine block vector. In the present disclosure, an "affine block vector" is a motion vector that points to a reconstructed neighboring pixel within the same frame as the current block referenced by the current block or a predetermined data unit within the current block. In the present disclosure, a prediction value of the current block is generated by referring to a reference pixel value within the current frame pointed to by the derived affine block vector.

[0007] According to the present disclosure, the compression efficiency of a video can be improved.

[0008] In addition, according to the present disclosure, intra prediction efficiency corresponding to movements such as zoom, rotation, and shear can be improved.

[0009] In addition, according to the present disclosure, the amount of transmission of CPMV information of the current block can be reduced by determining the CPMV of the current block by utilizing the block vector of the surrounding block encoded in the IBC (Intra Block Copy) or IntraTMP (Intra Template Matching Prediction) mode.

[0010] Figure 1 is a block diagram illustrating the configuration and operation of a video encoder for encoding an image.

[0011] Figure 2 is a drawing for explaining an embodiment of a method for dividing blocks of an image.

[0012] Figures 3 and 4 are drawings for explaining embodiments of an intra prediction method.

[0013] Figure 5 is a block diagram illustrating the configuration and operation of a video decoder for decoding an image.

[0014] FIG. 6 is a diagram for explaining an embodiment of an intra prediction method in IBC (Intra Block Copy) mode.

[0015] FIG. 7 is a diagram for explaining an embodiment of an intra prediction method in IntraTMP (Intra Template Matching Prediction) mode.

[0016] FIG. 8 is a reference diagram for explaining a process of deriving four parameters of an affine transformation utilizing four parameters according to one embodiment of the present disclosure.

[0017] FIG. 9 is a reference diagram for explaining a process of deriving six parameters of an affine transformation utilizing six parameters according to one embodiment of the present disclosure.

[0018] FIG. 10 illustrates adjacent neighboring blocks used to determine the CPMV of a current block according to one embodiment of the present disclosure.

[0019] FIG. 11 is a diagram illustrating a process of searching for a block vector of a non-adjacent surrounding block according to one embodiment of the present disclosure.

[0020] FIG. 12 is a diagram illustrating peripheral blocks used to determine the CPMV of a current block, according to one embodiment of the present disclosure.

[0021] FIG. 13 is a reference diagram for explaining a process of determining the CPMV of a current block using a virtual square according to one embodiment of the present disclosure.

[0022] FIG. 14 is a reference diagram for explaining a process of constructing an affine block vector by inheriting the CPMV of a surrounding block according to one embodiment of the present disclosure.

[0023] FIG. 15 is a reference diagram for explaining a process of constructing an affine block vector by inheriting the CPMV of a surrounding block according to another embodiment of the present disclosure.

[0024] FIG. 16a and FIG. 16b are reference diagrams for explaining a process of constructing a CPMV candidate list using block vectors of surrounding blocks according to one embodiment of the present disclosure.

[0025] FIG. 17 is a reference diagram for explaining a process of calculating a template cost using a CPMV candidate according to one embodiment of the present disclosure.

[0026] FIG. 18 is a flowchart of a video encoding method according to one embodiment of the present disclosure.

[0027] FIG. 19 is a flow chart of a video decoding method according to one embodiment of the present disclosure.

[0028] A video decoding method according to one embodiment of the present disclosure includes the steps of: determining motion vectors of control point positions of a current block; determining a motion vector of a predetermined data unit within the current block using the motion vectors of the control point positions; and performing intra prediction on the predetermined data unit using the motion vector of the data unit, wherein the motion vector of the control point position is a block vector of a neighboring block including neighboring pixels of the control point position.

[0029] In one embodiment, a block vector of a block including surrounding pixels of a control point position is a block vector derived for the surrounding block according to the IBC (Intra Block Copy) mode or the IntraTMP (Intra Template Matching Prediction) mode, when the surrounding block is in the IBC mode or the IntraTMP mode.

[0030] In one embodiment, the data unit is one of a sub-block unit of a predetermined size within the current block, a pixel unit, or a block unit of any size or shape.

[0031] In one embodiment, the surrounding pixel used to determine the motion vector of the control point position is one of the surrounding pixels previously processed based on the pixel of the control point position.

[0032] In one embodiment, in the case of a 4-parameter affine model, the control point positions are the upper left position and the upper right position of the current block, and in the case of a 6-parameter affine model, the control point positions are the upper left position, the upper right position, and the lower left position of the current block, and when the control point position of the upper left position is (0, 0), the surrounding pixels used to determine the motion vector of the control point position of the upper left position are the surrounding pixels located at one of the positions (-1, 0), (0, -1), and (-1, -1), and when the control point position of the upper right position is (w, 0), the surrounding pixels used to determine the motion vector of the control point position of the upper right position are the surrounding pixels located at one of the positions (w+1, 0), (w, -1), and (w+1, -1), and when the control point position of the lower left position is (0, h), the motion vector of the control point position of the lower left position is used to determine the A surrounding pixel is a surrounding pixel located at one of the positions (-1, h), (0, h+1), and (-1, h+1).

[0033] In one embodiment, the step of determining motion vectors of the control point position of the current block includes: determining whether an adjacent neighboring block including a neighboring pixel used to determine the motion vector of the control point position has a block vector; determining a motion vector of the control point position using the block vector of the adjacent neighboring block when the adjacent neighboring block has the block vector; and searching for a non-adjacent neighboring block having a block vector along one of a horizontal direction and a vertical direction with respect to the adjacent neighboring block when the adjacent neighboring block does not have the block vector; and determining a motion vector of the control point position of the current block using the block vector of a first non-adjacent neighboring block having the block vector according to the search direction.

[0034] In one embodiment, the step of searching for a non-adjacent neighboring block having the block vector includes searching for a non-adjacent neighboring block having a block vector in an order of proximity to the current block in a horizontal direction from right to left based on the adjacent neighboring block when the control point position is an upper left or lower left position of the current block, and searching for a non-adjacent neighboring block having a block vector in an order of proximity to the current block in a vertical direction from bottom to top based on the adjacent neighboring block when the control point position is an upper right position of the current block.

[0035] In one embodiment, the step of determining the motion vector of the control point position includes determining a nearest left non-adjacent neighboring block having a block vector in a left direction of the current block and being closest to the current block and a nearest upper non-adjacent neighboring block having a block vector in an upper direction of the current block and being closest to the current block, determining a virtual rectangle surrounding the nearest left non-adjacent neighboring block and the nearest upper non-adjacent neighboring block, and using block vector information of a control point position of the virtual rectangle corresponding to the control point position of the current block, determining a motion vector of the control point position of the current block.

[0036] In one embodiment, if there are no adjacent neighboring blocks and non-adjacent neighboring blocks having the block vector, a default vector having a predetermined value is determined as the motion vector of the control point position of the current block.

[0037] In one embodiment, the step of determining motion vectors of control point positions of a current block includes determining one control point motion vector candidate from a list of control point motion vector candidates including at least one of a control point motion vector candidate inherited from an adjacent neighboring block to which an affine-based intra prediction mode is applied, a control point motion vector candidate inherited from a non-adjacent neighboring block to which an affine-based intra prediction mode is applied, a control point motion vector candidate generated using a block vector of an adjacent neighboring block, a control point motion vector candidate generated using a block vector of a non-adjacent neighboring block, a zero-vector control point motion vector candidate, a control point motion vector candidate preset according to a preset rule using an intra prediction mode value, and a history-based control point motion vector candidate obtained based on CPMV information of previously processed previous blocks, and determining a control point motion vector of the current block using a control point motion vector included in the determined control point motion vector candidate.

[0038] In one embodiment, the step of determining a motion vector of a control point position of the current block includes the step of obtaining information indicating a control point motion vector candidate used as a motion vector of the control point position of the current block from among the control point motion vector candidate list from a bitstream; and the step of determining a control point motion vector included in a control point motion vector candidate determined based on the information indicating the control point motion vector candidate as the motion vector of the control point position of the current block.

[0039] In one embodiment, the method further includes a step of correcting a two-dimensional coordinate value of a control point motion vector inherited from a surrounding block by considering a reference point of the current block and a reference point of the surrounding block.

[0040] In one embodiment, the step of determining a motion vector of a control point position of the current block includes the steps of: obtaining information indicating a control point motion vector candidate used as a motion vector of the control point position of the current block from the control point motion vector candidate list from a bitstream and a differential motion vector value; determining a control point motion vector included in a control point motion vector candidate determined based on the information indicating the control point motion vector candidate as a predicted motion vector, and determining a motion vector of the control point position of the current block based on the predicted motion vector and the differential motion vector value.

[0041] In one embodiment, the step of determining a motion vector of a control point position of the current block includes: determining affine parameters by setting the control point motion vector of the determined control point motion vector candidate as the control point motion vector of the current block, and obtaining an affine block vector that points to a reference pixel referenced by pixels of the surrounding template using the determined affine parameters and pixel positions of a surrounding template of the current block; obtaining a prediction value of the surrounding template using the affine block vectors of pixels of the surrounding template; reordering the control point motion vector candidates included in the control point motion vector candidate list in the order of decreasing cost based on a cost that is a difference value between the prediction value of the surrounding template and the surrounding template; and obtaining index information indicating one of the reordered control point motion vector candidates from a bitstream.

[0042] A video encoding method according to one embodiment of the present disclosure includes the steps of: determining motion vectors of control point positions of a current block; determining a motion vector of a predetermined data unit within the current block using the motion vectors of the control point positions; and performing intra prediction on the predetermined data unit using the motion vector of the data unit, wherein the motion vector of the control point position is a block vector of a neighboring block including neighboring pixels of the control point position.

[0043] Hereinafter, a video encoding and decoding method and device according to an embodiment of the present invention will be described in detail with reference to the attached drawings.

[0044] In the following description of the present invention, detailed descriptions of related known functions or configurations will be omitted if they are deemed to unnecessarily obscure the gist of the invention. Furthermore, the terms described below are defined in light of their functions within the present invention and may vary depending on the intent or custom of the user or operator. Therefore, their definitions should be based on the overall content of the present invention.

[0045] In addition, in order to efficiently explain the technical components that make up the present invention, the preferred embodiments of the present invention implemented below omit, as much as possible, the system functional components that are already provided in each system functional configuration or are commonly provided in the technical field to which the present invention belongs, and focus on explaining the functional components that must be additionally provided for the present invention.

[0046] Anyone having ordinary skill in the art to which the present invention pertains will be able to easily understand the functions of components that have been conventionally used among the functional configurations that are not illustrated below, and will also be able to clearly understand the relationship between the components omitted as described above and the components added for the present invention.

[0047] In this specification, a device that encodes an image to generate a video signal bitstream is referred to as an encoding device, an encoding device, or an encoder, and a device that decodes the video signal bitstream to restore an image is referred to as a decoding device, a decoding device, or a decoder.

[0048] A pixel or pel is the smallest unit that constitutes an image, and the terms pixel and sample can be used interchangeably. A sample can generally represent a pixel or a pixel value, or it can represent only a pixel or pixel value of the luminance component, or only a pixel or pixel value of the chroma component.

[0049] In addition, the unit is used to refer to a basic unit of image processing or a specific location of a picture, and represents an image area including at least one of a luminance component and a chrominance component. Specifically, the unit can be used as a concept including a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), and a transform unit (TU). In addition, a block represents an image area including a specific component among luminance components and chrominance components, and an MxN block can represent a set of samples or transform coefficients consisting of M columns and N rows. Here, the terms unit, block, partition, signal, and area may be used interchangeably.

[0050] Meanwhile, a picture refers to a field or a frame, and these can be used interchangeably. For example, if the image is an interlaced image, one frame is divided into an odd (or odd, top) field and an even (or even, bottom) field, and each field is composed of a single picture unit, which can be encoded or decoded. If the image is a progressive image, one frame can be composed as a picture, which can be encoded or decoded.

[0051] FIG. 1 is a block diagram illustrating an encoding device according to an embodiment of the present invention, and is intended to explain the configuration and operation of a video encoder for encoding an image.

[0052] Referring to FIG. 1, a video encoder (100) may be configured to include a transformation unit (110), a quantization unit (120), an inverse quantization unit (130), an inverse transformation unit (140), a filtering unit (150), a prediction unit (160), a DPB (Decoded Picture Buffer, 170), and an entropy coding unit (180).

[0053] The conversion unit (110) converts the residual signal, which is the difference between the input video signal and the prediction signal generated by the prediction unit (160), to obtain a conversion coefficient value.

[0054] For example, the Discrete Cosine Transform (DCT), the Discrete Sine Transform (DST), or the Wavelet Transform can be used.

[0055] The transform kernel used for transforming the residual block may be a transform kernel having separable vertical and horizontal transform properties. In this case, the transform for the residual block may be performed separately as vertical and horizontal transforms. For example, the encoder may perform a vertical transform by applying the transform kernel in the vertical direction of the residual block. Alternatively, the encoder may perform a horizontal transform by applying the transform kernel in the horizontal direction of the residual block.

[0056] Meanwhile, the transform kernel may be used as a term referring to a set of parameters used for transforming the residual signal, such as a transform matrix, a transform array, a transform function, or a transform, and may be any one of a plurality of available kernels, and transform kernels based on different transform types may be used for each of the vertical transform and the horizontal transform.

[0057] The transformation coefficients may be distributed such that higher coefficients are distributed toward the upper left corner of the block, and coefficients closer to '0' are distributed toward the lower right corner of the block. Furthermore, as the current block size increases, there is a possibility that many '0' coefficients exist in the lower right area. To reduce the transformation complexity of large blocks, only the upper left area can be left, and the remaining areas can be reset to '0'.

[0058] Additionally, error signals may exist only in some regions of a coding block, in which case the conversion process may be performed only on some arbitrary regions. For example, in a block of size 2Nx2N, error signals may exist only in the first 2NxN block, in which case the conversion process may be performed only on the first 2NxN block, but the conversion process may not be performed on the second 2NxN block and may not be encoded or decoded.

[0059] The encoder may perform additional transformations before the transform coefficients are quantized. The transformation method described above may be referred to as a primary transform, and the additional transformation may be referred to as a secondary transform.

[0060] The secondary transform can be optional for each residual block. For example, the encoder can improve coding efficiency by performing the secondary transform in areas where it is difficult to focus energy in the low-frequency region using the primary transform alone.

[0061] Specifically, a secondary transformation may be additionally performed on blocks in which residual values ​​appear significantly in directions other than the horizontal or vertical direction of the residual block, and the secondary transformation may not be performed separately into vertical transformation and horizontal transformation, unlike the primary transformation. Such a secondary transformation may be referred to as a low frequency non-separable transform (LFNST).

[0062] The quantization unit (120) quantizes the transformation coefficient value output from the transformation unit (110).

[0063] In order to increase coding efficiency, a method is used in which, instead of coding the picture signal as is, a picture is predicted using an already coded area through a prediction unit (160), and a restored picture is obtained by adding a residual value between the original picture and the predicted picture to the predicted picture.

[0064] To avoid mismatches between the encoder and decoder, when performing prediction in the encoder, information available in the decoder must also be used, and for this purpose, the encoder can perform a process of reconstructing the current block that it has encoded.

[0065] The inverse quantization unit (130) inversely quantizes the transform coefficient values, and the inverse transformation unit (140) restores the residual values ​​using the inverse quantized transform coefficient values.

[0066] The filtering unit (150) performs filtering operations using a deblocking filter, a sample adaptive offset (SAO), an adaptive loop filter (ALF), etc. to improve the quality of the restored picture and enhance encoding efficiency.

[0067] A deblocking filter is a filter for removing distortion within a block generated at the boundary between blocks in a restored picture. The encoder can determine whether to apply a deblocking filter to a boundary based on the distribution of pixels included in several columns or rows based on an arbitrary boundary within a block.

[0068] When a deblocking filter is applied, the filtering unit (150) can apply a long filter, a strong filter, or a weak filter depending on the deblocking filtering strength, and can process horizontal filtering and vertical filtering in parallel.

[0069] Sample Adaptive Offset (SAO) can be used to correct the offset from the original image on a pixel-by-pixel basis for a residual block to which a deblocking filter is applied. In order to correct the offset for a specific picture, the filtering unit (150) can use a method (Band Offset) that divides the pixels included in the image into a certain number of regions, determines the regions to perform offset correction, and applies the offset to the regions. In addition, the filtering unit (150) can use a method (Edge Offset) that applies the offset by considering the edge information of each pixel.

[0070] Adaptive Loop Filtering (ALF) is a method that divides pixels in an image into predetermined groups, determines a filter to be applied to each group, and performs differential filtering for each group. Information regarding whether to apply an adaptive loop filter can be signaled on a coding unit basis, and the shape and filter coefficients of the ALF filter to be applied can vary depending on the block. Furthermore, the same adaptive loop filter can be applied regardless of the characteristics of the target block.

[0071] The filtered picture can be stored in DPB (170) to be used as a reference picture.

[0072] The prediction unit (160) includes an intra / IBC prediction unit (161) and an inter prediction unit (165). The intra / IBC prediction unit (161) performs intra prediction, IntraTMP prediction, IBC prediction, and affine-based intra prediction within the current picture, and the inter prediction unit (165) performs inter prediction that predicts the current picture using a reference picture stored in the DPB (170).

[0073] The intra / IBC prediction unit (161) performs intra prediction from restored areas within the current picture and transmits intra encoding information to the entropy coding unit (180). Here, the intra encoding information may include at least one of an intra prediction mode, an MPM (Most Probable Mode) flag, an MPM index, and information about a reference sample.

[0074] In addition, the intra / IBC prediction unit (161) can perform IBC (Intra BLock Copy) prediction from restored samples within the current picture and transfer the IBC encoding information to the entropy coding unit (180). In this case, the intra / IBC prediction unit (161) can obtain a block vector value indicating a reference region used for prediction of the current region by referring to a specific region within the current picture, and can perform IBC prediction using the obtained block vector value.

[0075] In addition, the intra / IBC prediction unit (161) can perform IntraTMP prediction, which generates prediction information by searching for a reference block on its own through a template matching process from restored samples within the current picture. In IntraTMP mode, the use of a reference block within the same frame as the IBC mode as prediction information for the target block is the same as the IBC mode, but the block vector (BV), which is motion information for the target block, is not transmitted to the decoding device, and the decoding device can search for the reference block through the same template matching process as the encoding side to generate prediction information.

[0076] In addition, the intra / IBC prediction unit (161) can perform "affine-based intra prediction" in which a CPMV is determined using a block vector of a surrounding block of the current block, an affine block vector in a sub-block unit or pixel unit within the current block is determined using the determined CPMV, and intra prediction is performed using surrounding pixels indicated by the affine block vector within a frame including the current block.

[0077] The inter prediction unit (165) refers to a specific area of ​​the restored reference picture to find the part most similar to the current area, obtains a motion vector value which is the distance between the areas, and transmits motion information (reference direction indication information (L0 prediction, L1 prediction, bidirectional prediction), reference picture index, motion vector information, etc.) for the obtained reference area to the entropy coding unit (180).

[0078] Additionally, the inter prediction unit (165) performs motion compensation using motion information to generate a prediction block for the current block, and transmits inter encoding information including motion information for the reference area to the entropy coding unit (180).

[0079] Meanwhile, the quantized transform coefficients in the form of a two-dimensional array can be rearranged into a one-dimensional array for entropy coding.

[0080] The method of scanning the quantized transform coefficients can be determined according to the size of the transform block and the intra prediction mode, and diagonal, vertical, and horizontal scans can be applied, and the scan information can be signaled on a block-by-block basis or derived from the decoder according to a set rule.

[0081] The entropy coding unit (180) generates a bitstream by entropy coding information representing quantized transform coefficients, intra-coding information, and inter-coding information, and for this purpose, a variable length coding (VLC) method and an arithmetic coding method can be used.

[0082] Variable-length coding (VLC) converts input symbols into a series of codewords, each of which can be of variable length. For example, frequently occurring symbols can be represented by shorter codewords, while less frequently occurring symbols can be represented by longer codewords.

[0083] As a variable length coding method, a context-based adaptive variable length coding (CAVLC) method can be used.

[0084] Arithmetic coding converts consecutive data symbols into a single prime number using the probability distribution of each data symbol, thereby obtaining the optimal prime number bits required to express each symbol.

[0085] As an arithmetic coding method, the context-based adaptive binary arithmetic code (CABAC) method can be used.

[0086] The generated bitstream is encapsulated into NAL (Network Abstraction Layer) units as basic units.

[0087] NAL units are divided into VCL (Video Coding Layer) NAL units containing video data and non-VCL NAL units containing parameter information for decoding video data, and various types of VCL or non-VCL NAL units may exist.

[0088] A NAL unit consists of NAL header information and data, a Raw Byte Sequence Payload (RBSP). The NAL header information includes summary information about the RBSP. The RBSP of a VCL NAL unit contains an integer number of encoded coding tree units.

[0089] In order to decode a bitstream in a decoder, the bitstream must first be divided into NAL units, and then each divided NAL unit must be decoded. Meanwhile, the information required for decoding the bitstream can be transmitted as part of a picture parameter set (PPS), a sequence parameter set (SPS), a video parameter set (VPS), etc.

[0090] Meanwhile, the configuration and operation of the encoder described with reference to FIG. 1 are according to one embodiment of the present invention, and some configurations may be omitted or added as needed.

[0091] Additionally, a single picture may be encoded by dividing it into sub-pictures, slices, tiles, etc. A sub-picture may include one or more slices or tiles. When a single picture is encoded by dividing it into multiple slices or tiles, all slices or tiles within the picture must be decoded before it can be displayed on the screen.

[0092] When a single picture is encoded into multiple subpictures, only any subpicture can be decoded and displayed on the screen. A slice can contain multiple tiles or subpictures, and a tile can contain multiple subpictures or slices.

[0093] Subpictures, slices, and tiles can be encoded or decoded independently of each other, which is effective for parallel processing and processing speed improvement, but the amount of bits may increase because the encoded information of adjacent subpictures, slices, and tiles cannot be used.

[0094] And subpictures, slices, and tiles can be encoded by being divided into multiple coding tree units (CTUs).

[0095] A coding tree unit can be composed of a luminance coding tree block (CTB) of size 128x128 and two chrominance coding tree blocks of size 64x64.

[0096] A single coding tree unit may be undivided and constitute a single coding unit (CU) itself, or may be divided into multiple coding units as illustrated in Fig. 2. A coding unit may be composed of a luminance coding block (CB) and two chrominance coding blocks.

[0097] A coding unit may consist of one transform unit (TU), or may be split into multiple transform units. A transform unit may consist of a luminance transform block (TB) and two chrominance transform blocks.

[0098] Here, a coding unit represents a basic unit for processing a picture in the process of prediction, transformation, quantization, entropy coding, and decoding, and the size and shape of a coding unit within a picture may not be constant.

[0099] A coding unit may have a square or non-square shape, and a rectangular coding unit may include a vertical coding unit whose height is greater than its width and a horizontal coding unit whose width is greater than its height.

[0100] The coding tree unit is first partitioned into a Quad Tree (QT) structure, such that a single node of size 2NX2N can be partitioned into four nodes of size NXN. Furthermore, the Quad Tree partitioning can be performed recursively, and not all nodes need to be partitioned to the same depth.

[0101] Leaf nodes of a quadtree can be further partitioned into a multi-type tree (MTT) structure. For example, in a multi-type tree structure, a single node can be partitioned into a binary or ternary tree structure with horizontal or vertical partitioning. Accordingly, a multi-type tree structure can have four partitioning structures: vertical binary partitioning, horizontal binary partitioning, vertical ternary partitioning, and horizontal ternary partitioning.

[0102] In each tree structure, both the width and height of a node can have a power of 2. For example, in a binary tree (BT) structure, a node of size 2NX2N can be split into two NX2N nodes by vertical binary splitting, and into two 2NXN nodes by horizontal binary splitting.

[0103] Also, in the Ternary Tree (TT) structure, a node of size 2NX2N can be split into nodes of size (N / 2)X2N, NX2N, and (N / 2)X2N by vertical ternary splitting, and into nodes of size 2NX(N / 2), 2NXN, and 2NX(N / 2) by horizontal ternary splitting. This multi-type tree splitting can be performed recursively.

[0104] A leaf node of a multi-type tree can be a coding unit. If the coding unit is no larger than the maximum transformation length, the coding unit can be used as a unit for prediction and transformation without further splitting. On the other hand, if the width or height of the coding unit is larger than the maximum transformation length, the coding unit can be split into multiple transformation units without explicit signaling regarding the splitting.

[0105] The tree partitioning structure as described above may have the same shape (Single Tree) for the luminance block and the chrominance block, or different shapes (Dual Tree) for the luminance block and the chrominance block.

[0106] Meanwhile, the block division from the coding tree unit (CTU) to the coding unit (CU) as described above can be performed through a rate-distortion optimization (RDO) process, whereby the division structure with the smallest rate-distortion cost (RD cost) value within the allowable size and depth conditions is selected to determine the final coding unit.

[0107] Hereinafter, embodiments of the intra prediction method will be described in more detail with reference to FIGS. 3 and 4.

[0108] For intra prediction, intra prediction mode information may be signaled, and the intra prediction mode information may indicate any one of a plurality of intra prediction modes. The plurality of intra prediction modes may include a directional intra prediction mode, an IntraTMP prediction mode, an IBC prediction mode, and an affine-based intra prediction as illustrated in FIG. 3. The video encoder (100) may include intra prediction mode information applied to a current block in a bitstream and transmit it to the video decoder (200), and the video decoder (200) may parse the intra prediction information included in the bitstream to determine the intra prediction mode of the current block. The affine-based intra prediction process according to the present disclosure, which will be described below, may be performed when the prediction mode information of the current block is set to the affine-based intra prediction mode.

[0109] As illustrated in FIG. 3, the directional intra prediction mode may include a planar mode, a DC mode, and 65 directional modes, and each intra prediction mode may be indicated through an intra prediction mode index.

[0110] Intra prediction mode index "0" indicates planar mode, intra prediction mode index "1" indicates DC mode, and intra prediction mode indexes "2" to "66" can indicate different directional modes, respectively.

[0111] The directional modes each indicate different angles within a preset angular range, for example, a directional mode can indicate an angle within an angular range from 45 degrees to -135 degrees clockwise.

[0112] In this case, the intra prediction mode index "2" may indicate the Horizontal Diagonal (HDIA) mode, the intra prediction mode index "18" may indicate the Horizontal (HOR) mode, the intra prediction mode index "34" may indicate the Diagonal (DIA) mode, the intra prediction mode index "50" may indicate the Vertical (VER) mode, and the intra prediction mode index "66" may indicate the Vertical Diagonal (VDIA) mode.

[0113] If the current block is a non-square block, 20 additional wide angular modes can be used, indicating angles greater than 45 degrees clockwise or less than -135 degrees.

[0114] Based on the intra prediction mode information as described above, reference samples to be used for intra prediction for the current block are determined.

[0115] For example, if the intra prediction mode index indicates a specific directional mode, the reference sample corresponding to the corresponding angle from the current sample of the current block is used for prediction of the current sample. For intra prediction, surrounding already reconstructed samples are used as reference samples, and the reference samples may be reconstructed samples located to the left or above the current block.

[0116] As another example, if the prediction mode of the current block is an affine-based intra prediction mode, an affine block vector in units of sub-blocks or pixels within the current block is determined using a CPMV determined using a block vector of a neighboring block of the current block, and neighboring pixels pointed to by the affine block vector within the frame including the current block can be used as reference samples.

[0117] Referring to FIG. 4, the reference samples may be samples adjacent to the left boundary and upper boundary of the current block.

[0118] For example, if the size of the current block is NxN and samples of a single reference line adjacent to the current block are used for intra prediction, reference samples can be set using (2N*2+1) surrounding samples located on the left (L, Left), top (T, Top), and top-left (TL, Top-left) of the current block.

[0119] Meanwhile, samples of multiple reference lines (MRL) may be used for intra prediction of the current block, and the multiple reference lines may be composed of n reference lines located within a preset range from the current block. In this case, separate reference line index information indicating the reference lines to be set as reference pixels may be signaled.

[0120] Additionally, if at least some of the samples to be used as reference samples have not yet been restored, reference samples can be obtained through a reference sample padding process, and a reference sample filtering process can be performed to reduce errors in intra prediction.

[0121] FIG. 5 is a block diagram illustrating a decoding device according to an embodiment of the present invention, and is intended to explain the configuration and operation of a video decoder for decoding an image.

[0122] Referring to FIG. 5, a video decoder (200) may be configured to include an entropy decoding unit (210), an inverse quantization unit (220), an inverse transformation unit (230), a filtering unit (240), a prediction unit (260), and a DPB (Decoded Picture Buffer, 270).

[0123] The entropy decoding unit (210) entropy decodes the bitstream to extract transform coefficient information, intra-coding information, inter-coding information, etc. for each region.

[0124] For example, the entropy decoding unit (210) can obtain a binary code for transform coefficient information of a specific area from a bitstream, and can obtain a quantized transform coefficient by de-binarizing the binary code.

[0125] The inverse quantization unit (220) inversely quantizes the quantized transform coefficients, and the inverse transform unit (230) restores the residual value using the inverse quantized transform coefficients. The inverse transform unit (230) can obtain the residual value by additionally performing a second inverse transform after performing a first inverse transform on a transform block including the inverse quantized transform coefficients.

[0126] Meanwhile, the residual value obtained from the inverse transformation unit (230) is combined with the predicted value obtained from the prediction unit (260) to restore the original pixel value.

[0127] The filtering unit (240) performs a filtering operation using a deblocking filter, a sample adaptive offset, an adaptive loop filter, etc. to improve the image quality of the restored picture, and the filtered picture can be output or stored in the DPB (270) to be used as a reference picture for the next picture.

[0128] The prediction unit (260) includes an intra / IBC prediction unit (261) and an inter prediction unit (265), and generates a prediction picture by utilizing the encoding type decoded through the entropy decoding unit (210), the transform coefficient for each region, and the intra / inter encoding information.

[0129] To restore the current block in which decoding is performed, the decoded region of the current picture or other pictures containing the current block can be used. A picture (or tile / slice) that performs intra prediction or intra BC prediction using only the current picture for restoration is called an intra picture or I picture (or tile / slice), and a picture (or tile / slice) that can perform all of intra prediction, inter prediction, and intra BC prediction is called an inter picture (or tile / slice).

[0130] Meanwhile, a picture (or tile / slice) that uses at most one motion vector and reference picture index to predict sample values ​​of each block among inter-pictures (or tiles / slices) is called a predictive picture or P-picture (or tile / slice), and a picture (or tile / slice) that uses at most two motion vectors and reference picture indices is called a bi-predictive picture or B-picture (or tile / slice).

[0131] That is, a P picture (or tile / slice) uses at most one motion information set to predict each block, and a B picture (or tile / slice) uses at most two motion information sets to predict each block. Here, a motion information set may include one or more motion vectors and one reference picture index.

[0132] The intra / IBC prediction unit (261) generates a prediction block using intra encoding information and restored samples within the current picture, and the intra encoding information may include at least one of an intra prediction mode, an MPM (Most Probable Mode) flag, and an MPM index.

[0133] The intra / IBC prediction unit (261) can predict sample values ​​of the current block using restored samples located on the left and / or upper side of the current block as reference samples.

[0134] For example, the reference samples may be samples adjacent to the left boundary of the current block and / or samples adjacent to the upper boundary, and may be samples located on a line within a preset distance from the left boundary of the current block among samples of neighboring blocks of the current block and / or samples located on a line within a preset distance from the upper boundary of the current block. In this case, the neighboring blocks of the current block may include at least one of a left (L) block, an upper (A) block, a below left (BL) block, an above right (AR) block, or an above left (AL) block adjacent to the current block.

[0135] Additionally, the intra / IBC prediction unit (261) can restore the current region by referring to a specific region including restored samples within the current picture. To this end, the intra / IBC prediction unit (261) can perform IBC prediction using IBC encoding information obtained from the entropy decoding unit (210), and the IBC encoding information can include block vector information.

[0136] Additionally, the intra / IBC prediction unit (261) can perform IntraTMP prediction, which generates prediction information by searching for a reference block on its own through a template matching process from restored samples within the current picture.

[0137] In addition, if the prediction mode of the current block is determined to be an affine-based intra prediction mode, the intra / IBC prediction unit (261) can determine an affine block vector in units of sub-blocks or pixels within the current block using a CPMV determined using a block vector of a surrounding block, and perform intra prediction using surrounding pixels indicated by the affine block vector within a frame including the current block.

[0138] The inter prediction unit (265) generates a prediction block using the reference picture and inter encoding information stored in the DPB (270), and the inter encoding information may include a set of motion information (reference picture index, motion vector information, etc.) of the current block for the reference block.

[0139] Meanwhile, inter prediction can include L0 prediction, L1 prediction, and bidirectional prediction (Bi-prediction).

[0140] L0 prediction refers to prediction using a single reference picture included in the L0 picture list, and L1 prediction refers to prediction using a single reference picture included in the L1 picture list. For this, a set of motion information (e.g., motion vector and reference picture index) may be required.

[0141] Bidirectional prediction can utilize up to two reference regions, which may reside in the same reference picture or in different pictures. Accordingly, among the two sets of motion information used in bidirectional prediction, two motion vectors may correspond to the same reference picture index or to different reference picture indices.

[0142] At this time, the reference pictures are pictures that are located temporally before or after the current picture, and may be completed pictures that have already been restored, and the two reference areas used in the bidirectional prediction method may be areas selected from the L0 picture list and the L1 picture list, respectively.

[0143] The inter prediction unit (265) can obtain a reference block of the current block using a motion vector and a reference picture index, and the reference block exists in a reference picture corresponding to the reference picture index.

[0144] Additionally, the sample values ​​of a block specified by a motion vector or their interpolated values ​​can be used as a predictor of the current block. For motion prediction with subpel-level pixel accuracy, an 8-tap interpolation filter can be used for the luminance signal, and a 4-tap interpolation filter can be used for the chrominance signal.

[0145] Meanwhile, the configuration and operation of the decoder described with reference to FIG. 5 are according to one embodiment of the present invention, and some configurations may be omitted or added as needed, and the decoder may decode an image by performing the reverse process of the encoding method of the encoder described above.

[0146] FIG. 6 illustrates an embodiment of an intra prediction method in IBC (Intra Block Copy) mode.

[0147] Referring to Figure 6, in the IBC mode, prediction information of the current block, which is the encoding target block, is obtained from a reference block that has already been decoded and is located within the same frame.

[0148] At this time, the vector information from the current block, which is the encoding target block, to the reference block is referred to as a block vector (BV).

[0149] Although the IBC prediction method described above is an intra-frame prediction method, it is similar to an inter-frame prediction method in that it derives and transmits motion information of the current block to be encoded.

[0150] The IBC prediction method can be divided into ibc_skip / merge mode and ibc_amvp mode, and ibc_skip / merge and ibc_amvp can use skip, merge, AMVP (Advanced Motion Vector Prediction) and similar encoding, information transmission and decoding methods in inter-frame prediction.

[0151] In ibc_skip / merge mode, the block vector (BV) value of the current block can be derived using the merge_idx information. In ibc_amvp mode, the block vector (BV) value of the corresponding block can be derived using the mvd, mvp, and amvr values.

[0152] FIG. 7 is a diagram illustrating an embodiment of an intra prediction method in IntraTMP (Intra Template Matching Prediction) mode.

[0153] As described above, in IBC mode, a block vector (BV), which is motion information for a target block, is derived and transmitted to the decoding device to use a reference block within the same frame as prediction information for the current block.

[0154] Meanwhile, in IntraTMP mode, the use of a reference block within the same frame as the prediction information for the target block is the same as in IBC mode, but the block vector (BV), which is motion information for the target block, is not transmitted to the decoding device, and the decoding device can find the prediction information by searching for the reference block on its own through a template matching process.

[0155] Referring to FIG. 7, a template area is defined around a target block (IntraTMP block), and a template area (Best matching Template) that most closely matches the template area of ​​the target block in the reconstructed area of ​​blocks within the same frame is searched for, thereby obtaining a reference block (ref. block).

[0156] And, based on the position of the reference block (ref. block) obtained as described above, the block vector (intraTMP BV) value for (IntraTMP block) can be derived.

[0157] As described above, IntraTMP prediction is one of the intra-frame prediction modes, in which a block vector (BV) of a target block can be derived in a decoding device, and the block vector (BV) value can be used for intra prediction in the same manner as the block vector (BV) value of the IBC mode.

[0158] Meanwhile, the IntraTMP method as described above can also be used in the process of refining or compensating block vector (BV) values.

[0159] For example, for a block encoded in IBC mode, after finding the location of the initial reference block using the transmitted block vector (BV) value, an updated reference block can be obtained by more precisely correcting the block vector (BV) value through template matching in a certain area around the initial reference block.

[0160] Hereinafter, an affine-based intra prediction operation according to embodiments of the present disclosure will be described in detail. The prediction operation according to the affine-based intra prediction mode described below can be performed by the intra / IBC prediction unit (161) of the video encoder (100) illustrated in FIG. 1 or the intra / IBC prediction unit (261) of the video decoder (200) illustrated in FIG. 5.

[0161] The present disclosure relates to an affine-based intra prediction technique for improving intra prediction encoding according to motions such as zoom, rotation, and shear. The affine-based intra prediction of the present disclosure can be performed on a current block, a sub-block of a predetermined size within the current block, or on a pixel basis.

[0162] To perform affine-based intra prediction, affine transformation formulas such as Equations 1 and 2 are used. The affine transformation using 4 parameters uses Equation 1, and the affine transformation using 6 parameters uses Equation 2. In Equations 1 and 2, (x, y) represents the position of the current pixel, and (x', y') represents the position of the reference pixel within the frame referenced by the current pixel.

[0163]

[0164]

[0165] The four parameters (a, b, e, f) of the affine transform utilizing the four parameters can be derived using the positions of two control points (CP) of the current block and the CPMV of the two CPs. When the size of the current block is W x H (W, H are positive integers), the two control points can be the upper left (0,0) position of the current block and the upper right (W, 0) position of the current block.

[0166] FIG. 8 is a reference diagram for explaining a process of deriving four parameters of an affine transformation utilizing four parameters according to one embodiment of the present disclosure.

[0167] Referring to Fig. 8, the four parameters (a, b, e, f) of the affine transformation utilizing four parameters are (V) of the first control point (CP0) at the upper left (0,0) position of the current block. 0x , V 0y ) of the motion vector V0, the second control point (CP1) at the upper right (W, 0) position of the current block (V 1x , V 1y ) can be derived according to the following mathematical expression 3 using the motion vector V1, the position of the first control point (CP0) (0,0), and the position of the second control point (CP1) (W,0).

[0168]

[0169] When the parameters of the affine transformation utilizing the 4 parameters are determined according to Equation 3, the affine block vector (Vx, Vy) that points to the surrounding reference pixels within the current frame referenced by the current pixel can be determined by substituting the position (x, y) of the current pixel into Equation 3.

[0170] The six parameters (a, b, c, d, e, f) of the affine transform utilizing the six parameters can be derived using the positions of three control points (CPs) of the current block and the CPMVs of the three CPs. When the size of the current block is W x H, the three control points can be the upper left (0, 0) position of the current block, the upper right (W, 0) position of the current block, and the lower left (0, H) position of the current block.

[0171] FIG. 9 is a reference diagram for explaining a process of deriving six parameters of an affine transformation utilizing six parameters according to one embodiment of the present disclosure.

[0172] Referring to Fig. 9, the six parameters (a, b, c, d, e, f) of the affine transformation utilizing six parameters are (V) of the first control point (CP0) at the upper left (0,0) position of the current block. 0x , V 0y ) of the motion vector V0, the second control point (CP1) at the upper right (W, 0) position of the current block (V 1x , V 1y ) of the motion vector V1, the third control point (CP2) at the lower left (0, H) position of the current block (V 2x , V 2y ) can be derived according to the following mathematical expression 4 using the motion vector V2, the position of the first control point (CP0) (0,0), the position of the second control point (CP1) (W,0), and the position of the third control point (CP2) (0,H).

[0173]

[0174] When the parameters of the affine transformation utilizing the 6 parameters are determined according to Equation 4, the affine block vector (Vx, Vy) that points to the reference pixel in the current frame referenced by the current pixel can be determined by substituting the position (x, y) of the current pixel into Equation 4.

[0175] According to an embodiment of the present disclosure, the CPMV of a current block can be determined using a block vector of a neighboring block. The block vector of the neighboring block can be directly set to the CPMV of the current block, or can be used as a prediction value of the CPMV. The neighboring block used to determine the CPMV of the current block is a block-based intra-predicted block that uses a reconstructed reference pixel in the current frame pointed to by the block vector as a prediction value of the current block based on the block vector. For example, block vector information of a neighboring block encoded in an IBC prediction mode or an IntraTMP prediction mode can be used to determine the CPMV of the current block. It is not limited to a block vector obtained according to the IBC or IntraTMP mode, and block vectors generated in various ways can be used as the block vectors of the neighboring blocks.

[0176] In a video encoding process, a video encoder (100) may encode information indicating a block vector of a neighboring block used to determine a CPMV of a current block. If the block vector of a neighboring block is used as a prediction value of a CPMV, the video encoder (100) may encode information about CPMVD (Control Point Motion Vector Differences), which is a difference value between the CPMV of the current block and the prediction value of the CPMV. In the present disclosure, a neighboring block refers to all blocks that are spatially adjacent or not adjacent to a current block, and a block adjacent to a current block may be referred to as an adjacent neighboring block, and a non-adjacent neighboring block may be referred to as a non-adjacent neighboring block.

[0177] In the video decoding process, the video decoder (200) can determine the CPMV of the current block by decoding information indicating the block vector of the surrounding block used to determine the CPMV of the current block from the bitstream, and can decode the affine block vector of the current block using the CP position and the CPMV. If the block vector of the surrounding block is used as a prediction value of the CPMV, the video decoder (200) can obtain information indicating the prediction value of the CPMV and the CPMVD from the bitstream, and decode the CPMV of the current block by adding the CPMV prediction value and the CPMVD. The video decoder (200) obtains the affine block vector of the current block using the CP position information of the current block and the decoded CPMV according to the above-described mathematical expression 3 or 4. The video decoder (200) can obtain the prediction value of the current block by determining a reference pixel within the current frame indicated by the affine block vector.

[0178] An affine block vector can be determined in any data unit. An affine block vector can be determined in one of the following units: the current block, a sub-block divided into the current blocks, a pixel unit, or a block unit of any size or shape. For example, if the current block is 32x32, an affine block vector can be generated in units of sub-blocks of 4x4 blocks. As the position information of the CP, in the case of a 4-parameter affine model, the upper left and upper right positions of the 4x4 block can be used, and in the case of a 6-parameter affine model, the upper left, upper right, and lower left positions can be used. A CPMV derived for the current block can be used as the CPMV, or a CPMV can be derived again in units of sub-blocks.

[0179] As an affine block vector of the current block or sub-block, one affine block vector is derived based on a predetermined reference position, and a prediction value is generated by determining a reference pixel in the current frame using one affine block vector, or an affine block vector is derived for each pixel included in the current block or sub-block, and a prediction value is generated by determining the reference pixel for each pixel. The reference position may be the same as the reference position applied to other block vector-based intra prediction methods, such as IBC or IntraTMP. For example, one affine block vector may be derived according to Equation 3 or Equation 4 based on the upper left position of the current block or sub-block, or an affine block vector may be derived for each pixel by substituting each pixel position in Equation 3 or Equation 4.

[0180] When performing intra prediction by deriving an affine block vector in block units of arbitrary size or shape, the video encoder (100) determines which data unit has which size and shape to use, and can explicitly transmit information about the determined data unit to the video decoder (200). The video encoder (100) can include information about the determined data unit for at least one unit among sequence, GOP, picture, tile, slice, CTU, CU, and TU units in a bitstream and transmit it to the video decoder (200).

[0181] Without signaling explicit information, the video encoder (100) and video decoder (200) may implicitly determine data units based on the RDO (Rate Distortion Optimization) cost in the same manner.

[0182] In the following description, the content of determining the affine block vector of the current block is explained, but the process of determining the affine block vector of the current block can be applied to the process of determining the affine block vector of a sub-block or an arbitrary data unit.

[0183] As described above, the affine block vector according to the present disclosure points to a reference pixel within the same current frame as the current block, and represents the relative position of the pixel within the current block and the reference pixel position. For example, if the affine block vector is (AffineBVx, AffineBVy) and the current pixel position is (x, y), the reference pixel at the position (AffineBVx+x, AffineBVy+y) within the current frame can be used as a predicted value of the current pixel.

[0184] The affine block vector according to the present disclosure can be used to perform affine-based intra prediction not only in slice or picture units where only intra prediction is performed, such as I-slice and I-picture, but also in slice or picture units that support inter prediction, such as P-slice, P-picture, B-slice, and B-picture.

[0185] The reference start position can be set to be the same as the reference start position used in IBC, IntraTMP, etc. For example, when the upper left of the current block is set as the reference start position of (0,0) in the IBC, IntraTMP mode, the upper left position of the current block can be set as the reference start position of (0,0) when applying the affine-based intra prediction mode according to the present disclosure. As another example, when the center of the current block is set as the reference start position of (0,0) in the IBC, IntraTMP mode, the center position of the current block can be set as the reference start position of (0,0) when applying the affine-based intra prediction mode according to the present disclosure. As another example, the reference start position can be set based on a data unit to which the affine-based intra prediction mode according to the present disclosure is applied. For example, when affine-based intra prediction is applied in units of 4x4 sub-blocks, the upper left position of the 4x4 sub-block may be set to the reference start position (0,0), or the center position of the 4x4 sub-block may be set to the reference start position (0,0).

[0186] As described above, according to the embodiment of the present disclosure, the CPMV of a current block can be determined using the block vector of a neighboring block. The block vector of the neighboring block can be directly set to the CPMV of the current block, or can be used as a prediction value of the CPMV. The neighboring block used to determine the CPMV of the current block is a block-based intra-predicted block that uses a reconstructed reference pixel in the current frame pointed to by the block vector as a prediction value of the current block based on the block vector. For example, block vector information of a neighboring block encoded in an IBC prediction mode or an IntraTMP prediction mode can be used to determine the CPMV of the current block.

[0187] Hereinafter, the process of determining the block vector of a surrounding block used to determine the CPMV of the current block will be described with reference to FIGS. 10 to 13.

[0188] FIG. 10 illustrates adjacent neighboring blocks used to determine the CPMV of a current block according to one embodiment of the present disclosure.

[0189] Referring to Fig. 10, when the size of the current block is wxh, in the 4-parameter affine model, a block vector of a neighboring block including one neighboring pixel based on two control points, CP0 at the upper left (0,0) position and CP1 at the upper right (w,0) position of the current block, can be used as the CPMV. In the 6-parameter affine model, a block vector of a neighboring block including one neighboring pixel based on three control points, including CP2 at the lower left (0, h) position in addition to CP0 and CP1, can be used as the CPMV.

[0190] For example, in order to determine the control point motion vector (CPMV0) of CP0 at the upper left (0,0) position, the block vector of one of the neighboring blocks (1031) including the neighboring pixel (1052) at the (-1,0) position, the neighboring block (1010) including the neighboring pixel (1051) at the (0, -1) position, or the neighboring block (1021) including the neighboring pixel (1053) at the (-1, -1) position can be used to determine CPMV0. As described above, the block vector of the neighboring block that performs a block vector-based intra prediction mode, such as IBC or IntraTMP, can be used as the block vector of the neighboring block. In addition, the determined block vector of the neighboring block can be directly used as the CPMV or can be used as a prediction value of the CPMV.

[0191] In order to determine the control point motion vector (CPMV1) of CP1 at the upper right (w, 0) position, a block vector of one of the neighboring blocks (1034) including the neighboring pixel (1056) at the (w+1, 0) position, the neighboring block (1010) including the neighboring pixel (1054) at the (w, -1) position, or the neighboring block (1023) including the neighboring pixel (1055) at the (w+1, -1) position can be used to determine CPMV1.

[0192] In order to determine the control point motion vector (CPMV2) of CP2 at the lower left (0, h) position, a block vector of one of the neighboring blocks (1031) including the neighboring pixel (1057) at the (-1, h) position, the neighboring block (1042) including the neighboring pixel (1059) at the (0, h+1) position, or the neighboring block (1041) including the neighboring pixel (1058) at the (-1, h+1) position can be used to determine CPMV2.

[0193] Meanwhile, when determining the CPMV of the current block using the block vectors of the neighboring blocks, the neighboring blocks adjacent to the current block may not have block vectors. That is, the neighboring blocks adjacent to the current block may not have block vectors determined according to the IBC or IntraTMP mode. In this case, if there is no block vector in the neighboring blocks adjacent to the current block, the block vectors of non-adjacent neighboring blocks that are not adjacent to the current block but have block vectors can be used to determine the CPMV of the current block.

[0194] FIG. 11 is a diagram illustrating a process of searching for a block vector of a non-adjacent surrounding block according to one embodiment of the present disclosure.

[0195] If there is no block vector in a neighboring block adjacent to the current block used to determine the CPMV of the current block, a non-adjacent neighboring block is searched for in a predetermined search direction, either horizontal or vertical, based on the adjacent neighboring blocks to search for a non-adjacent neighboring block having a block vector, and the block vector of the non-adjacent neighboring block searched for first to have a block vector can be used to determine the CPMV of the current block.

[0196] For example, in a 4-parameter affine model, if an adjacent neighboring block used to determine the CPMV0 of CP0 at the upper left position of the current block (1110) does not have a block vector, a non-adjacent neighboring block having a block vector in the horizontal direction from right to left based on the adjacent neighboring block is searched for, and the block vector of the non-adjacent neighboring block determined to have the block vector can be used to determine the CPMV0 of CP0 of the current block. For example, referring to FIG. 11, assuming that it is first determined whether a block vector exists in an adjacent neighboring block (1111) in order to determine the CPMV0 of CP0 at the upper left position of the current block (1110), if the adjacent neighboring block (1111) does not have a block vector, a non-adjacent neighboring block is searched for in the horizontal direction, and the block vector (BV1) of the non-adjacent neighboring block (1120) having the block vector for the first time can be used to determine the CPMV0 of CP0.

[0197] In the 4-parameter affine model, if the adjacent neighboring block used to determine the CPMV1 of CP1 at the upper right position of the current block (1110) does not have a block vector, a non-adjacent neighboring block having a block vector in the vertical direction from the lower to the upper side based on the adjacent neighboring block is searched for, and the block vector of the non-adjacent neighboring block determined to have the block vector can be used to determine the CPMV1 of CP1 of the current block. For example, assuming that it is first determined whether a block vector exists in the adjacent neighboring block (1112) in order to determine the CPMV1 of CP1 at the upper right position of the current block (1110), if the adjacent neighboring block (1112) does not have a block vector, a non-adjacent neighboring block is searched for in the vertical direction, and the block vector (BV2) of the non-adjacent neighboring block (1130) having the block vector for the first time can be used to determine the CPMV1 of CP1.

[0198] In the case of the 6-parameter affine model, in addition to CP0 and CP1 as described above, the CPMV2 of CP2 at the lower left position of the current block (1110) must be determined. If the adjacent neighboring block used to determine the CPMV2 of CP2 at the lower left position of the current block (1110) does not have a block vector, a non-adjacent neighboring block having a block vector in the horizontal direction from left to right based on the adjacent neighboring block is searched, and the block vector of the non-adjacent neighboring block determined to have a block vector can be used to determine the CPMV2 of CP2 of the current block. For example, assuming that in order to determine CPMV2 of CP2 at the lower left position of the current block (1110), it is first determined whether a block vector exists in an adjacent neighboring block (1113), if the adjacent neighboring block (1113) does not have a block vector, a non-adjacent neighboring block is searched in the horizontal direction, and the block vector (BV3) of the non-adjacent neighboring block (1140) that has a block vector for the first time can be used to determine CPMV2 of CP2.

[0199] In the above examples, when the control point position is the upper left or lower left of the current block, such as CP0 or CP2, the search for non-adjacent neighboring blocks having block vectors in the order of proximity to the current block in the horizontal direction from the right to the left based on the adjacent neighboring blocks is exemplified, but the search is not limited thereto and the search direction may be changed. In addition, in the above examples, when the control point position is the upper right of the current block, such as CP1, the search for non-adjacent neighboring blocks having block vectors in the order of proximity to the current block in the vertical direction from the bottom to the top based on the adjacent neighboring blocks is exemplified, but the search is not limited thereto and the search direction of non-adjacent neighboring blocks having block vectors may be changed.

[0200] FIG. 12 is a diagram showing surrounding blocks used to determine the CPMV of a current block in one embodiment of the present disclosure.

[0201] When searching for a neighboring block having a block vector, instead of searching for neighboring blocks sequentially in a predetermined search direction, the positions of the neighboring blocks to be searched can be predetermined by being patterned or tabulated based on their relative positions with respect to the current block. For example, the positions of the neighboring blocks to be searched are predetermined based on the position of the current block, and it is determined whether the neighboring blocks at the predetermined positions have block vectors according to a predetermined order, and the block vector of the first neighboring block determined to have a block vector can be used to determine the CPMV. For example, as illustrated in FIG. 12, the positions and search order of the neighboring blocks used to determine the CPMV of the current block can be predetermined. The positions and search order of the surrounding blocks may be set in advance in the video encoder (100) and the video decoder (200), or the video encoder (100) may set a table with the positions and search order of the surrounding blocks as elements, and may be configured to transmit an index of an element indicating the positions and search order of the applied surrounding blocks to the video decoder (200).

[0202] In one embodiment, a virtual rectangle is determined that surrounds the left non-adjacent neighboring block and the upper non-adjacent neighboring block having the closest block vector to the current block, a CPMV is determined based on the virtual rectangle, and then the CPMV of the current block can be determined using the CPMV determined for the virtual rectangle.

[0203] FIG. 13 is a reference diagram for explaining a process of determining the CPMV of a current block using a virtual square according to one embodiment of the present disclosure.

[0204] Referring to FIG. 13, assuming that the nearest left non-adjacent neighboring block (1310) and the nearest non-adjacent upper neighboring block (1320) are determined as the left and upper neighboring blocks having the closest block vectors to the current block, a virtual rectangle (1330) surrounding the nearest left non-adjacent neighboring block (1310) and the nearest upper non-adjacent neighboring block (1320) is determined, and block vector information (Virtual CPMV0, Virtual CPMV1, Virtual CPMV2) of the control point positions of the virtual rectangle can be used to determine the motion vector of the control point positions of the current block.

[0205] Meanwhile, if there are no adjacent or non-adjacent neighboring blocks with block vectors, a default vector having a predetermined value can be determined as the motion vector of the control point position of the current block. For example, if there are no block vectors in adjacent or non-adjacent neighboring blocks, the zero vector of (0,0) can be used to determine the CPMV.

[0206] As described with reference to FIGS. 10 to 13, when a block vector used to determine the CPMV of a CP of a current block from an adjacent neighboring block or a non-adjacent neighboring block is determined, the CPMV of the current block is determined by directly setting the determined block vector as the CPMV or setting it as a predicted value of the CPMV, and the affine block vector of the current block can be determined by substituting the position information of the CP of the current block and the CPMV information of the CP into Equation 3 or Equation 4.

[0207] Meanwhile, in the video encoding process, the video encoder (100) may determine a CPMV candidate list and include index information indicating a CPMV candidate used to determine the CPMV of the current block in the CPMV candidate list in order to efficiently signal information about the CPMV of the current block in the bitstream. As CPMV candidates of a neighboring block included in the CPMV candidate list, at least one of a CPMV inherited from an neighboring block to which an affine-based intra prediction mode is applied, a CPMV inherited from a non-adjacent neighboring block to which an affine-based intra prediction mode is applied, a CPMV generated using a block vector of an neighboring block, a CPMV generated using a block vector of a non-adjacent neighboring block, a CPMV preset according to a preset rule using a zero vector intra prediction mode value, and a history-based CPMV obtained based on CPMV information of previously processed previous blocks may be included.

[0208] Similar to the neighboring blocks used to determine the CPMV of the current block described above, the positions and search orders of neighboring blocks and non-adjacent neighboring blocks adjacent to the current block can be predetermined. For example, referring again to FIG. 12, when the upper left corner of the current block of size WxH is (0,0), the CPMVs of the neighboring blocks including the neighboring pixels of (-1, H), (W, -1), (W+1, -1), (-1, H+1), and (-1, -1) can be sequentially included in the CPMV candidate list, and then, as illustrated in FIG. 12, the CPMVs of non-adjacent neighboring blocks based on the current block can be sequentially included in the CPMV candidate list. The neighboring blocks illustrated in FIG. 12 are merely examples, and the positions and search orders of the adjacent blocks used as the CPMV candidate list of the current block can be changed.

[0209] Additionally, the surrounding blocks used to determine the CPMV of the current block are blocks having CPMV encoded in the affine-based intra prediction mode according to the present disclosure.

[0210] If a neighboring block does not have a CPMV, the CPMV of the neighboring block can be generated by deriving the CPMV using the block vector of another neighboring block of the neighboring block according to the embodiments of the present disclosure described with reference to FIGS. 8 to 13 described above, and the generated CPMV of the neighboring block can be included in the CPMV candidate list. For example, if the current neighboring block is a first neighboring block, and the first neighboring block does not have a CPMV, the CPMV of the first neighboring block can be determined using the block vector of a second neighboring block, which is a neighboring block of the first neighboring block.

[0211] A CPMV candidate list can be constructed with a set of candidate CPMVs for CPs (hereinafter referred to as 'CPMV candidate set') as one element. For example, assuming that a 4-parameter affine model is used, a CPMV candidate list can be constructed with a set of candidate CPMVs for CP0 and CP1 (CPMV candidate set) as one element, such as {[1st CPMV candidate of CP0, 1st CPMV candidate of CP1], [2nd CPMV candidate of CP0, 2nd CPMV candidate of CP1] 쪋[nth CPMV candidate of CP0, nth CPMV candidate of CP1]}.

[0212] The CPMV candidate list may be constructed with one element for each candidate CPMV for each CP. For example, assuming that a 4-parameter affine model is used, a CPMV candidate list including candidate CPMVs for each of CP0 and CP1 may be constructed, such as {the 1st CPMV candidate of CP0, the 2nd CPMV candidate of CP0, the nth CPMV candidate of CP0} and {the 1st CPMV candidate of CP1, the 2nd CPMV candidate of CP1, the mth CPMV candidate of CP1}}.

[0213] The number of CPMV candidates included in the CPMV candidate list can be preset, and if the number of CPMV candidates inherited from surrounding blocks or generated using block vectors of surrounding blocks is less than the number of CPMV candidates preset, a default vector such as a zero vector can be included in the CPMV candidate list.

[0214] The CPMV of the CPMV candidate group included in the CPMV candidate list can be directly used as the CPMV of the current block or can be used as a predicted value of the CPMV. If the CPMV of the CPMV candidate group included in the CPMV candidate list is directly used as the CPMV of the current block, the video encoder (100) includes index information indicating the CPMV candidate group used to determine the CPMV of the current block in the CPMV candidate list in the bitstream. For example, if the CPMV candidate list includes n CPMV candidates such as {[CPMV0_1, CPMV1_1], [CPMV0_2, CPMV1_2]쪋[CPMV0_n, CPMV1_n]}, and CPMV0_1 and CPMV1_1 included in the first CPMV candidate group [CPMV0_1, CPMV1_1] are each used to determine the CPMV of CP0 and CP1 of the current block, the video encoder (100) can encode index information indicating the first CPMV candidate group. A video decoder (200) obtains information indicating a CPMV candidate group used to determine a CPMV of a current block from a CPMV candidate list from a bitstream, and can determine CPMVs of a current block using CPMVs of a CPMV candidate group determined based on the information indicating the CPMV candidate group.

[0215] If a CPMV candidate is used as a CPMV prediction value of the current block, the video encoder (100) includes information about CPMVDs in the bitstream in addition to index information indicating a CPMV candidate group used as a CPMV prediction value of the current block from the CPMV candidate list. In this case, the video decoder (200) obtains information indicating a CPMV candidate group used as a CPMV prediction value of the current block from the CPMV candidate list from the bitstream and differential motion vector values ​​(CPMVDs), determines the CPMV candidate group determined based on the information indicating the CPMV candidate group as CPMV prediction values ​​of the CPMVs of the current block, and decode the CPMV of the current block based on the CPMV prediction value of each CP and the CPMVD.

[0216] FIG. 14 is a reference diagram for explaining a process of constructing an affine block vector by inheriting the CPMV of a surrounding block according to one embodiment of the present disclosure.

[0217] In one embodiment, the CPMVs of adjacent / non-adjacent peripheral blocks (hereinafter collectively referred to as “peripheral blocks”) may be used (inherited) as CPMV candidates for the current block. That is, the CPMV of the CP of the peripheral block may be used as the CPMV candidate for the corresponding CP of the current block.

[0218] Referring to Fig. 14, assuming a 6-parameter affine model, a CPMV candidate group of [CPMV0', CPMV1', CPMV2'] including CPMV0' of CP0', CPMV1' of CP1', and CPMV2' of CP2' of surrounding blocks can be included as one element in the CPMV candidate list. As described above, a CPMV candidate list can be constructed by searching adjacent / non-adjacent blocks in a predetermined order until the number of CPMV candidates included in the CPMV candidate list reaches a predetermined number.

[0219] The video encoder (100) can configure a CPMV candidate list including CPMVs of adjacent / non-adjacent surrounding blocks of the current block, and transmit index information indicating a CPMV candidate applied to the current block among the CPMV candidates included in the CPMV candidate list. The video decoder (200) can derive and set a CPMV candidate list in the same manner as the video encoder (100), and then determine a CPMV candidate indicated by the index information among the CPMV candidates included in the CPMV candidate list using the index information included in the bitstream. For example, if [CPMV0', CPMV1', CPMV2'] among the candidates included in the CPMV candidate list are determined as CPMV candidates for the current block, the video decoder (200) uses CPMV0' as the CPMV0 of CP0 of the current block directly or as a predicted value, uses CPMV1' as the CPMV1 of CP1 of the current block directly or as a predicted value, and uses CPMV2' as the CPMV2 of CP2 of the current block directly or as a predicted value. In this way, when the CPMV of the current block is determined using the CPMVs inherited from the surrounding blocks, the CPMV determined as described above can be substituted into Equation 3 or Equation 4, and the pixel location (x, y) of the current block can be substituted to determine the affine block vector of the current block. When the CPMV is used as it is as described above, the CPMV value of the surrounding block inherited from the surrounding block can be substituted as the two-dimensional vector values ​​of V0, V1, and V2 corresponding to the CPMV in Equation 3 or Equation 4. If the CPMV of the surrounding block is used as a CPMV prediction value, the CPMV and CPMVD of the surrounding block can be added to restore the CPMV, and then the restored CPMV can be substituted into the CPMV value of Equation 3 or Equation 4. In this case, the (x, y) value, which is the location of the pixel in the current block, can be the location value of the pixel in the current block set with the upper left of the current block as (0, 0).

[0220] In the case of the above-mentioned Fig. 14, the (x, y) value, which is the pixel position within the current block, is the position value of the pixel within the current block set with the upper left of the current block as (0, 0). That is, the value (x, y) of the pixel position substituted into mathematical expression 3 or mathematical expression 4 is a value set based on the upper left of the current block. The CPMV of the surrounding block is also a vector expressed in the form of a two-dimensional coordinate with the upper left of the surrounding block as a reference point, and the two-dimensional coordinate value indicating the CPMV inherited from the surrounding block can be used as the two-dimensional coordinate value of the CPMV of the current block.

[0221] FIG. 15 is a reference diagram for explaining a process of constructing an affine block vector by inheriting the CPMV of a surrounding block according to another embodiment of the present disclosure.

[0222] Vectors like CPMV can be expressed as two-dimensional coordinates (horizontal coordinates, vertical coordinates) of the point the vector points to, starting from the origin, depending on the vector's magnitude and direction relative to the origin. When expressing a vector in two-dimensional coordinate format, the two-dimensional coordinate values ​​representing the vector, even for vectors of the same magnitude and direction, can change depending on the reference point.

[0223] Referring to FIG. 15, after the affine parameters are determined according to the aforementioned mathematical expression 3 or 4 using the CPMV (V0', V1', V2') inherited from the surrounding blocks, the position value (x, y) of the pixel of the current block may be substituted into the mathematical expression 3 or 4 to generate an affine block vector of the current block according to the affine parameter model. At this time, instead of the two-dimensional coordinate value determined with the upper left corner of the current block as the reference position of (0, 0), the relative position value determined with the upper left corner of the previous block as the reference position of (0, 0) may be used as the position value (x, y) of the pixel of the current block. That is, the relative position value determined with the previous surrounding block as the position value (x, y) of the pixel of the current block of the mathematical expression 3 or 4 may be used. In this way, when the relative position value determined based on the previous surrounding block is used as the pixel position value of the current block, the MV expressed as the two-dimensional coordinate of (Vx, Vy) is determined by substituting the relative position value into Equation 3 or Equation 4, and the determined MV is a vector having a two-dimensional coordinate value based on the previous block, so in order to convert the upper left of the current block into a vector based on (0,0), the determined MV must be corrected by considering the relative position difference between the current block and the previous block.

[0224] For example, referring to FIG. 15, if the upper left of the current block is located at (a, b) in a coordinate system that uses the upper left of the previous neighboring block as the reference point of (0, 0), when the upper left of the current block is used as the reference point of (0, 0), the pixel at position (x, y) within the current block corresponds to the pixel at position (x+a, y+b) in the coordinate system that uses the upper left (0, 0) of the previous neighboring block as the reference point. In this case, instead of (x, y) as the position value of the pixel of the current block in Equation 3 or Equation 4, (x+a, y+b), which is a position value based on the upper left (0, 0) of the previous neighboring block, is substituted, and MV having (Vx, Vy) is determined according to Equation 3 or Equation 4. And, the determined (Vx, Vy) is corrected to a two-dimensional vector value with the upper left corner of the current block as the reference point (0,0) considering the relative positions of the current block and the surrounding blocks. In the case of the above example, MV is corrected to (Vx-a, Vy-b), and (Vx-a, Vy-b) can be determined as an affine block vector that points to the surrounding pixels referenced by the pixel of the current block at the position (x,y) with the upper left corner of the current block as the reference point (0,0).

[0225] In addition, as described above, the CPMV candidate list may include, in addition to CPMV candidates inherited from neighboring blocks, CPMVs generated using block vectors (BVs) of neighboring blocks, such as CPMVs generated using block vectors (BVs) of neighboring blocks, CPMVs generated using block vectors (BVs) of non-adjacent neighboring blocks, as candidates.

[0226] FIG. 16a and FIG. 16b are reference diagrams for explaining a process of constructing a CPMV candidate list using block vectors of surrounding blocks according to one embodiment of the present disclosure.

[0227] Referring to FIG. 16a, the block vector (BV') of a neighboring block can be used as a CPMV candidate for at least one CP among the CPs of the current block. That is, the block vector (BV') of a neighboring block can be used as a CPMV candidate for at least one CPMV among CPMV0, CPMV1, and CPMV2 of the current block.

[0228] In addition, a set of block vectors that combine block vectors of neighboring blocks can be configured as one CPMV candidate group of the current block. For example, referring to FIG. 16B, a block vector (BV0') of neighboring block 0 can be used as a CPMV candidate of CP0 of the current block, a block vector (BV1') of neighboring block 1 can be used as a CPMV candidate of CP1, and a block vector (BV2') of neighboring block 2 can be used as a CPMV candidate of CP2. [BV0', BV1', BV2'] can be included in the CPMV candidate list as one CPMV candidate group of the current block. The search process of neighboring blocks used to determine the CPMV candidate of the current block can be performed in the same manner as the process of searching block vectors of neighboring blocks according to the embodiments of the present disclosure described above.

[0229] Meanwhile, when using the block vector (BV') of a surrounding block as a CPMV candidate for the current block, the block vector (BV') of the surrounding block can be used as a CPMV candidate for the current block as is depending on the reference points of the surrounding block and the current block, or the block vector (BV') of the surrounding block can be corrected depending on the reference point, and the corrected block vector (BV') of the surrounding block can be used as a CPMV candidate for the current block. For example, if the current block and the surrounding blocks are each generated as vectors of pixel positions and two-dimensional coordinates using the upper left corner of the corresponding blocks as a reference point, that is, if the (x, y) value, which is the pixel position within the current block, is set using the upper left corner of the current block as a reference point (0,0), and the block vector (BV') of the surrounding block is set using the upper left corner of the surrounding block as a reference point (0,0), then the block vector (BV') of the surrounding block can be used as a CPMV candidate for the current block as is without any separate correction.

[0230] If it is assumed that the upper left of the current block is located at the position (a,b) in a coordinate system with the upper left of the previous surrounding block as the reference point (0,0) and the block vector (BV) of the surrounding block is expressed as the two-dimensional coordinate (Vx, Vy) based on the surrounding block, then when the upper left of the current block is set as the reference point (0,0), the block vector (BV) of the previous surrounding block is corrected to (Vx-a, Vy-b) by correcting the relative position difference (a, b) between the surrounding block and the current block, and the block vector of the corrected surrounding block can be used as a CPMV candidate of the current block.

[0231] In one embodiment of the present disclosure, in order to efficiently encode a CPMV candidate used as the CPMV of the current block among the CPMV candidate list, a template cost may be obtained using template matching, and CPMV candidates included in the CPMV candidate list may be rearranged based on the template cost. For example, if the determined CPMV candidate list includes four CPMV candidates, such as {[CPMV0_0, CPMV1_0], [CPMV0_1, CPMV1_1], [CPMV0_2, CPMV1_2], [CPMV0_3, CPMV1_3]}, and the template costs increase in ascending order in the order of [CPMV0_2, CPMV1_2], [CPMV0_1, CPMV1_1], [CPMV0_3, CPMV1_3], [CPMV0_0, CPMV1_0], the video encoder (100) can rearrange the CPMV candidates included in the CPMV candidate list from the order of the smallest template cost to the largest. For the example described above, the CPMV candidate list is rearranged as {[CPMV0_2, CPMV1_2], [CPMV0_1, CPMV1_1], [CPMV0_3, CPMV1_3], [CPMV0_0, CPMV1_0]}. In addition, various cost calculation methods such as SAD (Sum of Absolute Differences), SATD (Sum of Absolute Transformed Differences), MRSAD (Motion-Refined SAD), and MRSATD (Motion-Refined SATD) can be used when calculating the cost.

[0232] The video encoder (100) determines an optimal CPMV candidate based on RDO among CPMV candidates included in the rearranged CPMV candidate list, and encodes information indicating the determined CPMV candidate.

[0233] FIG. 17 is a reference diagram for explaining a process of calculating a template cost using a CPMV candidate according to one embodiment of the present disclosure.

[0234] The video encoder (100) sets the CPMV of the determined CPMV candidate to the CPMV of the current block to determine affine parameters, and obtains an affine block vector that points to a reference pixel referenced by pixels of the surrounding template using the determined affine parameters and the pixel positions of the surrounding templates of the current block. Then, the video encoder (100) obtains a prediction value of the surrounding template using the affine block vectors of the pixels of the surrounding templates, and rearranges the CPMV candidates included in the CPMV candidate list in the order of decreasing cost based on the cost, which is the difference between the prediction value of the surrounding template and the surrounding template. Then, the video encoder (100) encodes information indicating a CPMV candidate applied to the current block among the CPMV candidates included in the CPMV candidate list.

[0235] Specifically, the video encoder (100) obtains an affine block vector (Vx, Vy) by substituting the CPMV of the CPMV candidate into Equation 3 or Equation 4 according to the applied affine model. For example, if a 6-parameter affine model is applied and the CPMV candidate list includes four CPMV candidates such as {[CPMV0_0, CPMV1_0], [CPMV0_1, CPMV1_1], [CPMV0_2, CPMV1_2], [CPMV0_3, CPMV1_3]}, the video encoder (100) sets each CPMV candidate to the CPMV of the CP of the current block, and determines affine parameters for obtaining an affine block vector (Vx, Vy) by applying Equation 4. When a 3-parameter affine model is used, affine parameters for obtaining an affine block vector can be obtained according to mathematical expression 3. Then, the video encoder (100) can substitute the positions of pixels included in the template (1710) into the values ​​of (x, y) in mathematical expression 4 to obtain an affine block vector (Vx, Vy) indicating a reference pixel of each pixel included in the template, and can obtain a reference pixel in the current frame indicated by the affine block vector (Vx, Vy) as a predicted value of the template (1710).

[0236] In addition, the video encoder (100) can calculate a cost, which is a difference value between the template (1710) and the predicted value of the template (1710) obtained using the affine block vector, and rearrange the CPMV candidates included in the CPMV candidate list in order of decreasing cost based on the template cost.

[0237] Meanwhile, when determining an affine block vector pointing to a reference pixel referenced by a pixel of the template (1710) by substituting the pixel position (x,y) of the template (1710) into mathematical expression 3 or mathematical expression 4, the pixel position (x,y) of the template (1710) may be a value set based on the upper left (0,0) of the current block, or a value set based on the upper left (0,0) of the template (1710).

[0238] The video decoder (200) rearranges CPMV candidates included in the CPMV candidate list from the smallest template cost to the largest template cost in the same manner as the video encoder (100), decodes information indicating a CPMV candidate used to determine the CPMV of the current block among the CPMV candidates included in the rearranged CPMV candidate list, and can derive the CPMV of the current block using the CPMV of the determined CPMV candidate. That is, the video decoder (200) sets the CPMV of the CPMV candidate included in the CPMV candidate list to the CPMV of the current block to determine affine parameters, obtains an affine block vector indicating a reference pixel referenced by pixels of the surrounding templates using the determined affine parameters and pixel positions of the surrounding templates of the current block, and obtains a prediction value of the surrounding templates using the affine block vectors of the pixels of the surrounding templates. Then, the video decoder (200) can rearrange CPMV candidates included in the CPMV candidate list in order of decreasing cost based on the predicted value of the surrounding template and the cost, which is the difference value between the surrounding templates, obtain index information indicating one of the rearranged CPMV candidates from the bitstream, and determine a CPMV candidate applied to the current block among the CPMV candidate list based on the index information. Then, the video decoder (200) can determine a CPMV included in the determined CPMV candidate as the CPMV of the current block.

[0239] FIG. 18 is a flowchart of a video encoding method according to one embodiment of the present disclosure.

[0240] Referring to FIG. 18, the video encoder (100) determines motion vectors of control point positions of the current block (S1810). As described above, the CPMV of the current block may be determined using a block vector of an adjacent neighboring block including pixels surrounding the CP, or, if the adjacent neighboring block does not have a block vector, may be determined using a block vector of a non-adjacent neighboring block. As the block vector of the neighboring block, a block vector of a neighboring block on which block-based intra prediction, such as IBC or IntraTMP, is performed may be used. The block vector of the neighboring block may be directly used as the CPMV or may be used as a predicted value of the CPMV.

[0241] The video encoder (100) determines an affine block vector, which is a motion vector of a predetermined data unit within the current block, using motion vectors at control point locations (S1820). The predetermined data unit may be one of a sub-block unit of a predetermined size within the current block, a pixel unit, or a block unit of an arbitrary size or shape. The video encoder (100) may signal information about the determined data unit.

[0242] The video encoder (100) performs intra prediction on a given data unit using an affine block vector of the data unit (S1830). As described above, the affine block vector is a motion vector that points to a reconstructed surrounding pixel within the same frame as the current block or the current block referenced by the given data unit within the current block. In the present disclosure, a prediction value of the current block can be generated by referring to a reference pixel value within the current frame pointed to by the derived affine block vector.

[0243] The video encoder (100) can encode information indicating a block vector of a neighboring block used to determine the CPMV of the current block. If the block vector of the neighboring block is used as a prediction value of the CPMV, the video encoder (100) can encode information on a CPMVD, which is a difference value between the CPMV of the current block and the prediction value of the CPMV. In addition, the video encoder (100) can encode information indicating an affine-based intra prediction mode as prediction mode information of the current block.

[0244] FIG. 19 is a flow chart of a video decoding method according to one embodiment of the present disclosure.

[0245] Referring to FIG. 19, the video decoder (200) determines motion vectors of control point positions of the current block (S1910). As described above, the CPMV of the current block may be determined using a block vector of an adjacent neighboring block including pixels surrounding the CP, or, if the adjacent neighboring block does not have a block vector, may be determined using a block vector of a non-adjacent neighboring block. As the block vector of the neighboring block, a block vector of a neighboring block on which block-based intra prediction, such as IBC or IntraTMP, is performed may be used. The block vector of the neighboring block may be directly used as the CPMV or may be used as a predicted value of the CPMV.

[0246] The video decoder (200) uses the motion vectors of the control point positions to determine an affine block vector, which is a motion vector of a predetermined data unit within the current block (S1920). As described above, the affine block vector, which is a motion vector of a predetermined data unit, can be determined by substituting the CPMV determined from the control point positions of the predetermined data unit within the current block and the block vector obtained from the surrounding blocks into the above-described mathematical expression 3 or mathematical expression 4.

[0247] Then, the video decoder (100) performs intra prediction for a given data unit using the affine block vector of the data unit (S1930). As described above, the prediction value of the current block can be generated by referring to the reference pixel value within the current frame pointed to by the derived affine block vector.

[0248] The video decoder (200) can decode information indicating a block vector of a neighboring block used to determine the CPMV of the current block from the bitstream. If the block vector of the neighboring block is used as a prediction value of the CPMV, the video decoder (200) can decode information about a CPMVD, which is a difference value between the information indicating the CPMV used to determine the CPMV of the current block and the prediction value of the CPMV. In addition, the video decoder (200) can obtain prediction mode information of the current block from the bitstream, and if the prediction mode information of the current block indicates an affine-based intra prediction mode, perform the above-described affine-based intra prediction process (S1910 to S1930).

[0249] The methods described herein may be performed by a processor of a video encoder or video decoder. Furthermore, the encoder may generate a bitstream that is decoded by a video signal processing method, and the bitstream generated by the encoder may be stored in a computer-readable, non-transitory storage medium (recording medium).

[0250] The embodiments of the present invention described above may be implemented through various means. For example, the embodiments of the present invention may be implemented using hardware, firmware, software, or a combination thereof.

[0251] Some embodiments may also be implemented in the form of a computer-executable storage medium containing computer-executable instructions, such as program modules executed by a computer. Computer-readable media may be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media.

[0252] Additionally, computer-readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Communication media typically includes other data, such as computer-readable instructions, data structures, or program modules, in a modulated data signal, or other transport mechanism, and includes any information delivery media.

Claims

1. A step of determining the motion vectors of the control point positions of the current block; A step of determining a motion vector of a predetermined data unit within the current block using the motion vectors of the control point positions; A step of performing intra prediction for the predetermined data unit using the motion vector of the data unit, The motion vector of the above control point position is A block vector of a surrounding block containing surrounding pixels of the above control point location, How to decrypt video.

2. In paragraph 1, The block vector of the block containing the surrounding pixels of the above control point location is When the above-mentioned surrounding block is in IBC (Intra Block Copy) mode or IntraTMP (Intra Template Matching Prediction) mode, the block vector derived for the surrounding block according to the IBC mode or IntraTMP mode, How to decrypt video.

3. In paragraph 1, The above data unit is one of a sub-block unit of a predetermined size within the current block, a pixel unit, or a block unit of an arbitrary size or shape. How to decrypt video.

4. In paragraph 1, The surrounding pixels used to determine the motion vector of the above control point position are one of the surrounding pixels previously processed based on the pixel of the above control point position. How to decrypt video.

5. In paragraph 1, For the 4-parameter affine model, the control point locations are the upper left and upper right locations of the current block, For the 6-parameter affine model, the control point locations are the upper left, upper right, and lower left locations of the current block. When the control point position at the upper left position is (0, 0), the surrounding pixels used to determine the motion vector of the control point position at the upper left position are the surrounding pixels located at one of the positions (-1, 0), (0, -1), and (-1, -1). When the control point position at the upper right position is (w, 0), the surrounding pixels used to determine the motion vector of the control point position at the upper right position are the surrounding pixels located at one of the positions (w+1, 0), (w, -1), and (w+1, -1). When the control point position at the lower left position is (0, h), the surrounding pixels used to determine the motion vector of the control point position at the lower left position are the surrounding pixels located at one of the positions (-1, h), (0, h+1), and (-1, h+1). How to decrypt video.

6. In paragraph 1, The step of determining the motion vectors of the control point positions of the current block above is A step of determining whether an adjacent neighboring block including neighboring pixels used to determine a motion vector of the above control point position has a block vector; When the adjacent surrounding block has a block vector, a step of determining a motion vector of the control point position using the block vector of the adjacent surrounding block; and A step of searching for a non-adjacent neighboring block having a block vector in one of a horizontal direction and a vertical direction based on the adjacent neighboring block, when the adjacent neighboring block does not have a block vector; Including a step of determining a motion vector of a control point position of the current block using a block vector of a first non-adjacent surrounding block having a block vector according to the search direction. How to decrypt video.

7. In paragraph 6, The step of searching for non-adjacent surrounding blocks having the above block vector is If the control point position is the upper left or lower left position of the current block, non-adjacent neighboring blocks having block vectors in the order of proximity to the current block in the horizontal direction from the right to the left based on the adjacent neighboring blocks are searched, If the control point position is the upper right position of the current block, non-adjacent neighboring blocks having block vectors in order of proximity to the current block in the vertical direction from the bottom to the top based on the adjacent neighboring blocks are searched. How to decrypt video.

8. In paragraph 7, The step of determining the motion vector of the above control point position is Determine the nearest left non-adjacent neighboring block having a block vector in the left direction of the current block and the nearest upper non-adjacent neighboring block having a block vector in the upper direction of the current block and the nearest to the current block, Determine a virtual rectangle surrounding the nearest left non-adjacent surrounding block and the nearest upper non-adjacent surrounding block, Using the block vector information of the control point position of the virtual rectangle corresponding to the control point position of the current block, a motion vector of the control point position of the current block is determined. How to decrypt video.

9. In paragraph 6, If there are no adjacent neighboring blocks or non-adjacent neighboring blocks having the above block vector, a default vector having a predetermined value is determined as the motion vector of the control point position of the current block. How to decrypt video.

10. In paragraph 1, The step of determining the motion vectors of the control point positions of the current block is A control point motion vector candidate is determined from a list of control point motion vector candidates including at least one of a control point motion vector candidate inherited from an adjacent neighboring block to which an affine-based intra prediction mode is applied, a control point motion vector candidate inherited from a non-adjacent neighboring block to which an affine-based intra prediction mode is applied, a control point motion vector candidate generated using a block vector of an adjacent neighboring block, a control point motion vector candidate generated using a block vector of a non-adjacent neighboring block, a zero-vector control point motion vector candidate, a control point motion vector candidate preset according to a preset rule using an intra prediction mode value, and a history-based control point motion vector candidate obtained based on CPMV information of previously processed previous blocks, and a control point motion vector of the current block is determined using a control point motion vector included in the determined control point motion vector candidate. How to decrypt video.

11. In paragraph 10, The step of determining the motion vector of the control point position of the current block above is A step of obtaining information indicating a control point motion vector candidate used as a motion vector of the control point position of the current block from among the control point motion vector candidate list from the bitstream; A step of determining a control point motion vector included in a control point motion vector candidate determined based on information indicating the control point motion vector candidate as a motion vector of a control point position of the current block, How to decrypt video.

12. In paragraph 11, Further comprising a step of correcting the two-dimensional coordinate values ​​of the control point motion vector inherited from the surrounding block by considering the reference point of the current block and the reference points of the surrounding block. How to decrypt video.

13. In paragraph 10, The step of determining the motion vector of the control point position of the current block above is A step of obtaining information and a differential motion vector value indicating a control point motion vector candidate used as a motion vector of the control point position of the current block from the control point motion vector candidate list from the bitstream; A step of determining a control point motion vector included in a control point motion vector candidate determined based on information indicating the control point motion vector candidate as a predicted motion vector, and determining a motion vector of a control point position of the current block based on the predicted motion vector and a differential motion vector value, How to decrypt video.

14. In paragraph 10, The step of determining the motion vector of the control point position of the current block above is A step of determining affine parameters by setting the control point motion vector of the above-determined control point motion vector candidate as the control point motion vector of the current block, and obtaining an affine block vector that points to a reference pixel referenced by pixels of the surrounding template by using the determined affine parameters and the pixel positions of the surrounding template of the current block; A step of obtaining a prediction value of the surrounding template using affine block vectors of pixels of the surrounding template; A step of rearranging the control point motion vector candidates included in the control point motion vector candidate list in order of decreasing cost based on the cost, which is the difference value between the predicted value of the surrounding template and the surrounding template; A step of obtaining index information pointing to one of the rearranged control point motion vector candidates from the bitstream; and A step of determining a control point motion vector included in a control point motion vector candidate determined based on the above index information as a motion vector of a control point position of the current block, How to decrypt video.

15. A video decoding device performing the method of any one of claims 1 to 14.

16. A computer-readable recording medium recording a program for executing the method of any one of claims 1 to 14 on a computer.

17. A step of determining the motion vectors of the control point positions of the current block; A step of determining a motion vector of a predetermined data unit within the current block using the motion vectors of the control point positions; A step of performing intra prediction for the predetermined data unit using the motion vector of the data unit, The motion vector of the above control point position is A block vector of a surrounding block containing surrounding pixels of the above control point location, Video encoding method.

18. A video encoding device performing the method of Article 17.

19. A computer-readable recording medium recording a program for executing the method of Article 17 on a computer.

Images