Inter-frame prediction method and device based on merging mode

Abstract

The present invention relates to a video encoding / decoding method and apparatus, characterized by comprising: a step of constituting a merge candidate list of a current block, deriving motion information of the current block from the merge candidate list, and performing inter prediction of the current block using the motion information; the merge candidate list includes at least one of a spatial merge candidate, a temporal merge candidate, or a combined merge candidate, and the combined merge candidate is derived by combining n merge candidates belonging to the merge candidate list, since not only the spatial / temporal merge candidate is utilized, but also the combined merge candidate is utilized, the combined merge candidate is derived by combining n merge candidates belonging to the merge candidate list. Therefore, the accuracy of the motion information can be improved.

Description

[0001] This application is a divisional application of the PCT international patent application PCT / KR2019 / 007981 with an application date of July 1, 2019, which entered the Chinese national phase with Chinese patent application number 201980041821.0 and invention name “Inter-frame prediction method and device based on merge mode”. Technical Field

[0002] The invention relates to an inter-frame prediction method and device. Background Art

[0003] Recently, in various application fields, the demand for high-resolution and high-quality videos such as HD (High Definition) videos and UHD (Ultra High Definition) videos has been increasing, and therefore, research on efficient video compression technology is being conducted.

[0004] As video compression technologies, there are various technologies such as inter-frame prediction technology that predicts the pixel values ​​contained in the current image based on the images before or after the current image, intra-frame prediction technology that uses pixel information in the current image to predict the pixel values ​​contained in the current image, and entropy coding technology that assigns short codes to values ​​with higher occurrence frequencies and long codes to values ​​with lower occurrence frequencies. These video compression technologies can be used to effectively compress video data for transmission or storage. Summary of the invention

[0005] [Problems to be solved by the invention]

[0006] The object of the present invention is to provide an inter-frame prediction method and device.

[0007] The object of the present invention is to provide a method and device for forming a merge candidate list.

[0008] The object of the present invention is to provide a sub-block unit motion compensation method and device.

[0009] The object of the present invention is to provide a method and device for determining affine candidates.

[0010] The object of the present invention is to provide an inter-frame prediction method and device corresponding to the projection format of a 360 video.

[0011] [Methods of solving the problem]

[0012] The video encoding / decoding method and apparatus of the present invention can construct a merge candidate list of a current block, derive motion information of the current block according to the merge candidate list, and use the motion information to perform inter-frame prediction of the current block.

[0013] In the video encoding / decoding method and apparatus of the present invention, the merge candidate list may include at least one of a spatial merge candidate, a temporal merge candidate, or a combined merge candidate.

[0014] In the video encoding / decoding method and apparatus of the present invention, the combined merge candidate may be derived by combining n merge candidates belonging to the merge candidate list.

[0015] In the video encoding / decoding method and apparatus of the present invention, the n merge candidates may be merge candidates corresponding to indexes from 0 to (n-1) in the merge candidate list.

[0016] In the video encoding / decoding method and apparatus of the present invention, the index of the combined merging candidate may be greater than the index of the temporal merging candidate.

[0017] In the video encoding / decoding method and apparatus of the present invention, the n merge candidates include a first merge candidate and a second merge candidate, and the motion information of the combined merge candidate can be derived by considering the prediction direction of the first merge candidate and the prediction direction of the second merge candidate.

[0018] In the video encoding / decoding method and apparatus of the present invention, the motion information of the combined merge candidate may be derived by taking a weighted average of the motion information of the first merge candidate and the motion information of the second merge candidate.

[0019] In the video encoding / decoding method and apparatus of the present invention, the weighted value of the weighted average may be any one of [1:1], [1:2], [1:3] or [2:3].

[0020] The video encoding / decoding method and device of the present invention can generate a candidate list for predicting motion information of a current block, derive a control point vector of the current block based on the candidate list and a candidate index, derive a motion vector of the current block based on the control point vector of the current block, and use the motion vector to perform inter-frame prediction on the current block.

[0021] In the video encoding / decoding apparatus of the present invention, the candidate list may include a plurality of affine candidates.

[0022] In the video encoding / decoding apparatus of the present invention, the affine candidate may include at least one of a spatial candidate, a temporal candidate, or a constructed candidate.

[0023] In the video encoding / decoding apparatus of the present invention, the motion vector of the current block can be derived in units of sub-blocks of the current block.

[0024] In the video encoding / decoding apparatus of the present invention, the spatial candidate may be determined by considering whether the boundary of the current block is adjacent to the boundary of a coding tree block (CTU boundary).

[0025] In the video encoding / decoding apparatus of the present invention, the constructed candidate may be determined based on a combination of at least two control point vectors among the control point vectors corresponding to the corners of the current block.

[0026] The video encoding / decoding method and apparatus of the present invention can obtain all or part of the pixels of the reference area by using data of the associated area when the reference area used for inter-frame prediction includes the boundary of the reference image or the boundary between discontinuous surfaces.

[0027] [Effects of the Invention]

[0028] According to the present invention, not only spatial / temporal merging candidates but also combined merging candidates are used, so the accuracy of motion information can be improved.

[0029] According to the present invention, the encoding / decoding performance of a video can be improved by using inter-frame prediction based on an affine model.

[0030] According to the present invention, the prediction accuracy can be improved through inter-frame prediction in sub-block units.

[0031] According to the present invention, affine candidates can be determined efficiently, thereby improving encoding / decoding efficiency of inter-frame prediction.

[0032] According to the present invention, the reference area can be set in consideration of the correlation, so the encoding efficiency of the inter-frame prediction can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a block diagram of a video encoding apparatus according to an embodiment of the present invention.

[0034] Figure 2 is a block diagram of a video decoding apparatus according to an embodiment of the present invention.

[0035] Figure 3 This is an example diagram showing a tree-based block form.

[0036] Figure 4 This is an illustrative diagram showing various block forms that can be obtained from the block dividing unit of the present invention.

[0037] Figure 5 FIG. 2 shows a block segmentation process according to an embodiment of the present invention.

[0038] Figure 6 As one embodiment to which the present invention is applied, this diagram shows a method of performing inter-frame prediction based on a merge mode of block units.

[0039] Figure 7 As an embodiment of the present invention, this diagram shows an inter-frame prediction method based on an affine model.

[0040] Figure 8 As an embodiment to which the present invention is applied, a diagram is shown showing a method of deriving affine candidates from control point vectors of spatial / temporal neighboring blocks.

[0041] Fig. 9 As one embodiment to which the present invention is applied, this is a diagram showing a method of deriving a constructed candidate based on a combination of motion vectors of spatial / temporal neighboring blocks.

[0042] Fig.10 As one embodiment to which the present invention is applied, this is a diagram showing a method of deriving motion information based on a temporal candidate of a sub-block.

[0043] Fig.11 As an embodiment of the present invention, it is a diagram showing an inter-frame prediction method in an ERP (Equi-Rectangular Projection) projection format.

[0044] Fig.12 and Fig.15 As an embodiment of the present invention, it is a diagram showing an inter-frame prediction method in a CMP (CubeMap Projection) projection format. DETAILED DESCRIPTION

[0045] The video encoding / decoding method and apparatus of the present invention can construct a merge candidate list of a current block, derive motion information of the current block according to the merge candidate list, and use the motion information to perform inter-frame prediction of the current block.

[0046] In the video encoding / decoding method and apparatus of the present invention, the merge candidate list may include at least one of a spatial merge candidate, a temporal merge candidate, or a combined merge candidate.

[0047] In the video encoding / decoding method and apparatus of the present invention, the combined merge candidate may be derived by combining n merge candidates belonging to the merge candidate list.

[0048] In the video encoding / decoding method and apparatus of the present invention, the n merge candidates may be merge candidates corresponding to indexes from 0 to (n-1) in the merge candidate list.

[0049] In the video encoding / decoding method and apparatus of the present invention, the index of the combined merging candidate may be greater than the index of the temporal merging candidate.

[0050] In the video encoding / decoding method and apparatus of the present invention, the n merge candidates include a first merge candidate and a second merge candidate, and the motion information of the combined merge candidate can be derived by considering the prediction direction of the first merge candidate and the prediction direction of the second merge candidate.

[0051] In the video encoding / decoding method and apparatus of the present invention, the motion information of the combined merge candidate may be derived by taking a weighted average of the motion information of the first merge candidate and the motion information of the second merge candidate.

[0052] In the video encoding / decoding method and apparatus of the present invention, the weighted value of the weighted average may be any one of [1:1], [1:2], [1:3] or [2:3].

[0053] The video encoding / decoding method and device of the present invention can generate a candidate list for predicting motion information of a current block, derive a control point vector of the current block based on the candidate list and a candidate index, derive a motion vector of the current block based on the control point vector of the current block, and use the motion vector to perform inter-frame prediction on the current block.

[0054] In the video encoding / decoding apparatus of the present invention, the candidate list may include a plurality of affine candidates.

[0055] In the video encoding / decoding apparatus of the present invention, the affine candidate may include at least one of a spatial candidate, a temporal candidate, or a constructed candidate.

[0056] In the video encoding / decoding apparatus of the present invention, the motion vector of the current block can be derived in units of sub-blocks of the current block.

[0057] In the video encoding / decoding apparatus of the present invention, the spatial candidate may be determined by considering whether the boundary of the current block is adjacent to the boundary of a coding tree block (CTU boundary).

[0058] In the video encoding / decoding apparatus of the present invention, the constructed candidate may be determined based on a combination of at least two control point vectors among the control point vectors corresponding to the corners of the current block.

[0059] The video encoding / decoding method and apparatus of the present invention can obtain all or part of the pixels of the reference area by using data of the associated area when the reference area used for inter-frame prediction includes the boundary of the reference image or the boundary between discontinuous surfaces.

[0060] The present invention can be implemented in various changes and can have multiple embodiments, and specific embodiments are shown in the drawings and described in detail. However, it should be understood that these specific embodiments are not intended to limit the present invention to specific implementation methods, and all changes, equivalents or substitutes within the scope of the present invention's ideas and techniques are included.

[0061] The terms first, second, A, B, etc. may be used to describe constituent elements, but the constituent elements shall not be limited by the terms. The terms are only used to distinguish one constituent element from other constituent elements. For example, the first constituent element may be named as the second constituent element without departing from the scope of the present invention, and similarly, the second constituent element may be named as the first constituent element. The so-called and / or terms refer to a combination of multiple related items recorded or any item among multiple related items recorded.

[0062] When a certain component is described as being “connected” or “linked” to another component, it should be understood that it may be directly connected or linked to the other component, or other components may exist between the component and the other component. Conversely, when a certain component is described as being “directly connected” or “directly linked” to another component, it should be understood that no other components exist between the component and the other component.

[0063] The terms used in this application are only used to illustrate specific embodiments and are not intended to limit the present invention. If no other meaning is clearly indicated in the text, the expression in the singular includes the expression in the plural. In this application, the terms such as "including" or "having" should be understood to be used to indicate the presence of the features, numbers, steps, actions, constituent elements, parts or combinations thereof recorded in the specification, and do not preclude the possibility of the existence or additional possibility of one or more other features, numbers, steps, actions, constituent elements, parts or combinations thereof.

[0064] If not otherwise defined, all terms including technical or scientific terms have the same meaning as those commonly understood by ordinary technicians in the technical field to which the present invention belongs. Terms defined in commonly used dictionaries should be interpreted as consistent with the meanings in relevant technical documents, and if not explicitly defined in this application, they should not be interpreted as ideal meanings or overly formal meanings.

[0065] The video encoding device and decoding device can be a user terminal such as a personal computer (PC), a laptop computer, a personal digital assistant (PDA), a portable multimedia player (PMP), a portable game console (PSP), a wireless communication terminal (Wireless Communication Terminal), a smart phone (Smart Phone), a TV (Television), a virtual reality device (Virtual Reality, VR), an augmented reality device (Augmented Reality, AR), a mixed reality device (Mixed Reality, MR), a head mounted display (Head Mounted Display, HMD), smart glasses, etc., or a server terminal such as an application server and a service server, and can include the following and other devices: a communication device such as a communication modem for communicating with various devices or wired and wireless communication networks; a memory (Memory) for storing various programs and data for intra-frame (Intra) or inter-frame (Inter) prediction for video encoding or decoding; a processor (Processor) for running programs for calculation and control. In addition, the video encoded into a bit stream by the video encoding device can be transmitted to the video decoding device in real time or non-real time through the Internet, short-range wireless communication network, wireless area network, wireless broadband network, mobile communication network and other wired wireless communication networks (Network), or through various communication interfaces such as cables, Universal Serial Bus (USB), etc., and decoded by the video decoding device to reconstruct the video for playback.

[0066] In addition, the video encoded into the bit stream by the video encoding device can also be transmitted from the encoding device to the decoding device via a recording medium that can be read by a computer.

[0067] The video encoding device and the video decoding device may be independent devices, but may be configured as one video encoding / decoding device according to specific implementation. In this case, a part of the video encoding device is configured as a substantially same technical element as a part of the video decoding device, and may be implemented in a manner of at least including the same structure or at least performing the same function.

[0068] Therefore, when the technical elements and their operating principles are described in detail below, repeated descriptions of corresponding technical elements will be omitted.

[0069] Furthermore, the video decoding device is equivalent to a computing device that applies the video encoding method performed by the video encoding device to decoding, so in the following description, the video encoding device is mainly used for explanation.

[0070] The computing device may include: a memory storing a program or software module implementing the video encoding method and / or the video decoding method; and a processor connected to the memory to execute the program. In addition, the video encoding device may be referred to as an encoder, and the video decoding device may be referred to as a decoder.

[0071] Generally, a video may include a series of still images (Still Image), which can be divided into GOP (Group of Pictures) units, and each still image can be called a picture (Picture). At this time, the image can represent one of the frames (Frame) and fields (Field) in a progressive signal or an interlace signal. When encoding / decoding is performed in units of frames, the video can be represented by "frames", and when encoding / decoding is performed in units of fields, the video can be represented by "fields". In the present invention, a progressive signal is used as an example for explanation, but it can also be applied to an interlace signal. As a higher-level concept, there can be units such as GOP and sequence. In addition, each image can be divided into specific areas such as segments, blocks, and blocks. In addition, a GOP can include units such as I images, P images, and B images. An I picture may refer to an image that is encoded / decoded by itself without using a reference picture, and a P picture and a B picture may refer to an image that is encoded / decoded by performing processes such as motion estimation and motion compensation using a reference picture. Generally, a P picture may use an I picture and a P picture as a reference picture, and a B picture may use an I picture and a P picture as a reference picture. Here, the above definitions may also be changed according to encoding / decoding settings.

[0072] Here, the image referenced during encoding / decoding is called a reference image (Reference Picture), and the referenced block or pixel is called a reference block (Reference Block) or a reference pixel (Reference Pixel). In addition, in addition to the pixel values ​​in the spatial domain (Spatial Domain), the reference data (Reference Data) can also be coefficient values ​​in the frequency domain (Frequency Domain), and various encoding / decoding information generated and determined during the encoding / decoding process. For example, in the prediction unit, it can be intra-frame prediction related information or motion related information, in the transform unit / inverse transform unit, it can be transform related information, in the quantization unit / inverse quantization unit, it can be quantization related information, in the encoding unit / decoding unit, it can be encoding / decoding related information (context information), in the in-loop filtering unit, it can be filtering related information, etc.

[0073] The smallest unit constituting a video may be a pixel, and the number of bits required to represent one pixel is called the bit depth. Typically, the bit depth may be 8 bits, and a bit depth of more than 8 bits may be supported depending on the encoding setting.

[0074] The bit depth may support at least one bit depth according to the color space. In addition, it may be composed of at least one color space according to the color format of the video. It may be composed of more than one image with a fixed size or more than one image with different sizes according to the color format. For example, in the case of YCbCr4∶2∶0, it may include a brightness component (Y in this example) and two color difference components (Cb / Cr in this example), and the ratio of the color difference component to the brightness component may be 1∶2 horizontally and vertically. As another example, in the case of 4∶4∶4, it may have the same ratio horizontally and vertically. In the case of including more than one color space as described above, segmentation of the image on each color space may be performed.

[0075] In the present invention, a part of the color space (in this example, Y) of a part of the color format (in this example, YCbCr) is used as a reference for explanation, and the same or similar application can also be made to other color spaces (in this example, Cb, Cr) based on the color format (depending on the settings of the specific color space). However, a part of the difference can also be retained in each color space (independent of the settings of the specific color space). That is, the setting that depends on each color space can refer to a setting that is proportional to the composition ratio of each component (for example, determined according to 4:2:0, 4:2:2, 4:4:4, etc.) or a dependent setting, and the setting that is independent of each color space can refer to a setting that is independent of the composition ratio of each component or is independent and only applies to the corresponding color space. In the present invention, a part of the composition can be set independently or dependently according to the encoder / decoder.

[0076] The setting information or syntax elements (Syntax Element) required in the video encoding process can be determined at the unit level of video, sequence, image, segment, block, block, etc., and can be included in the bitstream and transmitted to the decoder in units such as VPS (Video Parameter Set), SPS (Sequence Parameter Set), PPS (Picture Parameter Set), Slice Header, Tile Header, Block Header, etc. The decoder can parse (Parsing) at the same level of units and reconstruct the setting information transmitted from the encoder for the video decoding process. In addition, the relevant information can be transmitted to the bitstream in the form of SEI (Supplement Enhancement Information) or metadata (Metadata) and parsed for use. Each parameter set has an inherent ID value, and the lower parameter set may have an ID value of the upper parameter set that needs to be referenced. For example, the lower parameter set can refer to the information of the upper parameter set with a consistent ID value in more than one upper parameter set. In the examples of various units described above, when a certain unit includes more than one other unit, the corresponding unit may be referred to as a superordinate unit, and the included unit may be referred to as a subordinate unit.

[0077] In the case of setting information generated by the units, each corresponding unit may include independent setting content, or content that depends on the setting of the previous, next or upper unit, etc. Here, the dependent setting can be understood as flag information for following the setting of the previous, next or upper unit (for example, when the 1-bit flag is 1, it is followed, and when it is 0, it is not followed) to represent the setting information of the corresponding unit. In the present invention, the setting information is explained with the example of independent settings as the center, but it may also include examples of content added to or replaced with a relationship of dependency on the setting information of the previous, next or upper unit of the current unit.

[0078] Figure 1 is a block diagram of a video encoding apparatus according to an embodiment of the present invention. Figure 2 is a block diagram of a video decoding apparatus according to an embodiment of the present invention.

[0079] Reference Figure 1 The video encoding device may include a prediction unit, a subtraction unit, a transformation unit, a quantization unit, an inverse quantization unit, an inverse transformation unit, an addition unit, an in-loop filtering unit, a memory and / or an encoding unit. It may not include some of the above structures, may selectively include some or all of the structures according to the specific implementation, and may include another part of the structures not shown in the figure.

[0080] Reference Figure 2 The video decoding device may include a decoding unit, a prediction unit, an inverse quantization unit, an inverse transformation unit, an addition unit, an in-loop filtering unit and / or a memory. It may not include some of the above structures, may selectively include some or all of the structures according to the specific implementation, and may include another part of the structures not shown in the figure.

[0081] The video encoding device and the video decoding device may be separate devices, but may be configured as one video encoding / decoding device according to specific implementation. In this case, a part of the video encoding device is configured as a substantially same technical element as a part of the video decoding device, and may be implemented in a manner that includes at least the same structure or performs at least the same function.

[0082] Therefore, when the following technical elements and their operating principles are described in detail, the repeated description of the corresponding technical elements is omitted. The video decoding device is equivalent to a computing device that applies the video encoding method performed by the video encoding device to decoding, so in the following description, the video encoding device is used as the center for description. The video encoding device can be referred to as an encoder, and the video decoding device can be referred to as a decoder.

[0083] The prediction unit may include an intra-frame prediction unit for performing intra-frame prediction and an inter-frame prediction unit for performing inter-frame prediction. Intra-frame prediction may use pixels of blocks adjacent to the current block as reference pixels to determine the intra-frame prediction mode, and use the intra-frame prediction mode to generate a prediction block. Inter-frame prediction may use more than one reference image to determine the motion information of the current block, and use the motion information to perform motion compensation to generate a prediction block. After determining which method of intra-frame prediction or inter-frame prediction is to be used for the current block (coding unit or prediction unit), the specific information corresponding to each prediction method (for example, intra-frame prediction mode, motion vector, reference image, etc.) may be determined. At this time, the processing unit for performing prediction and the processing unit for determining the prediction method and specific content may be determined according to the encoding / decoding settings. For example, the prediction method, prediction mode, etc. are determined in prediction units (or coding units), and the prediction is performed in prediction block units (or coding units, transform units).

[0084] The subtraction unit generates a residual block by subtracting the prediction block from the current block. That is, the subtraction unit calculates the difference between the pixel value of each pixel of the current block to be encoded and the predicted pixel value of each pixel of the prediction block generated by the prediction unit to generate a residual signal (Residual Signal) in the form of a block, i.e., a residual block.

[0085] The transform unit may transform a signal belonging to the spatial domain into a signal belonging to the frequency domain. In this case, the signal obtained through the transform process is called a transform coefficient. For example, a residual block having a residual signal received from the subtraction unit may be transformed to obtain a transform block having a transform coefficient. The received signal is determined by the encoding setting and is not limited to the residual signal.

[0086] The transformation unit may use transformation methods such as Hadamard Transform, Discrete Sine Transform (DST Based-Transform, Discrete Sine Transform), Discrete Cosine Transform (DCT Based-Transform, Discrete Cosine Transform) to transform the residual block, but is not limited to these. Various transformation methods obtained by improving and deforming these transformation methods may also be used.

[0087] The transformation can be performed in the horizontal / vertical direction. For example, a two-dimensional transformation can be performed by performing a one-dimensional transformation in the horizontal direction and a one-dimensional transformation in the vertical direction using the basis vectors in the transformation, thereby transforming the pixel value in the spatial domain into the pixel value in the frequency domain.

[0088] In addition, the transformation unit can transmit the information required to generate the transformation block to the encoding unit to encode the information, include the information obtained in the bit stream and transmit the bit stream to the decoder, and the decoding unit of the decoder can parse the information of the bit stream for the inverse transformation process.

[0089] The quantization unit may quantize the received signal, and in this case, the signal obtained through the quantization process is called a quantized coefficient. For example, a residual block having a residual transform coefficient received from the transform unit may be quantized to obtain a quantized block having a quantized coefficient, and the received signal is determined by the encoding setting and is not limited to the residual transform coefficient.

[0090] The quantization unit may quantize the transformed residual block using quantization methods such as Dead Zone Uniform Threshold Quantization and Quantization Weighted Matrix, but is not limited thereto and various quantization methods obtained by improving and deforming these quantization methods may also be used.

[0091] The quantization unit can transmit the information required to generate the quantization block to the encoding unit to encode the information, include the information obtained in the bit stream and transmit the bit stream to the decoder, and the decoding unit of the decoder can parse the information of the bit stream for the inverse quantization process.

[0092] In the above example, the transformation and quantization of the residual block by the transformation unit and the quantization unit are used as an example for explanation, but the residual block may be transformed into a residual signal to generate a residual block having a transformation coefficient without performing a quantization process, the residual signal of the residual block may not be transformed into a transformation coefficient but only a quantization process may be performed, or neither the transformation nor the quantization process may be performed. This may be determined according to the encoder setting.

[0093] The encoding unit may scan the quantization coefficients, transform coefficients, or residual signals of the generated residual block in accordance with at least one scanning order (e.g., zigzag scanning, vertical scanning, horizontal scanning, etc.) to generate a quantization coefficient sequence, a transform coefficient sequence, or a signal sequence, and encode using at least one entropy coding method. At this time, the information of the scanning order may be determined according to the encoding settings (e.g., encoding mode, prediction mode, etc.), and the relevant information may be implicitly determined or explicitly generated. For example, one of a plurality of scanning orders may be selected according to the intra-frame prediction mode. At this time, the scanning pattern may be set to one of various patterns such as zig-zag, diagonal, and raster.

[0094] In addition, coded data including coded information transmitted from each component may be generated and output to a bit stream, which may be achieved by a multiplexer (MUX). At this time, methods such as Exponential Golomb, Context Adaptive Variable Length Coding (CAVLC), Context Adaptive Binary Arithmetic Coding (CABAC) may be used as coding methods for encoding, but are not limited thereto, and various coding methods obtained by improving and deforming these coding methods may also be used.

[0095] When entropy coding (in this example, CABAC is assumed) is performed on the syntax elements (Syntax) such as the residual block data and the information generated in the encoding / decoding process, the entropy coding device may include a binarizer, a context modeler, and a binary arithmetic coder. At this time, the binary arithmetic coder may include a regular coding engine and a bypass coding engine. At this time, the regular coding engine may be a process performed in relation to the context modeler, and the bypass coding engine may be a process performed independently of the context modeler.

[0096] The syntax elements input to the entropy coding device may not be binary values, so when the syntax elements are not binary values, the binarization unit may binarize the syntax elements and output a binary string (BinString) consisting of 0 or 1. At this time, the binary represents bits consisting of 0 or 1, which can be encoded by the binary arithmetic coding unit. At this time, one of the conventional coding unit or the bypass coding unit can be selected based on the generation probability of 0 and 1, which can be determined according to the encoding / decoding settings. If the syntax element is data with the same frequency of 0 and 1, the bypass coding unit can be used, otherwise the conventional coding unit can be used, which can be referenced when the next conventional coding unit is executed through context modeling (or context information update).

[0097] At this time, the context is information about the generation probability of a binary (Bin), and context modeling is a process of estimating the probability of a binary required for binary arithmetic coding by inputting a binarization result, i.e., a binary. In order to estimate the probability, the syntax element information of the binary, the position of the binary in the binary string, i.e., the index, the probability of the binary contained in the block surrounding the block, etc. can be used, and in order to estimate the probability, at least one context table can be used. For example, information for a part of the flag can use a plurality of context tables depending on whether the combination of the flags is used by the surrounding blocks.

[0098] When binarization is performed on the syntax element, various methods can be used. For example, it can be divided into fixed length binarization (Fixed Length Binarization) and variable length binarization (Variable Length Binarization). In the case of variable length binarization, unary binarization (Unary Binarization, Trunacted Unary Binarization), Rice binarization (Truncated Rice Binarization), K-th Exp-Golomb binarization, Truncated binary (truncated binary) binarization, etc. can be used. In addition, signed binarization or unsigned binarization can be performed according to the range of values ​​​​of the syntax element. The binarization process for the syntax element in the present invention can not only perform the binarization described in the above example, but also perform other binarization methods besides this.

[0099] The inverse quantization unit and the inverse transformation unit may perform the processes in the transformation unit and the quantization unit in reverse. For example, the inverse quantization unit may inversely quantize the quantized transformation coefficients generated by the quantization unit, and the inverse transformation unit may inversely transform the inversely quantized transformation coefficients to generate a reconstructed residual block.

[0100] The adding unit adds the prediction block to the reconstructed residual block to reconstruct the current block. The reconstructed block can be stored in a memory and used as reference data (for the prediction unit and the filtering unit, etc.).

[0101] The in-loop filtering section may include at least one post-processing filtering process of deblocking filtering, sample adaptive offset (SAO), adaptive loop filtering (ALF), etc. Deblocking filtering can remove block distortion occurring at the boundary between blocks from the reconstructed video. ALF can perform filtering according to a value obtained by comparing the reconstructed video with the input video. In detail, after filtering the block by deblocking filtering, filtering can be performed according to a value obtained by comparing the reconstructed video with the input video. Alternatively, filtering can be performed according to a value obtained by comparing the reconstructed video with the input video after filtering the block by SAO.

[0102] The memory can store the reconstructed blocks or images. The reconstructed blocks or images stored in the memory can be provided to a prediction unit that performs intra-frame prediction or inter-frame prediction. In detail, the storage space in the form of a queue of the bit stream compressed in the encoder can be used as a coded picture buffer (CPB) for processing, and the space for storing decoded video in picture units can be used as a decoded picture buffer (DPB) for processing. In the CPB, the decoding units can be stored in the decoding order, the decoding action can be simulated in the encoder, and the bit stream compressed in the simulation process can be stored. The bit stream output from the CPB can be reconstructed through the decoding process, and the reconstructed video is stored in the DPB. The image stored in the DPB is referenced in the subsequent video encoding and decoding process.

[0103] The decoding unit can reverse the process in the encoding unit. For example, it can receive a quantization coefficient sequence, a transformation coefficient sequence or a signal sequence from a bit stream and decode them, and can parse the decoded data containing the decoding information and transmit it to each component.

[0104] The video encoding device and the video decoding device of one embodiment of the present invention may include a block segmentation unit. The basic coding unit may refer to the basic (or starting) unit of prediction, transformation, quantization, etc. in the video encoding / decoding process. At this time, the coding unit may be composed of a brightness coding block and two color difference coding blocks according to the color format (YCbCr in this example), and the size of each block may be determined according to the color format. In the example described below, the block (in this example, the brightness component) is used as a reference for explanation. At this time, it is assumed that the block is a unit that can be obtained after each unit is determined, and similar settings can be applied to other types of blocks for explanation.

[0105] The block division unit can be set according to each component of the video encoding device and the decoding device, and the size and shape of the block can be determined through this process. At this time, the set block can be defined differently according to the component, and can be a prediction block in the prediction unit, a transform block in the transform unit, and a quantization block in the quantization unit. It is not limited to this, and block units corresponding to other components can also be defined. The size and shape of the block can be defined according to the horizontal length and vertical length of the block.

[0106] In the block division unit, blocks can be expressed as M×N, and the maximum and minimum values ​​of each block can be obtained within a range. For example, the shape of the block supports a square, and when the maximum value of the block is set to 256×256 and the minimum value is set to 8×8, a block of 2m×2m size (in this example, m is an integer from 3 to 8, such as 8×8, 16×16, 32×32, 64×64, 128×128, 256×256), a block of 2m×2m size (in this example, m is an integer from 4 to 128), or a block of m×m size (in this example, m is an integer from 8 to 256) can be obtained. Alternatively, the block shape supports squares and rectangles. When having the same range as the above example, a block of 2m×2n size can be obtained (in this example, m and n are integers from 3 to 8. When the aspect ratio is up to 2:1, for example, 8×8, 8×16, 16×8, 16×16, 16×32, 32×16, 32×32, 32×64, 64×32, 64×64, 64×128, 128×64, 128×128, 128×256, 256×128, 256×256. Depending on the encoding / decoding settings, the aspect ratio may be unlimited or there may be a maximum value of the aspect ratio). Alternatively, a block of 2m×2n size can be obtained (in this example, m and n are integers from 4 to 128). Alternatively, a block of m×n size can be obtained (in this example, m and n are integers from 8 to 256).

[0107] In the block division unit, the input block can obtain more than one block. For example, the input block can be directly output, or divided into two or more sub-blocks. At this time, the number of sub-blocks can be 2, 3, 4, etc., which can be determined according to the division method (division type). The form of the sub-block can be obtained based on the block before division (in this example, the input block), and can be in various forms according to the block division setting.

[0108] In the present invention, the tree-based partitioning method is mainly described, but the present invention is not limited to this.

[0109] The blocks that can be obtained can be determined according to the encoding / decoding settings (e.g., the type of block, the partitioning method, the partitioning settings, etc.). For example, a coding block can obtain a block of 2m×2n size, a prediction block can obtain a block of 2m×2n or m×n size, and a transform block can obtain a block of 2m×2n size. Information such as block size and range (e.g., information about exponents, multiples, etc.) can be generated based on the above settings.

[0110] The range may be determined according to the type of block (in this example, determined according to the maximum value and the minimum value). In addition, some blocks may explicitly generate block range information, and some blocks may implicitly determine block range information. For example, a coding block and a transform block may explicitly generate relevant information, and a prediction block may implicitly process relevant information.

[0111] In the case of explicit indication, at least one range information may be generated. For example, in the case of a coding block, as information about the range, information about the maximum value and the minimum value may be generated. Alternatively, it may be generated based on the difference between the maximum value and a preset minimum value (e.g., 8) (e.g., generated according to the setting, exponential difference information of the maximum value and the minimum value, etc.). In addition, information of multiple ranges of the horizontal length and the vertical length of a rectangular block may be generated.

[0112] In the case of an implicit, range information may be obtained based on encoding / decoding settings (e.g., block type, partitioning method, partitioning settings, etc.). For example, in the case of a prediction block, maximum and minimum value information may be obtained based on a candidate group (M×N and m / 2×n / 2 in this example) that may be obtained from a coding block as a superordinate unit (e.g., a coding block with a maximum size of M×N and a minimum size of m×n) based on a partitioning setting of the prediction block (e.g., quadtree partitioning+partitioning depth 0).

[0113] The size and shape of the initial (or starting) block of the block partitioning part can be determined according to the upper unit. In the case of a coding block, a basic coding block can be the initial block, in the case of a prediction block, a coding block can be the initial block, and in the case of a transform block, a coding block or a prediction block can be the initial block, which can be determined according to the encoding / decoding settings. For example, when the coding mode is intra, the prediction block can be the upper unit of the transform block, and when the coding mode is inter, the prediction block can be a unit independent of the transform block. The initial block, as the starting unit of the partition, can be partitioned into blocks of small sizes. If the optimal partition size and shape of each block are determined, the corresponding block can be determined as the initial block of the lower unit. For example, the former can be a coding block, and the latter (lower unit) can be a prediction block or a transform block. As in the above example, if the initial block of the lower unit is determined, the partition process of finding the block of the optimal size and shape can be performed like the upper unit.

[0114] In summary, the block partitioning unit may partition a basic coding unit (or a maximum coding unit) into at least one coding unit (or a lower coding unit). In addition, a coding unit may be partitioned into at least one prediction unit and may be partitioned into at least one transform unit. A coding unit may be partitioned into at least one coding block, a coding block may be partitioned into at least one prediction block, and may be partitioned into at least one transform block. A prediction unit may be partitioned into at least one prediction block, and a transform unit may be partitioned into at least one transform block.

[0115] As in the above example, when searching for a block of optimal size and shape through a mode determination process, mode information (e.g., segmentation information, etc.) about the block is generated. The mode information can be included in the bitstream together with information generated by a component to which the block belongs (e.g., prediction-related information, transformation-related information, etc.) and transmitted to the decoder, and the decoder can parse it at the same level and use it in the video decoding process.

[0116] In the example described below, the segmentation method is explained by taking the initial block in a square shape as an example, but a similar application can also be achieved when the initial block is in a rectangular shape.

[0117] Figure 3 This is an example diagram showing a tree-based block form.

[0118] a represents an example of obtaining a 2N×2N without performing a split, b represents an example of obtaining two 2N×N by a part of a split flag (in this example, the horizontal split of the binary tree), c represents an example of obtaining two N×2N by a part of a split flag (in this example, the vertical split of the binary tree), and d represents an example of obtaining 4 N×N by a part of a split flag (in this example, the four splits of the quadtree). The form of the obtained block can be determined according to the type of tree used in the split. For example, in the case of performing a quad tree split, the candidate blocks that can be obtained may be a and d. In the case of performing a binary tree split, the candidate blocks that can be obtained may be a, b, and c. The quad tree supports a split flag, and a can be obtained when the flag is "0", and d can be obtained when it is "1". The binary tree supports multiple split flags, one of which may be a flag indicating whether to perform a split, and one of which may be a flag indicating whether to perform a horizontal / vertical split. Quadtree can be a basic tree-based partitioning method. In addition, a tree partitioning method (binary tree in this example) can be included in the tree-based partitioning method. When a flag that allows additional tree partitioning to be performed is implicitly or explicitly activated, multiple tree partitioning can be performed. Tree-based partitioning can be a method that can achieve recursive partitioning (Recursive Partition). That is, the partitioned block can be set as the initial block again and the tree-based partitioning can be performed, which can be determined according to the partitioning settings such as the partitioning range and the partitioning allowed depth. This partitioning can be an example of a hierarchical partitioning method.

[0119] Figure 4 This is an illustrative diagram showing various block forms that can be obtained by the block dividing unit of the present invention.

[0120] Reference Figure 4 , blocks a to s can be obtained according to the segmentation settings and segmentation methods, and other block forms not shown in the figure can also be realized.

[0121] As an example (1), tree-based partitioning may allow asymmetric partitioning. For example, in the case of a binary tree, blocks such as b and c may be implemented, or blocks such as b to g may be implemented. In the case where the flag allowing asymmetric partitioning is explicitly or implicitly prohibited according to the encoding / decoding setting, the candidate blocks that can be obtained may be b or c, and in the case where the flag allowing asymmetric partitioning is activated, the candidate blocks that can be obtained may be b, d, e (horizontal partitioning in this example), or may be c, f, g (vertical partitioning in this example).

[0122] In the above examples, the length ratio of left:right or top:bottom of the asymmetric partition is taken as 1:3 or 3:1, but it is not limited to this. Candidate groups with ratios other than these (for example, 1:2, 1:4, 2:3, 2:5, 3:5, etc.) may also be included according to the encoding settings.

[0123] Next, various examples of partition information generated in binary tree partition (in this example, candidate groups of 1:1, 1:3, and 3:1) are described.

[0124] For example, a flag indicating the segmentation form may be generated in addition to a flag indicating whether to perform segmentation and a flag indicating the segmentation direction. In this case, the segmentation form may refer to symmetrical segmentation or asymmetrical segmentation. In the case where asymmetrical segmentation is determined as the segmentation form, a flag indicating the segmentation ratio is generated, and an index may be assigned according to a preset candidate group. In the case where the candidate group supports a segmentation ratio of 1:3 or 3:1, the segmentation ratio may be selected by a 1-bit flag.

[0125] Alternatively, a flag indicating the division ratio may be generated separately from a flag indicating whether to perform division and a flag indicating the division direction. In this example, the candidate group of the division ratio may include candidates having a symmetric ratio of 1:1.

[0126] In the present invention (when the flag for allowing asymmetric partitioning is activated), it is assumed that the binary tree partitioning has the structure as exemplified above, and unless otherwise specified, the binary tree refers to a symmetric binary tree.

[0127] As an example (2), tree-based segmentation may allow for additional tree segmentation. For example, segmentation using a ternary tree, a quad tree, an octa tree, etc. may be performed, thereby obtaining n segmentation blocks (3, 4, 8 in this example, where n is an integer). The blocks supported by the ternary tree (in this example, the blocks are divided into multiple blocks) may be h to m, the blocks supported by the quad tree may be n to p, and the blocks supported by the octa tree may be q. As to whether the tree-based segmentation is supported, relevant information may be implicitly determined or explicitly generated according to the encoding / decoding settings. In addition, it may be used alone or mixed with binary tree segmentation, quad tree segmentation, etc. according to the encoding / decoding settings.

[0128] For example, in the case of a binary tree, blocks such as b and c can be realized, and in the case of a mixed use of a binary tree and a ternary tree (in this example, it is assumed that the use range of the binary tree overlaps with a part of the use range of the ternary tree), blocks such as b, c, i, l can be realized. In the case where the flag that allows other partitions in addition to the existing tree partition is explicitly or implicitly prohibited according to the encoding / decoding setting, the candidate blocks that can be obtained may be b or c, and in the case of being activated, the candidate blocks that can be obtained may be b, i or b, h, i, j (in this example, horizontal partition), or may be c, l or c, k, l, m (in this example, vertical partition).

[0129] In the above examples, it is assumed that the length ratio of left: middle: right or top: middle: bottom of the ternary tree partition is 2:1:1, 1:2:1 or 1:1:2 for illustration, but it is not limited thereto and may have other ratios according to encoding settings.

[0130] Next, an example of segmentation information generated in ternary tree segmentation (in this example, 1:2:1 candidates) will be described.

[0131] For example, a flag indicating whether to perform segmentation or not and a flag indicating the segmentation direction may be separately generated as a flag indicating the segmentation type. In this case, the segmentation type may be binary tree segmentation or ternary tree segmentation.

[0132] In the present invention, adaptive encoding / decoding settings may be applied according to the partitioning manner.

[0133] As an example, the partitioning method may be determined according to the type of block. For example, the coding block and the transform block may be partitioned using a quadtree, and the prediction block may be partitioned using a quadtree or a binary tree (or ternary tree, etc.).

[0134] As an example, the partitioning method may be determined according to the size of the block. For example, between the maximum value and the minimum value of the block, quadtree partitioning may be performed on a portion of the range (e.g., a×b to c×d, where the latter is larger), and binary tree (or ternary tree, etc.) partitioning may be performed on a portion of the range (e.g., e×f to g×h). In this case, range information corresponding to the partitioning method may be explicitly generated or implicitly determined, and the ranges may overlap.

[0135] As an example, the partitioning method may be determined according to the shape of the block (or the block before partitioning). For example, when the block is in a square shape, quadtree partitioning and binary tree (or ternary tree, etc.) partitioning may be performed. Alternatively, when the block is in a rectangular shape, binary tree (or ternary tree, etc.) partitioning may be performed.

[0136] As an example, the partition setting may be determined according to the type of block. For example, in tree-based partitioning, the coding block and the prediction block may be partitioned using a quadtree, and the transform block may be partitioned using a binary tree. Alternatively, in the case of the coding block, the partition allowable depth may be set to m, in the case of the prediction block, the partition allowable depth may be set to n, and in the case of the transform block, the partition allowable depth may be set to o, and m, n, and o may be the same or different.

[0137] As an example, the segmentation setting may be determined according to the size of the block. For example, a portion of the range of the block (e.g., a×b to c×d) may be segmented by a quadtree, a portion of the range (e.g., e×f to g×h, in this example, assuming that c×d is greater than g×h) may be segmented by a binary tree, and a portion of the range (e.g., i×j to k×l, in this example, assuming that g×h is greater than or equal to k×l) may be segmented by a ternary tree. At this time, the range may include all ranges between the maximum value and the minimum value of the block, and the ranges may be set to be non-overlapping or overlapping with each other. For example, the minimum value of a portion of the range may be the same as the maximum value of a portion of the range, or the minimum value of a portion of the range may be less than the maximum value of a portion of the range. In the case of overlapping ranges, the segmentation method with a larger maximum value may be prioritized, or information on which segmentation method to use may be explicitly generated. That is, among the segmentation methods with priority, whether to execute the segmentation method ranked lower may be determined based on the segmentation result, or which segmentation method to use may be determined based on the segmentation method selection information.

[0138] As an example, the partitioning setting may be determined according to the shape of the block. For example, if the block is in a square shape, quadtree partitioning may be performed. Alternatively, if the block is in a rectangular shape, binary tree partitioning or ternary tree partitioning may be performed.

[0139] As an example, the segmentation setting may be determined based on encoding / decoding information (e.g., segment type, color component, encoding mode, etc.). For example, when the segment type is I, a portion of the range (e.g., a×b to c×d) may be segmented by a quadtree (or a binary tree, a ternary tree), when the segment type is P, a portion of the range (e.g., e×f to g×h) may be segmented by a quadtree (or a binary tree, a ternary tree), and when the segment type is B, a portion of the range (e.g., i×j to k×l) may be segmented by a quadtree (or a binary tree, a ternary tree). In addition, when the segment type is I, the segmentation allowable depth of the quadtree (or binary tree, ternary tree segmentation) segmentation may be set to m, when the segment type is P, the segmentation allowable depth may be set to n, and when the segment type is B, the segmentation allowable depth may be set to o, and m, n, and o may be the same or different. Some segment types may have the same settings as other segments (e.g., P and B segments).

[0140] As another example, when the color component is a brightness component, the allowed depth of the quadtree (or binary tree, ternary tree) segmentation can be set to m, and when the color component is a color difference component, the allowed depth of the quadtree (or binary tree, ternary tree) segmentation can be set to n, and m and n can be the same or different. In addition, the quadtree (or binary tree, ternary tree) segmentation range (for example, a×b to c×d) when the color component is a brightness component and the quadtree (or binary tree, ternary tree) segmentation range (for example, e×f to g×h) when the color component is a color difference component can be the same or different.

[0141] As another example, when the coding mode is intra (Intra), the allowed depth of the quadtree (or binary tree, ternary tree) segmentation may be m, and when the coding mode is inter (Inter), the allowed depth of the quadtree (or binary tree, ternary tree) segmentation may be n (in this example, it is assumed that n is greater than m), and m and n may be the same or different. In addition, the quadtree (or binary tree, ternary tree) segmentation range when the coding mode is intra (Intra) may be the same as or different from the quadtree (or binary tree, ternary tree) segmentation range when the coding mode is inter (Inter).

[0142] In the case of the above-mentioned example, the information on whether or not to support the formation of the adaptive partition candidate group corresponding to the encoding / decoding information may be explicitly generated, or may be implicitly determined.

[0143] Through the above examples, the case where the segmentation mode and segmentation setting are determined according to the encoding / decoding setting is described. The above examples represent a part of the cases corresponding to each element, and can also be changed to other cases. In addition, the segmentation mode and segmentation setting can also be determined according to a combination of multiple elements. For example, the segmentation mode and segmentation setting can be determined according to the type, size, shape, encoding / decoding information, etc. of the block.

[0144] In the above example, the elements related to the division method and division setting may be implicitly determined, or information may be explicitly generated to determine whether or not the adaptation situation as in the above example is permitted.

[0145] In the segmentation setting, the segmentation depth refers to the number of times the initial block is segmented in space (in this example, the segmentation depth of the initial block is 0). The deeper the segmentation depth, the smaller the block can be segmented. The depth-related settings can be changed according to the segmentation method. For example, in the method of performing tree-based segmentation, different depths can be used according to the type of tree. For example, the segmentation depth of a binary tree and the segmentation depth of a ternary tree can use the same depth, and the segmentation depth of a quadtree and the segmentation depth of a binary tree can use different depths, etc.

[0146] In the above example, when different segmentation depths are used depending on the type of tree, the segmentation depth of the tree segmentation start position (in this example, the block before segmentation) can be set to 0. The segmentation depth can be calculated based on the position where segmentation starts, not based on the segmentation range of each tree (in this example, the maximum value), but with the segmentation start position as the center.

[0147] Figure 5 The block segmentation process of one embodiment of the present invention is shown. Figure 5 , examples of block sizes and shapes that can be obtained using one or more partitioning methods starting with a basic coding block are shown.

[0148] In the figure, the thick solid line represents the basic coding block, the thick dotted line represents the quadtree segmentation boundary, the double solid line represents the symmetric binary tree segmentation boundary, the solid line represents the ternary tree segmentation boundary, and the thin dotted line represents the asymmetric binary tree segmentation boundary. Except for the thick solid line, the remaining lines represent the boundaries divided according to each segmentation method. The segmentation settings (for example, segmentation type, segmentation information, segmentation information composition order, etc.) described below are not limited to this example, and various modifications can be implemented.

[0149] For the sake of convenience, it is assumed that the basic coding block (2N×2N, 128×128) is used as a reference and different block partition settings are used in the upper left block, upper right block, lower left block, and lower right block (N×R, 64×64). First, it is assumed that four sub-blocks are obtained by performing a partition operation on the initial block (partition depth 0→1, that is, the partition depth increases by 1), and as the partition setting of the quadtree, the maximum coding block is 128×128, the minimum coding block is 8×8, and the maximum partition depth is 4, which is a common setting for each block.

[0150] (No. 1, upper left block, A0~A6)

[0151] In this example, single-tree partitioning is supported (in this example, quadtree), and the size and shape of the blocks that can be obtained can be determined based on a block partitioning setting such as the maximum coding block, the minimum coding block, and the partitioning depth. In this example, the block that can be obtained by partitioning is one (dividing the horizontal and vertical parts into two parts respectively), and the partitioning information required for one partitioning action (taking the block 4M×4N before the partition as the benchmark, and increasing the partitioning depth by 1) is a flag indicating whether to perform the partitioning (in this example, if it is 0, it will not be partitioned, and if it is 1, it will be partitioned), and the candidates that can be obtained are 4M×4N and 2M×2N.

[0152] (No. 2, upper right block, A7~A11)

[0153] In this example, multiple tree partitioning methods are supported (in this example, quadtree and binary tree), and the size and shape of the blocks that can be obtained can be determined by multiple block partitioning settings. In this example, it is assumed that in the case of a binary tree, the maximum coding block is 64×64, the length of one side of the minimum coding block is 4, and the maximum partition depth is 4.

[0154] In this example, the number of blocks that can be obtained through partitioning is more than two (2 or 4 in this example), and the partitioning information required for one partitioning action (the quadtree partitioning depth increases by 1) is a flag indicating whether to perform partitioning, a flag indicating the type of partitioning, a flag indicating the partitioning shape, and a flag indicating the partitioning direction. The candidates that can be obtained are 4M×4N, 4M×2N, 2M×4N, 4M×N / 4M×3N, 4M×3N / 4M×N, M×4N / 3M×4N, and 3M×4N / M×4N.

[0155] In the case where the quadtree segmentation range overlaps with the binary tree segmentation range (i.e., the range in which both quadtree segmentation and binary tree segmentation can be performed in the current step) and the current block (the state before segmentation) is a block obtained by quadtree segmentation (a block obtained from a parent block by quadtree segmentation), the segmentation information can be divided into the following cases. That is, in the case where a block supported by each segmentation setting can be obtained by a plurality of segmentation methods, the segmentation information can be generated by the following process.

[0156] (1) Overlap between quadtree partitioning and binary tree partitioning

[0157] [Table 1]

[0158] a b c d e QT 1 No Split 0 0 SBT 0 1 0 0 ABT hor 1 / 4 0 1 0 1 0 ABT hor 3 / 4 0 1 0 1 1 SBT ver 0 1 1 0 ABT ver 1 / 4 0 1 1 1 0 ABT ver 3 / 4 0 1 1 1 1

[0159] In the above table, a refers to a flag indicating whether to perform quadtree splitting. If it is 1, quadtree splitting (QT) is performed. If the flag is 0, the flag indicating whether to perform binary tree splitting, i.e., b, is confirmed. If b is 0, the corresponding block is no longer split (No Split), and if it is 1, binary tree splitting is performed.

[0160] c is a flag indicating the direction of the division. If it is 0, it indicates horizontal division (hor), and if it is 1, it indicates vertical division (ver). d is a flag indicating the division form. If it is 0, it indicates symmetric division (SBT, Symmetric Binary Tree), and if it is 1, it indicates asymmetric division (ABT, Asymmetric Binary Tree). Only when d is 1, the information of the fine division ratio in the asymmetric division is confirmed (1 / 4 or 3 / 4). When it is 0, in the left block / right block or upper block / lower block, the left block and the upper block have a ratio of 1 / 4, and the right block and the lower block have a ratio of 3 / 4. When it is 1, it has the opposite ratio.

[0161] (2) Only binary tree partitioning is possible

[0162] In the above table, the division information can be represented by flags b to e other than a.

[0163] exist Figure 5 In the block A7, quadtree division can be performed among the blocks (A7 to A11) before division (i.e., quadtree division can be performed, but binary division is performed instead of quadtree division). Therefore, this is equivalent to the case of generating the division information in (1).

[0164] On the contrary, A8 to A11 is a case where quadtree division is no longer performed in the block (A8~A11) before division, but binary division is performed (that is, the block <A8~A11> cannot be quadtree divided), which is equivalent to the case of generating the division information in (2).

[0165] (No. 3, lower left block, A12~A15)

[0166] In this example, multiple tree partitioning methods are supported (in this example, quadtree, binary tree, ternary tree), and the size and shape of the blocks that can be obtained can be determined according to multiple block partitioning settings. In this example, it is assumed that the maximum coding block in the case of binary tree / ternary tree is 64×64, the length of one side of the minimum coding block is 4, and the maximum partition depth is 4.

[0167] In this example, there are more than two blocks that can be obtained through segmentation (2, 3, or 4 in this example). The segmentation information required for a segmentation action is a flag indicating whether to perform segmentation, a flag indicating the type of segmentation, and a flag indicating the direction of segmentation. The candidates that can be obtained are 4M×4N, 4M×2N, 2M×4N, 4M×N / 4M×2N / 4M×N, and M×4N / 2M×4N / M×4N.

[0168] In the case where the partition range of the quadtree overlaps with the partition range of the binary tree / ternary tree and the current block is a block obtained by quadtree partitioning, the partition information can be classified into the following cases.

[0169] (1) Quadtree partitioning overlaps with binary / ternary tree partitioning

[0170] [Table 2]

[0171] a b c d QT 1 No Split 0 0 BT 0 1 0 0 TT 0 1 0 1 Bt ver 0 1 1 0 TT ver 0 1 1 1

[0172] In the above table, a is a flag indicating whether to perform quadtree segmentation. If it is 1, quadtree segmentation is performed. If the flag is 0, the flag indicating whether to perform binary tree segmentation or ternary tree segmentation, that is, b, is confirmed. If b is 0, the corresponding block is no longer segmented. If it is 1, binary tree segmentation or ternary tree segmentation is performed.

[0173] c is a flag indicating the direction of the partition. If it is 0, it indicates horizontal partition. If it is 1, it indicates vertical partition. d is a flag indicating the type of partition. If it is 0, it indicates binary tree partition (BT). If it is 1, it indicates ternary tree partition (TT).

[0174] (2) Only binary / ternary tree partitioning is possible

[0175] In the above table, the division information can be represented by flags b to d other than a.

[0176] exist Figure 5 In the example, blocks A12 and A15 are the cases where quadtree partitioning can be implemented in the blocks (A12 to A15) before partitioning, and thus correspond to the case where the partitioning information in (1) is generated.

[0177] On the contrary, A13 and A14 are cases where quadtree partitioning is not performed on the blocks before partitioning (A13, A14) but ternary tree partitioning is performed, which corresponds to the case where the partitioning information in (2) is generated.

[0178] (No. 4, lower left block, A16~A20)

[0179] In this example, multiple tree partitioning methods are supported (in this example, quadtree, binary tree, ternary tree), and the size and shape of the blocks that can be obtained can be determined according to multiple block partitioning settings. In this example, it is assumed that the maximum coding block in the case of binary tree / ternary tree is 64×64, the length of one side of the minimum coding block is 4, and the maximum partition depth is 4.

[0180] In this example, there are more than two blocks that can be obtained through segmentation (2, 3, or 4 in this example), and the segmentation information required for a segmentation action is a flag indicating whether to perform segmentation, a flag indicating the type of segmentation, a flag indicating the segmentation shape, and a flag indicating the segmentation direction. The candidates that can be obtained are 4M×4N, 4M×2N, 2M×4N, 4M×N / 4M×3N, 4M×3N / 4M×N, M×4N / 3M×4N, 3M×4N / M×4N, 4M×N / 4M×2N / 4M×N, and M×4N / 2M×4N / M×4N.

[0181] In the case where the partition range of the quadtree overlaps with the partition range of the binary tree / ternary tree and the current block is a block obtained by quadtree partitioning, the partition information can be classified into the following cases.

[0182] (1) Quadtree partitioning overlaps with binary / ternary tree partitioning

[0183] [Table 3]

[0184] a b c d e f QT 1 No Split 0 0 TT 0 1 0 0 SBT 0 1 0 1 0 ABT hor 1 / 4 0 1 0 1 1 0 ABT hor 3 / 4 0 1 0 1 1 1 TT ver 0 1 1 0 SBT ver 0 1 1 1 0 ABT ver 1 / 4 0 1 1 1 1 0 ABT ver 3 / 4 0 1 1 1 1 1

[0185] In the above table, a is a flag indicating whether to perform quadtree segmentation. If it is 1, quadtree segmentation is performed. If the flag is 0, the flag indicating whether to perform binary tree segmentation, i.e., b, is confirmed. If b is 0, the corresponding block is no longer segmented. If it is 1, binary tree segmentation or ternary tree segmentation is performed.

[0186] c is a flag indicating the direction of the division. If it is 0, it indicates horizontal division, and if it is 1, it indicates vertical division. d is a flag indicating the type of division. If it is 0, it indicates three-way division, and if it is 1, it indicates binary tree division. When d is 1, the flag e about the division form is confirmed. When e is 0, symmetric division is performed, and when e is 1, asymmetric division is performed. When e is 1, information about the fine division ratio in the asymmetric division is confirmed, which is the same as the above example.

[0187] (2) Only binary / ternary tree partitioning is possible

[0188] In the above table, the division information can be represented by flags b to f other than a.

[0189] exist Figure 5 In the example, block A20 is a case where quadtree partitioning can be performed on the blocks (A16 to A19) before the partitioning, and thus corresponds to the case where the partitioning information in (1) is generated.

[0190] On the contrary, A16 to A19 are cases where quadtree division is not performed on the blocks before division (A16 to A19) but binary tree division is performed, which is equivalent to the case where the division information in (2) is generated.

[0191] Hereinafter, an inter-frame prediction method based on a merge mode is described. In the merge mode, motion information can be derived in units of one block. Here, a block may refer to a coding block determined by the block partitioning. In addition, a coding block as a leaf node may refer to a block that cannot be further partitioned into a smaller coding block. In the merge mode, motion information can be derived in units of sub-blocks belonging to one block. To this end, a block may be further partitioned into a plurality of sub-blocks. For example, affine motion modeling may be performed based on the merge mode of the sub-block unit. The current block may selectively utilize any one of the merge modes of the block unit or the merge mode of the sub-block unit, for which a specific flag may be utilized. Here, the flag may be information indicating whether the merge mode of the sub-block unit can be applied. The flag may be encoded and transmitted in the encoding device based on specific encoding parameters, or derived from the decoding device. The encoding parameters may refer to the size / shape, component type, partition type, partition depth, etc. of the block. For example, if the size of the current block is greater than or equal to N×M, the flag may be passed, otherwise, the flag may be derived as a defined value. Here, N and M may be integers of 8, 16 or more. N and M may be the same or different. Figure 6 For an explanation of the block unit merging mode, refer to Figures 7 to 10 The merging mode of sub-block units is described.

[0192] Figure 6 As one embodiment to which the present invention is applied, this diagram shows a method of performing inter-frame prediction based on a merge mode of block units.

[0193] Reference Figure 6 , a merge candidate list of the current block can be constructed (S600).

[0194] The merge candidate list may include at least one of a spatial merge candidate or a temporal merge candidate for the current block.

[0195] The motion information of the spatial merge candidate may be derived based on the motion information of the spatial neighboring blocks of the current block. Here, the spatial neighboring block, as a block belonging to the same image as the current block, may refer to a block adjacent to the current block. The spatial neighboring block may include a block adjacent to at least one of the left side, top end, top right end, bottom left end, or top left end of the current block. The top left neighboring block may be used only when at least one of the blocks adjacent to the left side, top end, top right end, and bottom left end cannot be used.

[0196] The motion information of the temporal merge candidate can be derived based on the motion information of the temporal neighboring blocks of the current block. The temporal neighboring block, as a block belonging to a different image from the current block, can be defined as a block at the same position as the current block. Here, the block at the same position can refer to at least one of a block (BR) adjacent to the lower right corner of the current block, a block (CTR) containing the position of the central sample of the current block, or a block (TL) containing the position of the upper left sample of the current block. Alternatively, the block at the same position can also refer to a block that contains a position offset from the position of the upper left sample of the current block according to a specific displacement vector (disparity vector). Here, the displacement vector can be determined based on any motion vector of the motion vectors of the spatial neighboring blocks. Alternatively, the displacement vector can be determined based on a combination of at least two motion vectors of the motion vectors of the spatial neighboring blocks. The combination can refer to operations such as maximum value, minimum value, middle value, weighted average value, etc. For example, the displacement vector can be set according to the motion vector of the left neighboring block or the upper neighboring block. Alternatively, the displacement vector can be set according to the middle value or average value between the motion vector of the left neighboring block and the motion vector of the lower left neighboring block.

[0197] The motion vector and the reference image index of the temporal merging candidate may be derived according to the motion vector and the reference image index of the temporal neighboring block, respectively. Alternatively, the motion vector of the temporal merging candidate may be derived according to the motion vector of the temporal neighboring block, and the reference image index of the temporal merging candidate may be set according to a default value (e.g., 0) preset in the decoding device regardless of the temporal neighboring block.

[0198] The merge candidate list may further include a combined merge candidate. The combined merge candidate may be derived by combining n merge candidates belonging to the generated merge candidate list.

[0199] Here, n may be an integer of 2, 3, 4 or more. The number (n) of merge candidates as combination objects may also be a fixed value preset in the encoding / decoding device, or may be encoded and transmitted in the encoding device. The transmission may be performed in at least one unit of a sequence, an image, a fragment, a block, a sub-block (brick), or a specific block. The number (n) of merge candidates as combination objects may also be variably determined based on the number of remaining merge candidates. Here, the number of remaining merge candidates may refer to the difference between the maximum number of merge candidates that can be included in the merge candidate list and the current number of merge candidates belonging to the merge candidate list. The maximum number may be a number preset in the encoding / decoding device, or may be encoded and transmitted in the encoding device. The current number may refer to the number of merge candidates formed before the combined merge candidate is added. For example, when the number of remaining merge candidates is 1, two merge candidates may be used, and when the number of remaining merge candidates is greater than 1, 3 or more merge candidates may be used.

[0200] The positions of the n merge candidates may be preset positions in the merge candidate list. For example, each merge candidate belonging to the merge candidate list may be assigned an index (0 to (k-1)). Here, k may refer to the total number of merge candidates belonging to the merge candidate list. At this time, the positions of the n merge candidates may correspond to index 0 to index (n-1) in the merge candidate list. Alternatively, the n merge candidates may be determined by considering the prediction direction of each merge candidate belonging to the merge candidate list. For example, only bidirectionally predicted merge candidates may be selectively utilized among the merge candidates belonging to the merge candidate list, or only unidirectionally predicted merge candidates may be selectively utilized.

[0201] The combined merge candidate may be derived using both the spatial merge candidate and the temporal merge candidate, or may be derived using only one of the spatial merge candidate or the temporal merge candidate. For example, it may be limited to deriving the combined merge candidate using only the spatial merge candidate. In this case, the number of merge candidates to be combined may be limited to the number of spatial merge candidates in the generated merge candidate list.

[0202] The combined merge candidate may be appended to the spatial / temporal merge candidate in the merge candidate list. That is, the index of the combined merge candidate may be greater than the index of the spatial / temporal merge candidate. Alternatively, the combined merge candidate may also be appended between the spatial merge candidate and the temporal merge candidate in the merge candidate list. That is, the index of the combined merge candidate may be greater than the index of the spatial merge candidate and less than the index of the temporal merge candidate. Alternatively, the position of the combined merge candidate may also be variably determined by considering the prediction direction of the combined merge candidate. The position of the combined merge candidate may be rearranged in the merge candidate list according to whether the prediction direction of the combined merge candidate is a bidirectional prediction. For example, in the case where the combined merge candidate is a bidirectional prediction, an index smaller than the spatial merge candidate or the temporal merge candidate may be assigned, otherwise, an index larger than the spatial merge candidate or the temporal merge candidate may be assigned.

[0203] For the sake of convenience, a method of deriving a combined merge candidate based on two merge candidates is described below.

[0204] The motion information of the combined merge candidate can be derived by taking a weighted average of the motion information of the first merge candidate and the second merge candidate. Here, the weighted value of the weighted average is [1:1], [1:2], [1:3], [2:3], etc., but is not limited to this. The weighted value may be a weighted value defined in the encoding / decoding device, or may be derived from the decoding device. At this time, the weighted value may be derived by considering at least one of the distance between the current image and the reference image of the merge candidate or the prediction direction of the merge candidate. Alternatively, the motion information of the combined merge candidate may be derived in the following manner: obtaining the motion information in the L0 direction according to the first merge candidate, obtaining the motion information in the L1 direction according to the second merge candidate, and combining the motion information. The motion information of the combined merge candidate may be derived based on at least one of the derivation methods, and as described below, it may be derived by considering the prediction direction of the merge candidate as the combined object.

[0205] In this specification, the motion information may include at least one of a prediction direction flag, a reference image index, or a motion vector. The motion information may be defined for L0 prediction and L1 prediction, respectively. Here, L0 prediction may refer to prediction performed with reference to reference image list L0, and L1 prediction may refer to prediction performed with reference to reference image list L1.

[0206] 1. The first merge candidate and the second merge candidate are both unidirectional predictions

[0207] (Example 1) In the case where the first merge candidate is L0 prediction and the second merge candidate is L1 prediction, the reference image index of the L0 direction of the combined merge candidate may be derived according to the reference image index of the first merge candidate. The prediction direction flag of the L0 direction of the combined merge candidate may be derived as 1. The motion vector of the L0 direction of the combined merge candidate may be derived according to the motion vector of the first merge candidate. The reference image index of the L1 direction of the combined merge candidate may be derived according to the reference image index of the second merge candidate. The prediction direction flag of the L1 direction of the combined merge candidate may be derived as 1. The motion vector of the L1 direction of the combined merge candidate may be derived according to the motion vector of the second merge candidate.

[0208] (Example 2) In the case where the first merge candidate is L1 predicted and the second merge candidate is L0 predicted, the reference image index of the L0 direction of the combined merge candidate may be derived according to the reference image index of the second merge candidate. The prediction direction flag of the L0 direction of the combined merge candidate may be derived as 1. The motion vector of the L0 direction of the combined merge candidate may be derived according to the motion vector of the second merge candidate. The reference image index of the L1 direction of the combined merge candidate may be derived according to the reference image index of the first merge candidate. The prediction direction flag of the L1 direction of the combined merge candidate may be derived as 1. The motion vector of the L1 direction of the combined merge candidate may be derived according to the motion vector of the first merge candidate.

[0209] (Example 3) In the case where the first merge candidate and the second merge candidate are L0 predictions, the reference image index of the L0 direction of the combined merge candidate can be derived based on the reference image index of any merge candidate of the first merge candidate or the second merge candidate. For example, the reference image index of the merge candidate with the smallest index among the first merge candidate and the second merge candidate can be set as the reference image index of the L0 direction of the combined merge candidate. The prediction direction flag of the L0 direction of the combined merge candidate can be derived as 1. The motion vector of the L0 direction of the combined merge candidate can be derived based on the weighted average of the motion vector of the first merge candidate and the second merge candidate. The reference image index of the L1 direction of the combined merge candidate can be derived as -1, the prediction direction flag of the L1 direction can be derived as 0, and the motion information of the L1 direction can be derived as 0.

[0210] (Example 4) In the case where the first merge candidate and the second merge candidate are L1 predictions, the reference image index in the L0 direction of the combined merge candidate can be derived as -1, the prediction direction flag in the L0 direction can be derived as 0, and the motion information in the L0 direction can be derived as 0. The reference image index in the L1 direction of the combined merge candidate can be derived based on the reference image index of any merge candidate of the first merge candidate or the second merge candidate. For example, the reference image index of the merge candidate with the smallest index among the first merge candidate and the second merge candidate can be set as the reference image index in the L1 direction of the combined merge candidate. The prediction direction flag in the L1 direction of the combined merge candidate can be derived as 1. The motion vector in the L1 direction of the combined merge candidate can be derived based on the weighted average of the motion vector of the first merge candidate and the second merge candidate.

[0211] 2. The first merge candidate and the second merge candidate are both bidirectional predictions

[0212] (Example 5) The reference image index of the L0 direction of the combined merge candidate can be derived based on the reference image index of any merge candidate in the first merge candidate or the second merge candidate. For example, the reference image index of the merge candidate with the smallest index in the first merge candidate and the second merge candidate can be set as the reference image index of the L0 direction of the combined merge candidate. The predicted direction flag of the L0 direction of the combined merge candidate can be derived as 1. The motion vector of the L0 direction of the combined merge candidate can be derived based on the weighted average of the motion vector of the first merge candidate and the second merge candidate. The reference image index of the L1 direction of the combined merge candidate can be derived based on the reference image index of any merge candidate in the first merge candidate or the second merge candidate. For example, the reference image index of the merge candidate with the smallest index in the first merge candidate and the second merge candidate can be set as the reference image index of the L1 direction of the combined merge candidate. The predicted direction flag of the L1 direction of the combined merge candidate can be derived as 1. The motion vector of the L1 direction of the combined merge candidate can be derived based on the weighted average of the motion vector of the first merge candidate and the second merge candidate.

[0213] 3. The first merge candidate is a bidirectional prediction and the second merge candidate is a unidirectional prediction

[0214] (Example 6) In the case where the second merge candidate is an L0 prediction, the reference image index of the L0 direction of the combined merge candidate can be derived based on the reference image index of any merge candidate in the first merge candidate or the second merge candidate. For example, the reference image index of the merge candidate with the smallest index among the first merge candidate and the second merge candidate can be set as the reference image index of the L0 direction of the combined merge candidate. The prediction direction flag of the L0 direction of the combined merge candidate can be derived as 1. The motion vector of the L0 direction of the combined merge candidate can be derived based on the weighted average of the motion vector of the first merge candidate and the second merge candidate. The reference image index of the L1 direction of the combined merge candidate can be derived based on the reference image index of the first merge candidate. The prediction direction flag of the L1 direction of the combined merge candidate can be derived as 1. The motion vector of the L1 direction of the combined merge candidate can be derived based on the motion vector of the first merge candidate.

[0215] (Example 7) In the case where the second merge candidate is an L1 prediction, the reference image index of the L0 direction of the combined merge candidate can be derived based on the reference image index of the first merge candidate. The prediction direction flag of the L0 direction of the combined merge candidate can be derived as 1. The motion vector of the L0 direction of the combined merge candidate can be derived based on the motion vector of the first merge candidate. The reference image index of the L1 direction of the combined merge candidate can be derived based on the reference image index of any merge candidate of the first merge candidate or the second merge candidate. For example, the reference image index of the merge candidate with the smallest index among the first merge candidate and the second merge candidate can be set as the reference image index of the L1 direction of the combined merge candidate. The prediction direction flag of the L1 direction of the combined merge candidate can be derived as 1. The motion vector of the L1 direction of the combined merge candidate can be derived based on the weighted average of the motion vector of the first merge candidate and the second merge candidate.

[0216] Reference Figure 6 , the motion information of the current block can be derived according to the merge candidate list (S610).

[0217] Specifically, a merge index of the current block may be passed. The merge index may specify a merge candidate among multiple merge candidates in the merge candidate list. That is, a merge candidate having the same index as the merge index may be extracted, and the motion information of the current block may be derived using the motion information of the extracted merge candidate.

[0218] Reference Figure 6 , the derived motion information may be used to perform inter-frame prediction of the current block (S620).

[0219] Specifically, a reference image may be selected from a reference image list based on a reference image index of the current block. A reference block may be specified in the reference image based on a motion vector of the current block. Prediction pixels of the current block may be generated using reconstructed pixels of the specified reference block. The current block may be reconstructed by adding the prediction pixels of the current block to the residual pixels. Here, at least one of inverse quantization or inverse transformation may be performed on the residual coefficients transmitted through the bitstream to derive the residual pixels.

[0220] Figure 7 As an embodiment of the present invention, this diagram shows an inter-frame prediction method based on an affine model.

[0221] Reference Figure 7 , a candidate list for predicting motion information of the current block may be generated ( S700 ).

[0222] The candidate list may include one or more candidates based on an affine model (hereinafter referred to as affine candidates). An affine candidate may refer to a candidate having a control point vector. A control point vector refers to a motion vector of a control point used for an affine model, and the control point vector may be defined for a corner position of a block (e.g., at least one of an upper left corner, an upper right corner, a lower left corner, or a lower right corner).

[0223] The affine candidate may include at least one of a spatial candidate, a temporal candidate, or a constructed candidate. Here, the spatial candidate may be derived based on a vector of a neighboring block that is spatially adjacent to the current block, and the temporal candidate may be derived based on a vector of a neighboring block that is temporally adjacent to the current block. Here, the neighboring block may refer to a block encoded based on an affine model. The vector may refer to a motion vector or a control point vector.

[0224] Reference Figure 8 , a method for deriving spatial / temporal candidates based on vectors of spatial / temporal neighboring blocks is described in detail.

[0225] On the other hand, the constructed candidate can be derived based on the combination of motion vectors of the spatial / temporal neighboring blocks of the current block, referring to Fig. 9 Provide detailed description of this situation.

[0226] The multiple affine candidates may be arranged in the candidate list based on a specific priority order. For example, multiple affine candidates may be arranged in the candidate list in the order of spatial candidates, temporal candidates, and constructed candidates. Alternatively, multiple affine candidates may also be arranged in the candidate list in the order of temporal candidates, spatial candidates, and constructed candidates. However, this is not limited to this, and the temporal candidates may also be arranged after the constructed candidates. Alternatively, it may also be arranged in the following manner: a portion of the constructed candidates is arranged before the spatial candidates, and the remaining portion is arranged after the spatial candidates.

[0227] The candidate list may also include temporal candidates based on sub-blocks. The motion information of the temporal candidates may be derived based on a collocated block corresponding to the current block, referring to Fig.10 Provide detailed description of this situation.

[0228] A control point vector of the current block may be derived based on the candidate list and the candidate index ( S710 ).

[0229] The candidate index may refer to an index encoded for deriving a control point vector of the current block. The candidate index may specify an affine candidate among a plurality of affine candidates in the candidate list. The control point vector of the current block may be derived using the control point vector of the affine candidate specified by the candidate index.

[0230] For example, it is assumed that the type of the affine model of the current block is a four-parameter affine model (i.e., it is determined that two control point vectors are used for the current block). At this time, in the case where the affine candidate specified by the candidate index has three control point vectors, only two control point vectors of the three control point vectors (e.g., control point vectors with Idx=0, 1) may be selected, and the two control point vectors may be set as the control point vectors of the current block. Alternatively, the three control point vectors of the specified affine candidate may be set as the control point vectors of the current block. In this case, the type of the affine model of the current block may be updated to a six-parameter affine model.

[0231] On the contrary, it is assumed that the type of the affine model of the current block is a six-parameter affine model (i.e., it is determined that three control point vectors are used for the current block). At this time, in the case where the affine candidate specified by the candidate index has two control point vectors, an additional control point vector may be generated, and the two control point vectors of the affine candidate and the additional control point vector may be set as the control point vector of the current block. The additional control point vector may be derived based on at least one of the two control point vectors of the affine candidate, the size or position information of the current / neighboring block.

[0232] Alternatively, the two control point vectors of the identified affine candidates may be set as the control point vectors of the current block. In this case, the type of the affine model of the current block may be updated to a four-parameter affine model.

[0233] A motion vector of the current block may be derived based on the control point vector of the current block (S720).

[0234] The motion vector can be derived in units of sub-blocks of the current block. To this end, the current block can be divided into a plurality of N×M sub-blocks. Here, the N×M sub-blocks can be in the form of rectangles (N>M or N<M) or squares (N=M). The values ​​of N and M can be 4, 8, 16, 32 or more.

[0235] For example, the size / shape of the sub-block may be a fixed size / shape defined in the decoding device. For example, the size / shape of the sub-block may be a square such as 4×4, 8×8, 16×16, or a rectangle such as 2×4, 2×8, 4×8, 4×16, etc. Alternatively, the sub-block may be defined as a block whose sum of width and height is 8, 12, 16 or an integer greater than or equal to 16, 32, 64 or an integer greater than or equal to 16.

[0236] Alternatively, the size / shape of the sub-block may be variably derived based on the properties of the block. For example, when the size of the current block is greater than or equal to a specific threshold size, the current block may be segmented in units of a first sub-block (e.g., 8×8, 16×16), otherwise, the current block may be segmented in units of a second sub-block (e.g., 4×4).

[0237] Alternatively, information about the sub-block size / shape may be encoded and transmitted in the encoding device. The information indicates at least one of the sub-block size or shape, and may be transmitted at at least one level of a sequence, a picture, a block group, a block, or a CTU.

[0238] The control point representing the current block may include the upper left position and the upper right position. However, this is not limited thereto, and the control point may also include the upper left position, the upper right position, and the lower left position, or include other multiple positions.

[0239] When two control points are used, the motion vectors of each sub-block of the current block can be derived using at least one of a first control point vector corresponding to the first control point, a second control point vector corresponding to the second control point, the position (x, y) of the sub-block, or the size (width or height) of the current block.

[0240] In the case of using three control points, the motion vectors of each sub-block of the current block can be derived using at least one of a first control point vector corresponding to the first control point, a second control point vector corresponding to the second control point, a third control point vector corresponding to the third control point, the position (x, y) of the sub-block, or the size (width or height) of the current block. At this time, the differential vector between the second control point vector and the first control point vector can be used, and the differential vector between the third control point vector and the first control point vector can be used. The differential vectors can be calculated for the horizontal direction (x-axis direction) and the vertical direction (y-axis direction), respectively.

[0241] The motion vector derivation process may also include a process of performing a specific offset on the motion vector derived based on the control point vector. The offset may refer to a vector used to improve the derived motion vector. The offset may be determined based on information of at least one of the size or direction of the offset. The size (Absolute) may be an integer of 1, 2, 3 or more. The direction may include at least one of the left side, the right side, the top end or the bottom end. The size and / or direction information of the offset may be encoded and transmitted in the encoding device. Alternatively, the size of the offset may also be a fixed value preset in the decoding device.

[0242] Inter prediction may be performed on the current block using the derived motion vector (S730).

[0243] Specifically, the reference block can be identified using the motion vector of the current block. The reference block can be identified for each sub-block of the current block. The reference block of each sub-block can belong to one reference image. That is, the sub-blocks belonging to the current block can share one reference image. Alternatively, the reference image index can be set separately for each sub-block of the current block.

[0244] On the other hand, the derived motion vector may include at least one of an L0 motion vector and an L1 motion vector. When the motion vector includes an L0 motion vector and an L1 motion vector, the decoding device may also set any one of the L0 motion vector and the L1 motion vector to 0 to perform unidirectional prediction.

[0245] The setting may be selectively performed considering at least one of the size of the block or the prediction mode. Here, the block may refer to a current block or a subblock of the current block. The prediction mode may refer to a skip mode, a merge mode, an AMVP (Advanced Motion Vector Prediction) mode, a prediction mode based on an affine model, a current image reference mode, etc.

[0246] For example, when the size of the current block (or a sub-block of the current block) is less than or equal to a specific threshold size, the L1 motion vector may be set to 0 to perform unidirectional prediction, otherwise, the L0 motion vector and the L1 motion vector may be used to perform bidirectional prediction. Here, the threshold size may be defined as a block size of at least 4, 8, or 16 in width and height.

[0247] Alternatively, when the current block has been encoded by the first prediction mode, the L1 motion vector may be set to 0 to perform unidirectional prediction; otherwise, the L0 motion vector and the L1 motion vector may be used to perform bidirectional prediction.

[0248] The specified reference block may be set as a prediction block of the current block. A residual block may be added to the prediction block to reconstruct the current block.

[0249] Figure 8 As an embodiment to which the present invention is applied, a diagram is shown showing a method of deriving affine candidates from control point vectors of spatial / temporal neighboring blocks.

[0250] The width and height of the current block 800 are cbW and cbH respectively, and the position of the current block is (xCb, yCb). The width and height of the spatially adjacent blocks 810 to 850 are nbW and nbH respectively, and the position of the spatially adjacent blocks is (xNb, yNb). The spatially adjacent blocks may include at least one of the left block 810, the lower left block 840, the upper right block 830, the upper block 820, or the upper left block 850 of the current block. Alternatively, the spatially adjacent blocks may also include at least one of the blocks adjacent to the right side of the upper left block 850 or the blocks adjacent to the lower end.

[0251] The spatial candidate may have n control point vectors (cpMVs). Here, the value of n may be an integer of 1, 2, 3 or more. The value of n may be determined based on at least one of information about whether it can be decoded in units of subblocks, information about whether it is a block encoded based on an affine model, or information about the type of the affine model (four parameters or six parameters).

[0252] For example, according to the information, if the corresponding block can be decoded in units of sub-blocks or the corresponding block is a block encoded based on an affine model, the block will have two control point vectors. Otherwise, the prediction based on the affine model may not be performed on the block.

[0253] Alternatively, according to the information, when the corresponding block is a block encoded based on an affine model and the type of the affine model is a six-parameter affine model, the block may have three control point vectors. Otherwise, the block may not perform prediction based on the affine model.

[0254] The information may also be encoded and transmitted in the encoding device. Alternatively, all or part of the information may be derived from the decoding device based on the properties of the block. Here, the block may refer to the current block or the spatial / temporal neighboring block of the current block. The properties may refer to size, shape, position, segmentation type, inter-frame mode, parameters of residual coefficients, etc. The inter-frame mode, as a mode defined in the decoding device, may refer to a merge mode, a skip mode, an AMVP mode, an affine model, an intra-frame / inter-frame combination mode, a current image reference mode, etc. Alternatively, the value of n may be derived from the decoding device based on the properties of the block.

[0255] In this embodiment, the n control point vectors can be represented as a first control point vector (cpMV[0]), a second control point vector (cpMV[1]), a third control point vector (cpMV[2]), ..., an nth control point vector (cpMV[n-1]).

[0256] As an example, the first control point vector (cpMV[0]), the second control point vector (cpMV[1]), the third control point vector (cpMV[2]), and the fourth control point vector (cpMV[3]) may be vectors corresponding to the positions of the upper left sample, the upper right sample, the lower left sample, and the lower right sample of the block, respectively. Here, it is assumed that the spatial candidate has three control point vectors, and the three control point vectors may be control point vectors arbitrarily selected from the first control point vector to the nth control point vector. However, this is not limited to this, and the spatial candidate may have two control point vectors, and the two control point vectors may be control point vectors arbitrarily selected from the first control point vector to the nth control point vector.

[0257] The following describes a method for deriving control point vectors of spatial candidates.

[0258] 1. The boundary of the current block does not connect to the CTU boundary

[0259] The first control point vector may be derived based on at least one of the first control point vector of the spatial neighboring block, a specific differential value, position information (xCb, yCb) of the current block, or position information (xNb, yNb) of the spatial neighboring block.

[0260] The number of the differential values ​​may be 1, 2, 3 or more. The number of the differential values ​​may be variably determined in consideration of the properties of the block, or may be a fixed value preset in the decoding device. The differential value may be defined as a differential value between any one of the plurality of control point vectors and another control point vector. For example, the differential value may include at least one of a first differential value between the second control point vector and the first control point vector, a second differential value between the third control point vector and the first control point vector, a third differential value between the fourth control point vector and the third control point vector, or a fourth differential value between the fourth control point vector and the second control point vector.

[0261] For example, the first control point vector can be derived by the following formula 1.

[0262] [Formula 1]

[0263] cpMvLX[0][0]=(mvScaleHor+dHorX*(xCb-xNb)+dHorY*(yCb-yNb))

[0264] cpMvLX[0][1]=(mvScaleVer+dVerX*(xCb-xNb)+dVerY*(yCb-yNb))

[0265] In Formula 1, the variables mvScaleHor and mvScaleVer may refer to the first control point vector of the spatially adjacent block, or may refer to a value derived by performing a shift operation of about k on the first control point vector. Here, k may be an integer of 1, 2, 3, 4, 5, 6, 7, 8, 9 or more. The variables dHorX and dVerX correspond to the x component and y component of the first differential value between the second control point vector and the first control point vector, respectively. The variables dHorY and dVerY correspond to the x component and y component of the second differential value between the third control point vector and the first control point vector, respectively. The variables may be derived by the following Formula 2.

[0266] [Formula 2]

[0267] mvScaleHor=CpMvLX[xNb][yNb][0][0]<<7

[0268] mvScaleVer=CpMvLX[xNb][yNb][0][1]<<7

[0269] dHorX=(CpMvLX[xNb+nNbW-1][yNb][1][0]-CpMvLX[xNb][yNb][0][0])<<(7-log2NbW)

[0270] dVerX=(CpMvLX[xNb+nNbW-1][yNb][1][1]-CpMvLX[xNb][yNb][0][1])<<(7-log2NbW)

[0271] dHorY=(CpMvLX[xNb][yNb+nNbH-1][2][0]-CpMvLX[xNb][yNb][2][0])<<(7-log2NbH)

[0272] dVerY=(CpMvLX[xNb][yNb+nNbH-1][2][1]-CpMvLX[xNb][yNb][2][1])<<(7-log2NbH)

[0273] The second control point vector may be derived based on at least one of the first control point vector of the spatially adjacent block, a specific differential value, position information (xCb, yCb) of the current block, a block size (width or height), or position information (xNb, yNb) of the spatially adjacent block. Here, the block size may refer to the size of the current block and / or the spatially adjacent block. The differential value is the same as that described in the first control point vector, and a detailed description is omitted here. However, the range and / or number of differential values ​​used in the process of deriving the second control point vector may be different from that of the first control point vector.

[0274] For example, the second control point vector can be derived by the following formula 3.

[0275] [Formula 3]

[0276] cpMvLX[1][0]=(mvScaleHor+dHorX*(xCb+cbWidth-xNb)+dHorY*(yCb-yNb))

[0277] cpMvLX[1][1]=(mvScaleVer+dVerX*(xCb+cbWidth-xNb)+dVerY*(yCb-yNb))

[0278] In Formula 3, variables mvScaleHor, mvScaleVer, dHorX, dVerX, dHorY, and dVerY are the same as those in Formula 1, and detailed description thereof is omitted here.

[0279] The third control point vector may be derived based on at least one of the first control point vector of the spatially adjacent block, a specific differential value, position information (xCb, yCb) of the current block, a block size (width or height), or position information (xNb, yNb) of the spatially adjacent block. Here, the block size may refer to the size of the current block and / or the spatially adjacent block. The differential value is the same as that described in the first control point vector, and a detailed description is omitted here. However, the range and / or number of differential values ​​used in the process of deriving the third control point vector may be different from that of the first control point vector or the second control point vector.

[0280] For example, the third control point vector can be derived by the following formula 4.

[0281] [Formula 4]

[0282] cpMvLX[2][0]=(mvScaleHor+dHorX*(xCb-xNb)+dHorY*(yCb+cbHeight-yNb))

[0283] cpMvLX[2][1]=(mvScaleVer+dVerX*(xCb-xNb)+dVerY*(yCb+cbHeight-yNb))

[0284] In Formula 4, the variables mvScaleHor, mvScaleVer, dHorX, dVerX, dHorY, and dVerY are the same as those in Formula 1, and detailed description thereof is omitted here. On the other hand, the n-th control point vector of the spatial candidate can be derived through the above process.

[0285] 2. The boundary of the current block is connected to the boundary of the CTU

[0286] The first control point vector may be derived based on at least one of a motion vector (MV) of a spatial neighboring block, a specific differential value, position information (xCb, yCb) of a current block, or position information (xNb, yNb) of a spatial neighboring block.

[0287] The motion vector may be a motion vector of a subblock located at the bottom of a spatially adjacent block. The subblock may be a subblock located at the leftmost, center, or rightmost of a plurality of subblocks located at the bottom of a spatially adjacent block. Alternatively, the motion vector may also refer to an average value, a maximum value, or a minimum value of motion vectors of subblocks.

[0288] The number of the differential values ​​may be 1, 2, 3 or more. The number of the differential values ​​may be variably determined in consideration of the properties of the block, or may be a fixed value preset in the decoding device. The differential value may be defined as a differential value between any one of a plurality of motion vectors stored in units of sub-blocks in a spatially adjacent block and another motion vector. For example, the differential value may refer to a differential value between a motion vector of a lower right terminal block and a motion vector of a lower left terminal block of a spatially adjacent block.

[0289] For example, the first control point vector can be derived by the following formula 5.

[0290] [Formula 5]

[0291] cpMvLX[0][0]=(mvScaleHor+dHorX*(xCb-xNb)+dHorY*(yCb-yNb))

[0292] cpMvLX[0][1]=(mvScaleVer+dVerX*(xCb-xNb)+dVerY*(yCb-yNb))

[0293] In Formula 5, the variables mvScaleHor and mvScaleVer may refer to the motion vector (MV) of the spatial neighboring block or a value derived by performing a shift operation of about k on the motion vector. Here, k may be 1, 2, 3, 4, 5, 6, 7, 8, 9 or an integer greater than or equal to 9.

[0294] The variables dHorX and dVerX correspond to the x component and y component of a specific differential value, respectively. Here, the differential value refers to the differential value between the motion vector of the lower right terminal block and the motion vector of the lower left terminal block in the spatially adjacent blocks. The variables dHorY and dVerY can be derived based on the variables dHorX and dVerX. The variables can be derived by the following formula 6.

[0295] [Formula 6]

[0296] mvScaleHor=MvLX[xNb][yNb+nNbH-1][0]<<7

[0297] mvScaleVer=MvLX[xNb][yNb+nNbH-1][1]<<7

[0298] dHorX=(MvLX[xNb+nNbW-1][yNb+nNbH-1][0]-MvLX[xNb][yNb+nNbH-1][0])<<(7-log2NbW)

[0299] dVerX=(MvLX[xNb+nNbW-1][yNb+nNbH-1][1]-MvLX[xNb][yNb+nNbH-1][1])<<(7-log2NbW)

[0300] dHorY=-dVerX

[0301] dVerY=dHorX

[0302] The second control point vector may be derived based on at least one of a motion vector (MV) of a spatially neighboring block, a specific differential value, position information (xCb, yCb) of a current block, a block size (width or height), or position information (xNb, yNb) of a spatially neighboring block. Here, the block size may refer to the size of the current block and / or the spatially neighboring block. The motion vector and the differential value are the same as those described in the first control point vector, and a detailed description is omitted here. However, the position of the motion vector, the range of the differential value, and / or the number of the motion vectors used in the process of deriving the second control point vector may be different from those of the first control point vector.

[0303] For example, the second control point vector can be derived by the following formula 7.

[0304] [Formula 7]

[0305] cpMvLX[1][0]=(mvScaleHor+dHorX*(xCb+cbWidth-xNb)+dHorY*(yCb-yNb))

[0306] cpMvLX[1][1]=(mvScaleVer+dVerX*(xCb+cbWidth-xNb)+dVerY*(yCb-yNb))

[0307] In Formula 7, variables mvScaleHor, mvScaleVer, dHorX, dVerX, dHorY, and dVerY are the same as those in Formula 5, and detailed description thereof is omitted here.

[0308] The third control point vector may be derived based on at least one of the motion vector (MV) of the spatial neighboring block, a specific differential value, the position information (xCb, yCb) of the current block, the block size (width or height), or the position information (xNb, yNb) of the spatial neighboring block. Here, the block size may refer to the size of the current block and / or the spatial neighboring block. The motion vector and the differential value are the same as those described in the first control point vector, and detailed description is omitted here. However, the position of the motion vector used in the process of deriving the third control point vector, the range of the differential value and / or the number may be different from the first control point vector or the second control point vector.

[0309] For example, the third control point vector can be derived by the following formula 8.

[0310] [Formula 8]

[0311] cpMvLX[2][0]=(mvScaleHor+dHorX*(xCb-xNb)+dHorY*(yCb+cbHeight-yNb))

[0312] cpMvLX[2][1]=(mvScaleVer+dVerX*(xCb-xNb)+dVerY*(yCb+cbHeight-yNb))

[0313] In Formula 8, the variables mvScaleHor, mvScaleVer, dHorX, dVerX, dHorY, and dVerY are the same as those in Formula 5, and their detailed description is omitted here. On the other hand, the n-th control point vector of the spatial candidate can be derived through the above process.

[0314] The process of deriving the affine candidate may be performed on each defined spatial neighboring block. The defined spatial neighboring block may include at least one of a left block, a lower left block, an upper right block, an upper block, or an upper left block of the current block.

[0315] Alternatively, the process of deriving the affine candidate may be performed according to each group of the spatially adjacent blocks. Here, the spatially adjacent blocks may be divided into a first group including a left block and a lower left block, and a second group including an upper right block, an upper block, and an upper left block.

[0316] For example, an affine candidate may be derived from the spatially adjacent blocks belonging to the first group. The derivation may be performed based on a specific preferred order until an available affine candidate is found. The preferred order may be the order of left block → lower left block, or the reverse order. Whether the corresponding spatially adjacent blocks in the first group are blocks decoded by prediction based on an affine model may be determined based on the preferred order, and the block first decoded by prediction based on an affine model may be selected as the affine candidate.

[0317] Similarly, an affine candidate may be derived from the spatially adjacent blocks belonging to the second group. The derivation may be performed based on a specific preferred order until an available affine candidate is found. The preferred order may be the order of the upper right block→the upper block→the upper left block, or the reverse order. Whether the corresponding spatially adjacent blocks in the second group are blocks decoded by prediction based on the affine model may be determined based on the preferred order, and the block first decoded by prediction based on the affine model may be selected as the affine candidate.

[0318] The above embodiment can also be applied to temporal neighboring blocks in the same / similar manner. Here, the temporal neighboring block belongs to a different image from the current block, but can be a block at the same position as the current block. The so-called block at the same position can be a block containing the position of the upper left sample of the current block, the central position, or the position of the sample adjacent to the lower right sample of the current block.

[0319] Alternatively, the temporal neighboring block may refer to a block at a position displaced from the block at the same position by a specific displacement vector. Here, the displacement vector may be determined based on a motion vector of any spatial neighboring block among the spatial neighboring blocks of the current block.

[0320] Fig. 9 As one embodiment to which the present invention is applied, this is a diagram showing a method of deriving a constructed candidate based on a combination of motion vectors of spatial / temporal neighboring blocks.

[0321] The constructed candidate in the present invention can be derived based on a combination of at least two control point vectors corresponding to the corners of the current block (hereinafter referred to as control point vector (cpMVCorner[n])). Here, n can be 0, 1, 2, or 3.

[0322] The control point vector (cpMVCorner[n]) may be derived based on the motion vectors of the spatial neighboring blocks and / or the temporal neighboring blocks. Here, the spatial neighboring blocks may include at least one of a first neighboring block (A, B, or C) adjacent to the upper left sample of the current block, a second neighboring block (D or E) adjacent to the upper right sample of the current block, or a third neighboring block (F or G) adjacent to the lower left sample of the current block. The temporal neighboring block, as a block belonging to a different image from the current block, may refer to a block at the same position as the current block (hereinafter referred to as the fourth neighboring block (Col)). Here, the fourth neighboring block may refer to a block (H, I, or J) containing the position of the upper left sample, the upper right sample, or the lower left sample of the current block, or a block adjacent to the position of the lower right sample of the current block.

[0323] The first neighboring block may refer to a neighboring block at the upper left end (A), the upper end (B), or the left side (C) of the current block. Whether the motion vectors of the neighboring blocks A, B, and C are available may be determined according to a specific preferred order, and the control point vectors may be determined using the motion vectors of the available neighboring blocks. The availability determination may be performed until a neighboring block with an available motion vector is found. Here, the preferred order may be the order of A→B→C. However, it is not limited thereto, and may also be the order of A→C→B, C→A→B, B→A→C.

[0324] The second neighboring block may refer to the neighboring block at the upper end (D) or the upper right end (E) of the current block. Similarly, whether the motion vectors of the neighboring blocks D and E are available may be determined according to a specific preferred order, and the control point vectors may be determined using the motion vectors of the available neighboring blocks. The availability determination may be performed until a neighboring block with an available motion vector is found. Here, the preferred order may be the order of D→E or the order of E→D.

[0325] The third neighboring block may refer to a neighboring block on the left side (F) or the lower left end (G) of the current block. Similarly, whether the motion vector of the neighboring block is available may be determined according to a specific preferred order, and the control point vector may be determined using the motion vector of the available neighboring block. The availability determination may be performed until a neighboring block with an available motion vector is found. Here, the preferred order may be the order of G→F or the order of F→G.

[0326] For example, the first control point vector (cpMVCorner[0]) may be set according to the motion vector of the first neighboring block, the second control point vector (cpMVCorner[1]) may be set according to the motion vector of the second neighboring block, the third control point vector (cpMVCorner[2]) may be set according to the motion vector of the third neighboring block, and the fourth control point vector (cpMVCorner[3]) may be set according to the motion vector of the fourth neighboring block.

[0327] Alternatively, the first control point vector may be derived using a motion vector of at least one of the first neighboring block or the fourth neighboring block, where the fourth neighboring block may be a block containing the position of the upper left sample (H). The second control point vector may be derived using a motion vector of at least one of the second neighboring block or the fourth neighboring block. Here, the fourth neighboring block may be a block containing the position of the upper right sample (I). The third control point vector may be derived using a motion vector of at least one of the third neighboring block or the fourth neighboring block. Here, the fourth neighboring block may be a block containing the position of the lower left sample (J).

[0328] Alternatively, any of the first to fourth control point vectors may be derived based on another control point vector. For example, a specific offset vector may be applied to the first control point vector to derive the second control point vector. The offset vector may be a differential vector between the third control point vector and the first control point vector, or the offset vector may be derived by applying a specific scale vector to the differential vector. The scale vector may be determined based on at least one of the width or height of the current block and / or the adjacent block.

[0329] The K constituted candidates (ConstK) of the present invention can be determined by a combination of at least two control point vectors from the first control point vector to the fourth control point vector. The K value can be 1, 2, 3, 4, 5, 6, 7 or an integer greater than or equal to 7. The K value can be derived based on information transmitted from the encoding device, or it can be a preset value in the decoding device. The information can include the following information, that is, information indicating the maximum number of constituted candidates that can be included in the candidate list.

[0330] Specifically, the first constructed candidate (Const1) may be derived by combining the first control point vector to the third control point vector. For example, the first constructed candidate (Const1) may have a control point vector as shown in Table 4 below. On the other hand, in the case where the reference image information of the first neighboring block is the same as the reference image information of the second neighboring block and the third neighboring block, the control point vector may be restrictively constructed as in Table 4. Here, the reference image information may refer to a reference image index indicating the position of the reference image in the reference image list, or may refer to a POC (picture order count) value indicating the output order.

[0331] [Table 4]

[0332] Idx Control point vector 0 cpMvCorner[0] 1 cpMvCorner[1] 2 cpMvCorner[2]

[0333] The second constructed candidate (Const2) may be derived by combining the first control point vector, the second control point vector, and the fourth control point vector. For example, the second constructed candidate (Const2) may have a control point vector as shown in Table 5 below. On the other hand, in the case where the reference image information of the first neighboring block is the same as the reference image information of the second neighboring block and the fourth neighboring block, the control point vector may be restrictively constructed as shown in Table 5. Here, the reference image information is as described above.

[0334] [Table 5]

[0335]

[0336] The third constructed candidate (Const3) may be derived by combining the first control point vector, the third control point vector, and the fourth control point vector. For example, the third constructed candidate (Const3) may have a control point vector as shown in Table 6 below. On the other hand, in the case where the reference image information of the first neighboring block is the same as the reference image information of the third neighboring block and the fourth neighboring block, the control point vector may be restrictively constructed as shown in Table 6. Here, the reference image information is as described above.

[0337] [Table 6]

[0338]

[0339] The fourth constructed candidate (Const4) may be derived by combining the second control point vector, the third control point vector, and the fourth control point vector. For example, the fourth constructed candidate (Const4) may have a control point vector as shown in Table 7 below. On the other hand, in the case where the reference image information of the second neighboring block is the same as the reference image information of the third neighboring block and the fourth neighboring block, it may also be restrictively constructed as in Table 7. Here, the reference image information is as described above.

[0340] [Table 7]

[0341]

[0342] The fifth constructed candidate (Const5) may be derived by combining the first control point vector and the second control point vector. For example, the fifth constructed candidate (Const5) may have a control point vector as shown in Table 8 below. On the other hand, in the case where the reference image information of the first neighboring block is the same as the reference image information of the second neighboring block, the control point vector may be restrictively constructed as shown in Table 8. Here, the reference image information is as described above.

[0343] [Table 8]

[0344] Idx Control point vector 1 cpMvCorner[0] 2 cpMvCorner[1]

[0345] The sixth constructed candidate (Const6) may be derived by combining the first control point vector and the third control point vector. For example, the sixth constructed candidate (Const6) may have a control point vector as shown in Table 9 below. On the other hand, in the case where the reference image information of the first neighboring block is the same as the reference image information of the third neighboring block, the control point vector may be constructed as shown in Table 9 with limitation. Here, the reference image information is as described above.

[0346] [Table 9]

[0347] Idx Control point vector Control point vector 1 cpMvCorner[0] cpMvCorner[0] 2 cpMvCorner[2] cpMvCorner[1]

[0348] In Table 9, cpMvCorner[1] may be a second control point vector derived based on the first control point vector and the third control point vector. The second control point vector may be derived based on at least one of the first control point vector, a specific differential value, or a size of a current / neighboring block. For example, the second control point vector may be derived by the following formula 9.

[0349] [Formula 9]

[0350] cpMvCorner[1][0]=(cpMvCorner[0][0]<<7)+((cpMvCorner[2][1]-cpMvCorner[0][1])<<(7+Log2(cbHeight / cbWidth)))

[0351] cpMvCorner[1][1]=(cpMvCorner[0][1]<<7)+((cpMvCorner[2][0]-cpMvCorner[0][0])<<(7+Log2(cbHeight / cbWidth)))

[0352] Among the above-mentioned K constituted candidates (ConstK), the value of K does not limit the position or priority order of the constituted candidates arranged in the candidate list.

[0353] In addition, all of the first to sixth constituted candidates may be included in the candidate list, or only a part of them may be included in the candidate list.

[0354] For example, when it is determined that three control point vectors are used for the current block, only the candidate generated by combining three control point vectors from the first control point vector to the fourth control point vector may be used. When it is determined that two control point vectors are used for the current block, the candidate generated by combining at least two control point vectors from the first control point vector to the fourth control point vector or the candidate generated by combining only two control point vectors from the first control point vector to the fourth control point vector may be used.

[0355] Alternatively, the maximum number of affine candidates that can be included in the candidate list may be considered and only a portion of the constituted candidates may be included in the candidate list. Here, the maximum number may be determined based on the maximum number information transmitted by the encoding device, or may be variably determined considering the attributes of the current block. In this case, the K value of the constituted candidate (ConstK) may also refer to the priority order of insertion into the candidate list.

[0356] On the other hand, in the case where the difference between two control point vectors belonging to the constructed candidate is less than a specific threshold, the constructed candidate may not be inserted into the candidate list. The difference between the two control point vectors can be divided into a horizontal difference and a vertical difference. Here, the horizontal difference may refer to the difference between the first control point vector 910 and the second control point vector 920, and the vertical difference may refer to the difference between the first control point vector 910 and the third control point vector 930. The threshold may refer to 0 or a vector of a size preset in the encoding / decoding device.

[0357] Fig.10 As one embodiment to which the present invention is applied, this is a diagram showing a method of deriving motion information based on a temporal candidate of a sub-block.

[0358] The motion information of the temporal candidate of the present invention may be derived according to the motion information of the collocated block. The motion information may be derived in units of sub-blocks.

[0359] Here, the collocated block is a block belonging to an image different from the current block 1010 (i.e., a collocated image), and may also be a block 1040 at the same position as the current block, or a block 1030 at a position displaced from the position of the current block by a temporal vector. The temporal vector may be determined based on a motion vector of a spatially adjacent block of the current block. A spatially adjacent block may refer to a block adjacent to at least one of the left side, the lower left end, the upper end, the upper right end, or the upper left end of the current block. The temporal vector may be determined using only adjacent blocks at a position preset in the encoding / decoding device. For example, the preset position may be the left side 1011 or the upper end 1012, or may be the left side 1011 and the upper end 1012. In the case where there are multiple adjacent blocks on the left side, a block located at any position of the lowermost end, the uppermost end, or the center among the multiple adjacent blocks may be used. In the case where there are multiple adjacent blocks at the upper end, a block located at any position of the leftmost end, the rightmost end, or the center among the multiple adjacent blocks may be used.

[0360] When deriving the sub-block based temporal candidates, the current block and / or the collocated block may be partitioned into a plurality of sub-blocks.

[0361] Here, the sub-block may have a fixed size / shape preset in the encoding / decoding device. For example, the sub-block may be represented as an N×M block, where the values ​​of N and M are 4, 8, or integers greater than or equal to 4. The sub-block may be a square (N=M) or a rectangle (N>M, N<M). It may refer to size. Alternatively, the encoding device may encode and transmit information about the size / shape of the sub-block, and the decoding device may determine the size / shape of the sub-block based on the transmitted information.

[0362] Alternatively, the current block and / or the collocated block may be divided into a plurality of sub-blocks based on a specific number. Here, the number may be a fixed number preset in the encoding / decoding device, or the number may be variably determined in consideration of the block size / shape. Alternatively, the number may be determined based on number information transmitted by the encoding device.

[0363] The following describes a method for deriving motion information of a temporal candidate in units of sub-blocks. The motion vector of the temporal candidate may be set based on the motion vector stored in each sub-block of the collocated block. The reference image index of the temporal candidate may be set to a value defined in the encoding / decoding device (e.g., 0). Alternatively, the reference image index of the temporal candidate may be set based on the reference image index of the collocated block or the reference image index of the upper left terminal block in the sub-block. Alternatively, the reference image index of the temporal candidate may be set based on the reference image index stored in each sub-block in the same manner as the motion vector.

[0364] However, in the case where an unavailable sub-block exists in the sub-blocks belonging to the collocated block, the motion vector of the unavailable sub-block may be replaced by the motion vector of the available sub-block in the collocated block. Here, the available sub-block may refer to a block adjacent to any position of the left side, right side, top end or bottom end of the unavailable sub-block. Alternatively, the available sub-block may be a block at a preset position in the encoding / decoding device. For example, the preset position may be the position of the lower right terminal block in the collocated block, or the position of the sub-block containing the center position of the collocated block. In the case where the available sub-block exists, the sub-block-based temporal candidate may be appended to the candidate list. Alternatively, in the case where the sub-block at the preset position is available, the sub-block-based temporal candidate may be appended to the candidate list.

[0365] Fig.11 As an embodiment of the present invention, it is a diagram showing an inter-frame prediction method in an ERP projection format.

[0366] Reference Fig.11 In the ERP projection format, there may be a situation where the reference area specified by the motion vector overlaps with the boundary of the reference image.

[0367] In this case, the boundary of the corresponding reference image may be expanded, and the data adjacent to the boundary may be filled into the expanded area in the horizontal direction (see A). Alternatively, the boundary of the corresponding reference image may be expanded, and the data adjacent to the boundary on the opposite side (data having continuity or association with the boundary) may be filled into the expanded area (see B). Alternatively, the data of the non-existent area may be filled with the data adjacent to the boundary on the opposite side (see C).

[0368] Although A and B need to expand the data of the reference image, they can simplify the calculation process. Although C does not require additional memory due to the expansion, it may require additional calculation process.

[0369] In the cases A and B, the size of the current image is different from the size of the (enlarged) reference image. The process of obtaining the predicted pixels may include a specific adjustment process to take this size difference into account.

[0370] When only the horizontal length is compared, the current image may have a range of 0 to (pic_width-1), and the reference image (enlarged like the right side) may have a range of -L_offset to (pic_width-1+R_offset) or a range of 0 to (pic_width-1+L_offset+R_offset). In the former case, the predicted pixel can be directly obtained in the enlarged reference image according to the position obtained by adding the coordinates of the current block to the motion vector. In the latter case, the predicted pixel can be obtained at the readjusted position after adjusting the obtained position. Here, L_offest and R_offset may refer to the lengths enlarged in the left and right directions. In the present invention, it is assumed that the coordinates are calculated based on the range of the current image or the existing reference image as in the past.

[0371] In the case of C, the position in the reference image can be calculated by adding the coordinates of the current block to the motion vector. It can be confirmed whether the calculated pixel position is out of the range of the reference image (<0, 0>~<pic_width-1, pic_height-1>). In the case where the calculated pixel position does not deviate from the range of the reference image, the predicted pixel can be obtained at the calculated pixel position. On the contrary, in the case where the calculated pixel position is out of the range of the reference image, the position corresponding to the calculated position can be calculated at the opposite side position of the inside of the corresponding boundary <inside the image> (right side when on the left side, left side when on the right side), and the predicted pixel can be obtained at the calculated corresponding position.

[0372] Inter-frame prediction on the reference image boundary may be performed based on at least one of the above embodiments. The inter-frame prediction method on the reference image boundary may be a method preset in the encoding / decoding device, or may be determined based on selection information that specifies one of the above embodiments and information related to the selected method (data processing method, enlargement size information, etc.). The same applies to the embodiments described below.

[0373] Fig.12 and Fig.15 As an embodiment of the present invention, it is a diagram showing an inter-frame prediction method in the CMP projection format.

[0374] Reference Fig.12 In the CMP projection format, there is 3D spatial continuity between the upper and lower 3×1 surfaces due to region-wise packing. However, there may not be 3D spatial continuity between the upper 3×1 surface and the lower 3×1 surface.

[0375] Similarly, the reference area specified by the motion vector may overlap with the boundary of the reference image or overlap with a part of the surface boundary. In case A, it is inside the reference image but located at a part of the surface boundary (there is no mutual continuity in three-dimensional space, so the correlation is low), and in case B, the reference area is located at the boundary of the reference image.

[0376] In particular, in the case of A, although it is represented as being inside the reference image, the accuracy of inter-frame prediction is reduced because the reference region includes a discontinuous surface boundary.

[0377] In the case of A and B, the position of the reference area in the reference image can be calculated by adding the coordinates of the current block to the motion vector. It can be confirmed whether at least a part of the corresponding reference area is out of the range of the reference image or whether it includes a discontinuous boundary in the reference image.

[0378] When the reference area is out of the range of the reference image, a position having continuity or association in the three-dimensional space may be determined, and a predicted pixel may be obtained based on the determined position.

[0379] On the contrary, if the reference area does not deviate from the range of the reference image, it can be confirmed whether at least one pixel in the reference area indicated by the motion vector of the current block belongs to more than two surfaces. Here, the confirmation process can be performed according to the pixel positions of the upper left, upper right, lower left, lower right, etc. of the current block.

[0380] In the case where at least one pixel in the reference area does not belong to more than two surfaces, the predicted pixel can be obtained based on the reference area. Otherwise, it can be confirmed whether it is a surface that has continuity with the surface to which the current block belongs. In the case of the reference area A', the upper end area has continuity with the surface to which the current block belongs, and the lower end area does not have continuity with the surface to which the current block belongs.

[0381] In the case of the continuity, the predicted pixel can be obtained according to the reference region. Otherwise, the predicted pixel can be obtained according to the region having continuity or association in the three-dimensional space.

[0382] The above inter-frame prediction order is just an example, and various modifications can be achieved by changing the order, changing a part of the structure, etc.

[0383] In the case of C and D, the reference image can be expanded, and the expanded area can be filled with data adjacent to the boundary in the horizontal / vertical direction, or filled with data of the area with correlation. The case of D is the same as that described in ERP, so the detailed description is omitted here. In the case of C, the reference area is located inside the reference image as in A, but includes a surface boundary with low correlation. Therefore, if the expanded area is not generated inside the reference image, the accuracy of the inter-frame prediction will be reduced, and the reference image will be more accurate. Fig.13 This is explained in detail.

[0384] In the case of C and D, the size of the current image may be different from the size of the reference image due to the expansion of the reference image. At this time, the reference image may have a range of (-L_offset, -U_offset) to (pic_width-1+R_offset, pic_height-1+D_offset). Here, L_offset, R_offset, U_offset, and D_offset may refer to the lengths expanded in the reference image in the left, right, upper, and lower directions. The offsets may have the same value or different values ​​from each other. The length expanded according to the reference image may be explicitly transmitted or implicitly determined.

[0385] For example, one of the lengths supported by the block partition unit may be supported, such as 2, 4, 8, etc. Alternatively, the extended length may be determined based on a minimum coding block, a maximum coding block, a minimum prediction block, a maximum prediction block, etc. Alternatively, the extended length may be determined in units of blocks, bricks, etc.

[0386] For example, the maximum prediction block may be enlarged according to its length, in which case prediction blocks of all blocks may be obtained within the enlarged range. If the area indicated by the motion vector of the current block is out of the range of the enlarged reference image even after the reference image is enlarged, the prediction pixels of the portion corresponding to the out-of-range may be obtained by filling in the horizontal direction, the vertical direction, the diagonal direction, etc.

[0387] Reference Fig.13 , A to D may be equivalent to the case where a reference image is enlarged by one or more surface units. A and B are examples of enlarging a reference image (or a sub-reference image) by combining surfaces with continuity. C and D are examples of enlarging a reference image individually by surface units. The data processing of the enlarged area is the same / similar to that of the above-mentioned embodiment. In the above-mentioned embodiment, there is a range regarding the reference image, but in this example, the reference image is divided into two or more parts to enlarge each area (called a sub-reference image), so an overlapping range will be generated.

[0388] In the examples of A and B, the enlarged sub-reference image located on the upper side has a range of (-L_offset, -U_offset) to (pic_width-1+R_offset, pic_height / 2-1+D_offset), and the enlarged sub-reference image located on the lower side has a range of (-L_offset, pic_height / 2-U_offset) to (pic_width-1+R_offset, pic_height-1+D_offset). It is assumed that in each sub-reference image, the enlarged left, right, top, and bottom lengths are the same. However, the lengths of the enlargement in the left direction, right direction, top direction, and bottom direction may be set exactly the same as or different from the enlargement unit. Furthermore, the enlargement length may be set individually according to the enlargement unit.

[0389] At this time, the coordinates of the range expanded downward by the upper sub-reference image are the same as the coordinates inside the lower sub-reference image, but the corresponding area of ​​the upper sub-reference image has associated data after expansion, while the lower sub-reference image has the original data before expansion.

[0390] In general, the reference image may have a range of (-L_offset, -U_offset) to (pic_width-1+R_offset, pic_height-1+D_offset). Normally, one pixel value should be assigned to each position, but since the reference image is expanded by dividing it into two or more regions in consideration of the characteristics of the entire video, more than two pixel values ​​may be assigned to a part of the range. In this case, it is required to determine which pixel value is set as the predicted pixel value at the position where more than two pixel values ​​are assigned.

[0391] Reference Fig.14 In case E, after confirming which sub-reference image contains more areas, the prediction value can be obtained using the data of the sub-reference image containing more areas.

[0392] In the case of F, when the sub-reference image including the current block includes a preset number of pixels or more pixels, or includes a specific width, the data of the corresponding sub-reference image can be used. Otherwise, the data of the sub-reference image of the area including more areas can be used for processing like E.

[0393] G is the case where the block is evenly contained in two sub-reference images. In this case, priority can be given to the sub-reference image containing the current block to use the data of the corresponding sub-reference image. This example can include the detailed conditions of F and E.

[0394] In summary, when the area indicated by the motion vector of the current block is a boundary between sub-reference images, the data of which sub-reference image is used can be determined based on the included area, whether it is the same sub-reference image as the current block, etc. The data of which sub-reference image is used to perform inter-frame prediction can be determined based on all or part of the above conditions through encoding / decoding settings.

[0395] In the above example, the reference image is divided into two parts to expand each area, but in this example, a case where more sub-reference images are generated in surface units is described.

[0396] exist Fig.15 In , it is assumed that each surface unit is a sub-reference image. Similarly, it is necessary to determine which pixel value is set as the predicted pixel value at a position where more than two pixel values ​​are assigned.

[0397] Reference Fig.15 In the case of H, data of a sub-reference image containing more areas may be used, or data of a sub-reference image containing the current block may be used.

[0398] For example, if the sub-reference image including the current block is included in a preset number of pixels or a defined width or more, the data of the corresponding sub-reference image can be used. Alternatively, in the case of using the data of the sub-reference image including more areas, the data of the corresponding sub-reference image can be used. However, if the included areas are the same, the data of the sub-reference image including the current block can also be used restrictively.

[0399] In the case of I, data of a sub-reference image including a larger area can be used. In this example, the sub-reference image is supported in surface units, but the surface setting has a characteristic that the upper 3×1 area and the lower 3×1 area are discontinuous, so the sub-reference image on the side with the characteristic of continuity can be given priority over the sub-reference image on the side with the characteristic of discontinuity.

[0400] In the case of J, sub-reference image data covering a larger area can be used. However, although this example is a case where the area containing the sub-reference image is uniform, the sub-reference image containing the current block (on the right side in 3×2) is not an overlapping object like I. In addition, the corresponding sub-reference image (upper left, center in 3×2) belongs to the same 3×1 as the sub-reference image containing the current block. In this case, priority can be given to the sub-reference image that is closer to the sub-reference image containing the current block.

[0401] In summary, when the area indicated by the motion vector of the current block is a boundary between sub-reference images, the sub-reference image data to be used can be determined based on the included area, the use priority of the sub-reference image, etc.

[0402] On the other hand, refer to Figures 7 to 10 A method for performing inter-frame prediction based on affine motion modeling is described. The method can be applied to encoding / decoding of 360 videos in the same / similar manner, and a detailed description is omitted.

[0403] The method of the present invention can be implemented in the form of program commands that can be executed by various computer devices and recorded in a medium that can be read by a computer. The medium that can be read by a computer can contain program commands, data files, data structures, etc. alone or in combination. The program commands recorded in the medium that can be read by a computer can be program commands specially designed and constructed for the present invention, or program commands that are well known and used by computer software practitioners.

[0404] Examples of computer-readable media include hardware devices such as read-only memory (ROM), random access memory (RAM), and flash memory that are specifically configured to store and execute program commands. Examples of program commands include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be run by a computer using an interpreter. The hardware device can be configured to operate through at least one software module in order to execute the actions of the present invention, and vice versa.

[0405] In addition, in the above-mentioned method or apparatus, its configuration or function can be realized in a form of combining all or part of them, or can be realized in a form of separation.

[0406] Although the preferred embodiments of the present invention have been described above, it should be understood that those skilled in the art may make various modifications and changes to the present invention without departing from the spirit and scope of the present invention as described in the appended claims.

[0407] Industrial Availability

[0408] The present invention can be used to encode / decode video signals.

Claims

1. A method for decoding a video signal, the method comprising: Determine a merge candidate list for the current block, wherein the merge candidate list includes at least one of a spatial merge candidate or a temporal merge candidate; Determine a combined merge candidate based on the first merge candidate and the second merge candidate in the merge candidate list; Adding the combined merge candidate to the merge candidate list, wherein the first merge candidate is a merge candidate with an index equal to 0 in the merge candidate list, and the first merge candidate corresponds to unidirectional prediction, and the second merge candidate is a merge candidate with an index equal to 1 in the merge candidate list, and the second merge candidate corresponds to unidirectional prediction; Determining motion information of the current block according to the merge candidate list of the current block, wherein the motion information includes at least one of a prediction direction flag, a reference image index, or a motion vector; and Determine an inter-frame prediction value of the current block according to motion information of the current block; The reference image list indicated by the prediction direction flag corresponding to the first merge candidate and the reference image list indicated by the prediction direction flag corresponding to the second merge candidate are different or the same.

2. The method according to claim 1, further comprising: When the prediction direction flag corresponding to the first merge candidate indicates reference picture list 0, and the prediction direction flag corresponding to the second merge candidate indicates reference picture list 1, Determining, according to the reference image index of the reference image list 0 corresponding to the first merge candidate, the reference image index of the reference image list 0 corresponding to the combined merge candidate; Determine that the prediction direction flag of the reference picture list 0 corresponding to the combined merge candidate is equal to 1; as well as According to the motion vector of the reference picture list 0 corresponding to the first merge candidate, the motion vector of the reference picture list 0 corresponding to the combined merge candidate is determined.

3. The method according to claim 1 or 2, further comprising: When the prediction direction flag corresponding to the first merge candidate indicates reference picture list 0, and the prediction direction flag corresponding to the second merge candidate indicates reference picture list 1, determining, according to the reference image index of the reference image list 1 corresponding to the second merge candidate, a reference image index of the reference image list 1 corresponding to the combined merge candidate; Determine that the prediction direction flag of the reference picture list 1 corresponding to the combined merge candidate is equal to 1; as well as According to the motion vector of the reference picture list 1 corresponding to the second merge candidate, the motion vector of the reference picture list 1 corresponding to the combined merge candidate is determined.

4. The method according to claim 1, further comprising: When the prediction direction flag corresponding to the first merge candidate indicates reference picture list 1, and the prediction direction flag corresponding to the second merge candidate indicates reference picture list 0, Determining, according to the reference image index of the reference image list 0 corresponding to the second merge candidate, the reference image index of the reference image list 0 corresponding to the combined merge candidate; Determine that the prediction direction flag of the reference picture list 0 corresponding to the combined merge candidate is equal to 1; as well as According to the motion vector of the reference picture list 0 corresponding to the second merge candidate, the motion vector of the reference picture list 0 corresponding to the combined merge candidate is determined.

5. The method according to claim 1 or 4, further comprising: When the prediction direction flag corresponding to the first merge candidate indicates reference picture list 1, and the prediction direction flag corresponding to the second merge candidate indicates reference picture list 0, Determining, according to the reference image index of the reference image list 1 corresponding to the first merge candidate, the reference image index of the reference image list 1 corresponding to the combined merge candidate; Determine that the prediction direction flag of the reference picture list 1 corresponding to the combined merge candidate is equal to 1; as well as According to the motion vector of the reference picture list 1 corresponding to the first merge candidate, the motion vector of the reference picture list 1 corresponding to the combined merge candidate is determined.

6. The method according to claim 1, further comprising: When the prediction direction flag corresponding to the first merge candidate and the prediction direction flag corresponding to the second merge candidate indicate reference picture list 0, Determining, according to the reference image index of the reference image list 0 corresponding to the first merge candidate, the reference image index of the reference image list 0 corresponding to the combined merge candidate; Determine that the prediction direction flag of the reference picture list 0 corresponding to the combined merge candidate is equal to 1; as well as The motion vector of the reference image list 0 corresponding to the combined merge candidate is determined according to a weighted average of the motion vector of the reference image list 0 corresponding to the first merge candidate and the motion vector of the reference image list 0 corresponding to the second merge candidate.

7. The method according to claim 1, further comprising: When the prediction direction flag corresponding to the first merge candidate and the prediction direction flag corresponding to the second merge candidate indicate reference picture list 1, Determining, according to the reference image index of the reference image list 1 corresponding to the first merge candidate, the reference image index of the reference image list 1 corresponding to the combined merge candidate; Determine that the prediction direction flag of the reference picture list 1 corresponding to the combined merge candidate is equal to 1; as well as The motion vector of the reference image list 1 corresponding to the combined merge candidate is determined based on a weighted average of the motion vector of the reference image list 1 corresponding to the first merge candidate and the motion vector of the reference image list 1 corresponding to the second merge candidate.

8. The method according to claim 6 or 7, wherein: The weight of the weighted average is [1:1]; Alternatively, the weight of the motion vector corresponding to the first merge candidate is equal to the weight of the motion vector corresponding to the second merge candidate; Alternatively, a ratio of a weight of a motion vector corresponding to the first merge candidate to a weight of a motion vector corresponding to the second merge candidate is [1:1].

9. A method for encoding a video signal, the method comprising: Determine a merge candidate list for the current block, wherein the merge candidate list includes at least one of a spatial merge candidate or a temporal merge candidate; Determine a combined merge candidate based on the first merge candidate and the second merge candidate in the merge candidate list; Adding the combined merge candidate to the merge candidate list, wherein the first merge candidate is a merge candidate with an index equal to 0 in the merge candidate list, the first merge candidate corresponds to a unidirectional prediction, and the second merge candidate is a merge candidate with an index equal to 1 in the merge candidate list, the second merge candidate corresponds to the unidirectional prediction; Determining motion information of the current block according to the merge candidate list of the current block, wherein the motion information includes at least one of a prediction direction flag, a reference image index, or a motion vector; and Determine an inter-frame prediction value of the current block according to motion information of the current block; The reference image list indicated by the prediction direction flag corresponding to the first merge candidate and the reference image list indicated by the prediction direction flag corresponding to the second merge candidate are different or the same.

10. The method according to claim 9, further comprising: When the prediction direction flag corresponding to the first merge candidate indicates reference picture list 0, and the prediction direction flag corresponding to the second merge candidate indicates reference picture list 1, Determining, according to the reference image index of the reference image list 0 corresponding to the first merge candidate, the reference image index of the reference image list 0 corresponding to the combined merge candidate; Determine that the prediction direction flag of the reference picture list 0 corresponding to the combined merge candidate is equal to 1; as well as According to the motion vector of the reference picture list 0 corresponding to the first merge candidate, the motion vector of the reference picture list 0 corresponding to the combined merge candidate is determined.

11. The method according to claim 9 or 10, further comprising: When the prediction direction flag corresponding to the first merge candidate indicates reference picture list 0, and the prediction direction flag corresponding to the second merge candidate indicates reference picture list 1, determining, according to the reference image index of the reference image list 1 corresponding to the second merge candidate, a reference image index of the reference image list 1 corresponding to the combined merge candidate; Determine that the prediction direction flag of the reference picture list 1 corresponding to the combined merge candidate is equal to 1; as well as According to the motion vector of the reference picture list 1 corresponding to the second merge candidate, the motion vector of the reference picture list 1 corresponding to the combined merge candidate is determined.

12. The method according to claim 9, further comprising: When the prediction direction flag corresponding to the first merge candidate indicates reference picture list 1, and the prediction direction flag corresponding to the second merge candidate indicates reference picture list 0, Determining, according to the reference image index of the reference image list 0 corresponding to the second merge candidate, the reference image index of the reference image list 0 corresponding to the combined merge candidate; Determine that the prediction direction flag of the reference picture list 0 corresponding to the combined merge candidate is equal to 1; as well as According to the motion vector of the reference picture list 0 corresponding to the second merge candidate, the motion vector of the reference picture list 0 corresponding to the combined merge candidate is determined.

13. The method according to claim 9 or 12, further comprising: When the prediction direction flag corresponding to the first merge candidate indicates reference picture list 1, and the prediction direction flag corresponding to the second merge candidate indicates reference picture list 0, Determining, according to the reference image index of the reference image list 1 corresponding to the first merge candidate, the reference image index of the reference image list 1 corresponding to the combined merge candidate; Determine that the prediction direction flag of the reference picture list 1 corresponding to the combined merge candidate is equal to 1; as well as According to the motion vector of the reference picture list 1 corresponding to the first merge candidate, the motion vector of the reference picture list 1 corresponding to the combined merge candidate is determined.

14. The method according to claim 9, further comprising: When the prediction direction flag corresponding to the first merge candidate and the prediction direction flag corresponding to the second merge candidate indicate reference picture list 0, Determining, according to the reference image index of the reference image list 0 corresponding to the first merge candidate, the reference image index of the reference image list 0 corresponding to the combined merge candidate; Determine that the prediction direction flag of the reference picture list 0 corresponding to the combined merge candidate is equal to 1; as well as The motion vector of the reference image list 0 corresponding to the combined merge candidate is determined according to a weighted average of the motion vector of the reference image list 0 corresponding to the first merge candidate and the motion vector of the reference image list 0 corresponding to the second merge candidate.

15. The method according to claim 9, further comprising: When the prediction direction flag corresponding to the first merge candidate and the prediction direction flag corresponding to the second merge candidate indicate reference picture list 1, Determining, according to the reference image index of the reference image list 1 corresponding to the first merge candidate, the reference image index of the reference image list 1 corresponding to the combined merge candidate; Determine that the prediction direction flag of the reference picture list 1 corresponding to the combined merge candidate is equal to 1; as well as The motion vector of the reference image list 1 corresponding to the combined merge candidate is determined based on a weighted average of the motion vector of the reference image list 1 corresponding to the first merge candidate and the motion vector of the reference image list 1 corresponding to the second merge candidate.

16. The method according to claim 14 or 15, wherein: The weight of the weighted average is [1:1]; Alternatively, the weight of the motion vector corresponding to the first merge candidate is equal to the weight of the motion vector corresponding to the second merge candidate; Alternatively, a ratio of a weight of a motion vector corresponding to the first merge candidate to a weight of a motion vector corresponding to the second merge candidate is [1:1].

17. A video signal decoding device, comprising: A processor configured to execute the method according to any one of claims 1 to 8.

18. A video signal encoding device, comprising: A processor configured to execute the method according to any one of claims 9 to 16.

19. A non-transitory computer-readable storage medium comprising a bitstream, wherein: The bitstream is generated according to the method according to any one of claims 9 to 16.

Images